Demographic and Clinical Differences Between Applied Definitions of Medical Complexity Free

We performed a retrospective, cross-sectional study using the Pediatric Health Information System (PHIS), an administrative database of US children’s hospitals coordinated by the Children’s Hospital Association (Lenexa, Kansas). Data are deidentified within PHIS but can be tracked across encounters for individual patients within each hospital. This study incorporated data from 47 sites and was considered nonhuman research by the institutional review board.

We included the first admission for all hospitalized patients under 21 years old between January 1, 2017 and December 31, 2019. We included patients with both inpatient and observation statuses because of wide variation in how these are assigned within pediatric hospitals.³⁷ 

We classified each patient using medical complexity criteria. These are described in prior literature,^17–21,24  and are summarized here. Operational definitions are provided in Supplemental Table 3.

The CCC definition was based on the most recent classification.^18,38  CCC assigns 1 of 10 mutually exclusive diagnosis categories to relevant International Classification of Disease, 10th revision (ICD-10) codes. Each code may also be associated with 1 or both categories of technology dependence or transplant.

We characterized PMCA per the most recent classification.²¹  PMCA assigns each ICD-10 code to 1 of 18 mutually exclusive diagnosis categories. As we used only hospital-based data, we used the “least conservative” PMCA definition.¹⁹  Children with nonprogressive conditions in a single diagnosis category are labeled as having noncomplex chronic disease; children whose conditions are progressive, involve multiple body systems, or who have malignancies are labeled as having complex chronic disease.

Identification of CCI required both prolonged ICU admission (either within the index hospitalization or the year prior) and ongoing dependence on medical technology.²⁴  Dependence on medical technology was identified using the CCC technology-dependence definition since the original authors provided no qualifying list of technologies.¹⁸ 

We acquired data from the index encounter and any admissions in the 3 years prior (because of the recommended 3-year look-back period for PMCA). We acquired demographic, clinical, outcome, and cost characteristics of the index admission, and data from patient encounters (readmissions, ICU admissions, in-hospital mortality) within the year following the index admission. Demographic data included age, sex, race or ethnicity, payor type, census region, and median household income by US postal code. Race or ethnicity were reclassified to non-Hispanic White, non-Hispanic Black, Hispanic, and other (including multiple categories or missing). Insurance coverage was classified as public, private, and other. Clinical data included interventions, medications, clinical outcomes of the index admission (ICU admission, hospital and ICU length of stay [LOS], in-hospital mortality, and standardized costs), and posthospitalization metrics (1-year hospital readmission, 1-year ICU readmission, and 1-year in-hospital mortality). Standardized costs are provided by PHIS to facilitate comparison of resource utilization across hospitals but are not a measure of actual hospital costs. Procedural variables were selected for use because of their clinical relevance, availability within the PHIS database, and use in similar projects. Readmission to the ICU and the hospital were selected as a proxy for long-term medical utilization. Costs (both on the index admission and subsequently) serve as an additional proxy for utilization. Queried variable codes are in Supplemental Table 4. Primary diagnosis codes from index visits were classified using the Clinical Classifications Software from the Agency for Healthcare Research and Quality.³⁹ 

We classified patients using each definition. To focus on children with chronic conditions across multiple body systems, we subsequently narrowed our focus to multisystem complexity. We a priori established the following criteria as indicators of children with multisystem complexity: the presence of ≥2 CCC groupings (CCC); complex chronic disease designation (PMCA); and all identified CCI patients. We calculated the Fleiss’ κ statistic for multirater agreement, classifying a result of ≥0.75 as excellent, 0.40 to 0.74 as fair, and <0.40 as poor.⁴⁰  We additionally calculated the Cohen’s κ statistic for 2-rater agreement between each pair of definitions, classifying a result of ≥0.8 as almost perfect agreement, 0.61 to 0.79 as substantial, 0.41 to 0.60 as moderate, 0.21 to 0.40 as fair, and 0 to 0.20 as slight agreement.⁴¹  We constructed a Euler diagram to visualize the extent of overlap between criteria. To assess whether differences in patient identification were caused by definitional look-back periods or the number of diagnoses required for identification, we performed several exploratory analyses. In our first exploratory analysis, we constructed additional Euler diagrams using different thresholds for the number of required CCC groupings, keeping criteria for CCI and PMCA multisystem complexity constant. In a second set of exploratory analyses, we evaluated how the identification of patients by CCC and PMCA changed if the look-back period was adjusted from the index encounter to up to 3 years of prior data. We hypothesized that increasing the required number of CCC diagnoses would decrease the number of identified patients and that identification based on longer look-back periods would increase the number of identified patients.

We presented descriptive data (as counts and percentages or medians and interquartile ranges [IQR]) for each complexity grouping and for all admissions. We did not perform tests of significance between definitions of multisystem complexity because of the overlap between patient populations. Analyses were performed using the eulerr (version 6.1.0) package⁴²  in R version 4.0.3, (R Foundation for Statistical Computing, Vienna, Austria) and Stata SE (version 16.1, StataCorp, College Station, Texas).

From January 2017 to December 2019, we identified 1 795  094 patients. We identified 478 077 patients that had at least 1 CCC grouping. The most common CCC groupings were cardiac (121 578, 25.4% of CCC patients), neurologic (111 220, 23.3%), and gastrointestinal (97 003, 20.3%). In total, 110 620 (23.1%) CCC patients were dependent on medical technology. Approximately one-third of children with medical complexity as defined by CCC had more than 1 qualifying grouping (n = 157 725, 33.0%) and therefore were defined as having multisystem complexity. We identified 857 425 patients as having chronic disease by PMCA. The most common categories by PMCA were pulmonary (295 235, 34.4% of PMCA patients), mental health (204 757, 23.9%), and neurologic (186 397, 21.7%). We identified 402 811 children by PMCA as having complex chronic disease and therefore met criteria for multisystem complexity.

Children with multisystem complexity identified by PMCA were more frequently admitted to the ICU during the index admission, more frequently required interventions, had longer LOS, and had higher mortality compared with those with noncomplex chronic disease (Supplemental Table 5). Children with progressively more CCC diagnoses demonstrated a similar pattern (Supplemental Table 6).

The median age at admission was 3.8 years (IQR 0.2–11.7 years) and 46.6% were female. Overall, 345 847 (19.3%) had an ICU stay during their index admission. In-hospital mortality was 0.6%. There were 421 196 patients (23.5%) who met at least 1 definition of multisystem complexity (ie, ≥2 CCC groupings, complex chronic condition per PMCA, and/or CCI). Children with PMCA multisystem complexity had a median age at admission of 8.5 years (IQR 2.1–14.4 years), whereas those identified by CCC had a median age of 4.3 years (0.2–12.3), and CCI 0.1 years (0.0–1.1). Patients identified by CCI had a higher proportion of children classified as non-Hispanic Black (18.9%) than other definitions, whereas those identified by PMCA had a higher proportion categorized as non-Hispanic White (46.2%). Children with CCI were most frequently covered by public insurance (63.2%), whereas those identified by PMCA were the least likely (55.9%). Eight of the 10 most common reasons for admission in children with CCI were related to newborn status or congenital anomalies, compared with 5 in children with CCC, and 3 in children with PMCA. Epilepsy or convulsions was the most common reason for admission in children identified by PMCA and the third most common in CCC; it was not among the 10 most common admission diagnoses in children with CCI.

Children with multisystem complexity identified by CCI were most frequently admitted to the ICU during their index admission (95.9%), likely because the exposure and outcome are highly correlated at a definitional level. This was followed by children identified by CCC (51.4%) and PMCA (30.8%). Children identified by CCI also more frequently received mechanical ventilation, cardiopulmonary resuscitation, extracorporeal membrane oxygenation, renal replacement, and transfusions. They were also more frequently exposed to inhaled respiratory treatments, steroids, antimicrobial agents, and vasoactives or inotropes. Finally, patients with CCI had a longer median ICU and hospital LOS and had the highest proportion of in-hospital mortality (7.9%). In-hospital mortality was 2.0% in patients with multisystem complexity identified by PMCA and 4.7% in patients with CCC.

Median standardized charges were highest for children identified by CCI ($254 000 [$132 000–$479 000]), and lowest in children identified by PMCA ($11 500 [$5180–$34 900]). Children identified using the CCI definition were more frequently readmitted to the hospital and ICU within a year of their index admission. One-year in-hospital mortality was highest in children with CCI.

The PMCA definition identified the largest number of children with multisystem complexity (n = 402 811, 95.6% of children with any multisystem complexity, 22.4% of all patients); followed by CCC (n = 157 725, 37.5%, 8.8%) and CCI (n = 17 980, 4.3%, 1.0%). Most patients (n = 278 624; 66.2%) met only 1 definition, with only 14 748 (3.5%) meeting all 3 (Fig 1). The Fleiss’ κ was 0.21, indicating poor agreement. Cohen’s κ for individual pairings were 0.43 for PMCA-CCC (moderate agreement), 0.17 for CCC-CCI (slight agreement), and 0.06 for PMCA-CCI (slight agreement). As the CCC definition became more stringent (ie, more CCC groupings required), the overall number of patients identified by this measure decreased, with increasing overlap with the PMCA definition (Supplemental Fig 2). Using 3 years of data to identify patients with CCC did not substantially change the number of identified patients or degree of overlap between definitions (Supplemental Fig 3). Finally, using data from only the index encounter to identify patients with PMCA did not substantially change the number of identified patients or degree of overlap between definitions (Supplemental Fig 4).

In this multicenter cohort, we compared 3 definitions for pediatric multisystem complexity. Nearly one-quarter of hospitalized children met at least 1 definition of multisystem complexity. Our data suggest that each patient population demonstrates unique demographic, clinical, and resource use patterns. As these definitions were created for different purposes, it is unsurprising that differences exist between the groups. However, these differences highlight the lack of consensus regarding what constitutes multisystem complexity. To our knowledge, this investigation represents the first direct comparison showing the degree to which these definitions identify different populations. Our findings demonstrate that researchers and others should not use these, or other, definitions of pediatric medical complexity interchangeably or compare studies that use different definitions. Investigators and readers alike must understand the underlying assumptions, limitations, and population characteristics of each definition.

We identified differences between groupings with respect to their underlying clinical and demographic characteristics. Resource utilization and mortality were generally lowest in PMCA, which identified the largest cohort of patients. The CCI classification identifies the smallest cohort of patients but with the most intense hospitalization needs, as characterized by need for ICU admission and therapies. The young age of the patients with CCI may explain why a larger proportion of children identified by this definition is covered by public insurance, which is available in the United States for infants and children with complex medical conditions.⁴³  In addition, the groups varied with respect to their inclusion by race and ethnicity. In administrative databases, race and/or ethnicity labels are often assumed and not self-reported.⁴⁴  These labels are inadequate to reflect the social construct of race, which is associated with significant health disparities. Racial or ethnic differences have an impact on access to, provision of, and outcome of care for children.⁴⁵  Differing definitions of medical complexity may therefore over- or under-identify vulnerable subgroups. For example, the disparate impact of prematurity on Black infants may explain the higher number of Black children identified by CCI.⁴⁶ 

Our comparison of definitions of multisystem complexity demonstrated poor agreement between 3 definitions, and at most moderate agreement between any 2 definitions. The concordance between definitions remained low even when considering PMCA and CCC alone, as CCI represents an outlier in this case. Our findings identify differences in applied definitions of pediatric patients with multisystem complexity and provide guidance on their utilization for epidemiologic and outcomes-based research.

The 3 cohorts demonstrate similarities in some domains. All had higher rates of public insurance relative to the overall cohort, and all demonstrated a higher need for ICU admission and increased exposures to therapies, medication, and hospitalization relative to the overall population. Our results are consistent with prior work that initially defined^17–21,24  and later evaluated^{8,27,28,47,48}  pediatric medical complexity during hospital-based encounters. A population-based sample in Canada identified that patients with CCC accounted for nearly one-third of child health spending, and that these children also had higher rates of readmission and home care needs.²⁸  Similarly, an evaluation of an ICU-based dataset noted that children with a modified CCC definition had a higher mortality and longer LOS compared with patients in the ICU without such conditions.⁴⁸  An evaluation of the PMCA algorithm using the PHIS database reported that approximately half of patients admitted to the ICU had a chronic complex condition, and that such patients required approximately three-quarters of resources used.⁸ 

The differences between these definitions of multisystem complexity and their identified populations carry implications for research involving children with medical complexity. Researchers should not default to using a specific definition without considering potential underlying population differences, including the qualifying diagnosis categories. Understanding the implications for choosing 1 definition in terms of baseline procedural or medication exposure is important for researchers designing studies and the pediatric community in interpreting results. The inclusion of mental health disorders in PMCA is unique among these definitions, which is an important distinction, particularly in light of growing attention to such conditions in research and clinical care. CCI selects for a high-acuity population with high hospital resource use; as such, it may be more representative of significant complexity among tertiary-care ICU patients when compared with CCC and PMCA given the baseline presence of complexity in these patients.^4,8  Conversely, CCI is likely overly stringent for routine inpatient, emergency department, or outpatient use and would likely not be applicable in these contexts. The operationalization of PMCA and CCI requires longitudinal data, which may not be available in all datasets. Of note, changing the look-back period for either PMCA or CCC identification did not substantially alter the number of identified patients or its overlap with other definitions. These findings likely suggest that using only encounter-level data may be sufficient for the identification of multisystem complexity in inpatient encounters, perhaps related to increased coding density. Additionally, the patterns of identified chronic conditions differed between PMCA and CCC. Whereas the largest disease groupings of PMCA patients had pulmonary, mental health, and neurologic conditions, CCC more frequently identified patients with cardiac, neurologic, and gastrointestinal disorders. Finally, further work is needed to assess the generalizability of these findings in other contexts, such as in community pediatric hospitalizations, or in the outpatient setting.

It is possible that the operational definitions we chose a priori for each label of multisystem complexity are not intended to identify equivalent populations. For example, the PMCA definition includes children with progressive diseases or malignancies, which may not have explicit multisystem involvement and the original CCI definition requires the presence of “multiorgan dysfunction,” although does not provide a means to operationalize that concept.^21,24  Findings from an administrative dataset cannot truly represent the daily experiences of children with complex health care needs, as enumeration and classification of diagnoses do not encapsulate the lived experience of children and families. For example, none of the definitions explicitly measure physical care needs or functional capabilities, and our study includes data only about inpatient hospitalization.

Our findings are subject to limitations. First, though PHIS represents a high-quality and robust data source, it nonetheless lacks clinical granularity and may be subject to errors in abstraction or coding.⁴⁹  This study was conducted at US children’s hospitals, and the results may not be generalizable to other hospital-based settings as children’s hospitals care for a higher proportion of patients with medical complexity.⁵⁰  Similarly, our findings may not be generalizable to nonhospital based encounters, such as ambulatory care visits. For example, PMCA includes a separate category for mental health disorders, and therefore researchers studying ambulatory visits may identify a relatively larger number of patients through this mechanism than if using CCC, which does not have a category for mental health conditions. For the CCI definition, we chose to assess ongoing technology dependence using the CCC technology dependence flag, although there is limited agreement about this criterion.^51,52  Despite these limitations, we believe that a large dataset of hospitalized children in the US provides a comprehensive understanding of how these definitions identify children with multisystem complexity.

This large, multicenter analysis of hospital admissions identified varied populations of children with complex medical needs based on diagnoses and clinical variables. This is the first direct comparison of the demographic and clinical characteristics of children identified by each definition. Although each identified a population of children with medical needs greater than the overall sample, their lack of agreement should result in caution in drawing conclusions across studies using different definitions of medical complexity.