Patterns of Organ Dysfunction in Critically Ill Children Based on PODIUM Criteria

We conducted a retrospective observational cohort study of critically ill children admitted to 2 academic medical centers. We included all children aged 0 to 17 years admitted to the PICU between 2010 and 2018 (Ann & Robert H. Lurie Children’s Hospital of Chicago) or 2010 and 2016 (University of Chicago Comer Children’s Hospital). Patients were excluded if they had congenital heart disease or were recovering from cardiac surgery. Data were extracted from the 2 institutional databases using structured queries and underwent quality checks for conformity, completeness, and plausibility.¹⁸ 

Only the information pertaining to the first PICU encounter and up to 28 days, discharge, or death (whichever came first) in each hospitalization was included, as well as whether the patient survived the hospitalization or not. Each admission was treated independently. The primary outcome was in-hospital mortality. Oncologic comorbidity was determined using Feudtner’s complex chronic condition classification.¹⁹ 

The goal of this study was to test the face validity (ie, the degree to which an assessment appears effective in terms of its stated aims), prognostic performance, and generalizability of the PODIUM criteria using EHR data. To do this, only commonly recorded structured data elements from the EHRs were included in the analysis. Structured EHR data include laboratory test results, medications administered, diagnostic codes, and clinician-charted observations and interventions (eg, vital signs, intake/output, ventilator settings, etc).¹⁶  Unstructured data contained in EHRs and other clinical information systems (eg, clinician notes, diagnostic reports, imaging studies) may also contain important information related to organ dysfunction. However, extracting information from these data sources in a generalizable way can be challenging and impractical, and hence we excluded them from the analysis. Additionally, structured data elements that were recorded in <5% of patients were excluded to maximize the face validity and generalizability of the results using EHR data.

We assessed organ dysfunction criteria for each 24-hour period starting at PICU admission and up to 28 days, discharge, or death (whichever came first). When >1 criterion for a single organ dysfunction was recorded in a 24-hour period, the worst value was used. If no values were recorded in a 24-hour period, they were assumed to be normal (ie, not meeting criteria for organ dysfunction). The degree of severity of individual criteria (which was only graded in the respiratory, cardiovascular, and hematologic systems) was not considered for this analysis. Specific criteria used to determine organ dysfunction in the EHR data sets are detailed in the Supplemental Information.

We assessed the prognostic performance of the PODIUM organ dysfunction criteria using the number of concurrent PODIUM organ dysfunctions as predictors and in-hospital mortality as the outcome and using the area under the receiver operating characteristic curve to measure discrimination. We assessed the prognostic performance of the number of concurrent PODIUM organ dysfunctions on day 1 of PICU admission, as well as the maximum number of concurrent PODIUM organ dysfunctions met by day 3, day 7, and day 28 of admission. We compared the performance of these measures with the maximum number of concurrent organ dysfunctions met by the IPSCC organ dysfunction criteria at the same time intervals.⁶  Specific criteria used to determine organ dysfunction by the IPSCC criteria are detailed in the Supplemental Information.

We examined the prevalence of MOD on day 1 of PICU stay and the development of new or progressive MOD (NPMOD) and persistent MOD by day 7. Based on the consensus PODIUM criteria, MOD syndrome is considered a subset of MOD in which there is a shared common cause for the organ dysfunctions. MOD, on the other hand, is defined as the presence of ≥2 concurrent organ dysfunctions, regardless of cause. New MOD was defined as MOD that developed between days 2 and 7 in patients who had single or no organ dysfunction on day 1. Progressive MOD was defined as the development of one or more additional organ dysfunctions or death between days 2 and 7 in patients who already met MOD criteria on day 1. NPMOD was defined as patients who met either new or progressive MOD criteria.⁵  Persistent MOD was defined as the presence of MOD on day 7 among patients who already met MOD criteria on day 1.

Data were analyzed using R version 4.0 (R Foundation for Statistical Computing, Vienna, Austria). Categorical variables were compared using the χ² test and continuous variables using the nonparametric Kruskal-Wallis test. In-hospital mortality discrimination was assessed using the area under the curve (AUC) and compared using the DeLong method.

The institutional review boards at Ann & Robert H. Lurie Children’s Hospital of Chicago and The University of Chicago approved this study with a waiver of informed consent. The reporting of this observational cohort study was performed using the Strengthening the Reporting of Observational Studies in Epidemiology checklist for cohort studies.²⁰ 

Overall, 22 427 PICU admissions by 15 447 unique patients met inclusion criteria during the study period. The median age on admission was 4.6 years (interquartile range 1.4–11.6), and in-hospital mortality was 2.3%. Table 1 presents the demographic and clinical characteristics of patents who died and those who survived the admission.

The cumulative incidence of 7 of the 9 individual PODIUM organ dysfunctions by day 28 ranged from 15% to 30%, with an associated in-hospital mortality of 6% to 10% (Fig 1). Liver and coagulation dysfunction were more rare (cumulative incidence 3% and 5%, respectively) and associated with higher in-hospital mortality (30% and 16%, respectively).

When stratified by age group, the cumulative incidence and associated in-hospital mortality were similar, with some exceptions (Supplemental Fig 1). Infants <1 year of age had higher cumulative incidence of respiratory and cardiovascular dysfunction, and school-aged children >6 years had higher cumulative incidence of renal, endocrine, and immune dysfunction. When stratified by hospital, the cumulative incidence and associated mortality were comparable across all organ dysfunctions (Supplemental Fig 2).

The number of concurrent PODIUM organ dysfunction criteria met was associated with in-hospital mortality in a dose-dependent fashion (Fig 2). This association was stronger when the organ dysfunction criteria were met concurrently on the same day as opposed to cumulatively over the course of the admission. For example, meeting criteria for 6 organ dysfunctions by day 28 was associated with an in-hospital mortality of 13%, whereas meeting 6 organ dysfunctions on the same day was associated with an in-hospital mortality of 29% (Supplemental Fig 3).

The number of concurrent organ dysfunctions met by the IPSCC criteria had a similar dose-dependent relationship with in-hospital mortality (Fig 2).

Among patients who met any organ dysfunction criteria, 80% to 90% met criteria by day 3 and >90% met criteria by day 7 (Supplemental Fig 4). In general, endocrine and immunologic dysfunction criteria were met earlier and accumulated faster (∼80% on day 1 and 90% by day 3). Renal and coagulation dysfunction criteria were met later and accumulated slower (∼55%–60% on day 1 and 90% by day 6).

The number of concurrent PODIUM organ dysfunctions had good to excellent discrimination of in-hospital mortality at different intervals during the admission (Table 2). The AUCs ranged from 0.87 for the number of criteria met on day 1 to 0.93 for the maximum number of criteria met by day 28. In general, the performance of the PODIUM criteria was better than the IPSCC criteria (Table 2).

Based on the PODIUM criteria, 30.2% of patients had MOD on day 1. These patients had an in-hospital mortality rate of 6.4% and an odds ratio (OR) of 13.6 (95% confidence interval [CI] 10.7–17.4) for mortality compared with those who did not meet MOD criteria on day 1 (Supplemental Table 1). MOD on day 1 by PODIUM criteria had a sensitivity of 85% and a specificity of 71% for in-hospital mortality. When using the IPSCC criteria, 24.2% of patients had MOD on day 1, of whom 7.4% died (OR 12.2 for mortality, 95% CI 9.9–15.1), resulting in a sensitivity of 78% and a specificity of 77% for in-hospital mortality.

NPMOD or persistent MOD based on PODIUM criteria by day 7 occurred in 17.7% of patients, with an associated in-hospital mortality of 11.7%, and an OR of 47.8 (95% CI 36.1–64.8) for mortality compared with those who did not meet criteria. Meeting NPMOD or persistent MOD criteria had a sensitivity of 90% and a specificity of 84% for in-hospital mortality. When using the IPSCC criteria, 15.3% of patients had NPMOD or persistent MOD by day 7, of whom 12.5% died (OR 32.6 for mortality, 95% CI 25.9–41.7), resulting in a sensitivity of 84% and a specificity of 86% for in-hospital mortality (Supplemental Table 1).

Supplemental Fig 5 presents the average trajectory of organ dysfunction by type and day during the first 7 days among patients with MOD on day 1 based on the PODIUM criteria. Supplemental Table 2 presents this same information with the prevalence and associated in-hospital mortality for each individual organ dysfunction. In general, endocrine dysfunction was common in most patients on day 1. Nonsurvivors and those with progressive or persistent MOD had more cardiovascular, neurologic, and respiratory dysfunctions earlier in the course and persistent neurologic, respiratory, and hematologic dysfunction later in the course (Supplemental Fig 5). New, progressive, and persistent respiratory dysfunction was associated with the largest number of deaths compared with the other individual organ dysfunctions (Supplemental Table 2).

In this study, we present the first evaluation of the PODIUM organ dysfunction criteria in 2 large EHR databases. Seven of the 9 individual PODIUM organ dysfunctions had a cumulative incidence ranging from 15% to 30% by day 28, with an associated in-hospital mortality of 6% to 10%. The other 2 organ dysfunctions, liver and coagulation dysfunction, occurred in 3% to 5% of patients and were associated with an in-hospital mortality of 16% to 30%. The cumulative incidence and associated mortality were relatively stable across age groups and the 2 hospitals in the study, demonstrating generalizability. The number of concurrent PODIUM organ dysfunction criteria met by individual patients on day 1 of PICU stay demonstrated good discrimination for in-hospital mortality and better than the IPSCC criteria (AUC 0.87 vs 0.84, P < .001). Additionally, the maximum number of concurrent PODIUM organ dysfunction criteria met by individual patients by days 3, 7, and 28 demonstrated good to excellent discrimination for in-hospital mortality and compared favorably to the IPSCC criteria. Approximately 30% of patients met criteria for MOD on day 1 based on the PODIUM criteria and these had a 14-fold higher likelihood of dying than those who did not meet criteria. Meeting PODIUM criteria for MOD on day 1 was associated with higher sensitivity, lower specificity, and higher odds for in-hospital mortality than the IPSCC criteria. Finally, ∼18% of patients had new, progressive, or persistent MOD by day 7 based on the PODIUM criteria and these had a 48-fold higher likelihood of dying than those who did not meet criteria. Meeting PODIUM criteria for new, progressive, or persistent MOD by day 7 was also associated with higher sensitivity, lower specificity, and higher odds for in-hospital mortality than the IPSCC criteria.

Most of the difference we found in sensitivity and specificity between the PODIUM and IPSCC criteria is likely because PODIUM includes 9 organ dysfunctions versus the 6 proposed by IPSCC.⁶  The larger number and clearer delineation of organ dysfunctions in PODIUM, such as the addition of the immunologic and endocrine criteria and the distinction of the hematologic and coagulation dysfunctions, are desirable features in a set of organ dysfunction criteria. In recent years, there has been an increasing interest in understanding the effect that different organ dysfunction-based phenotypes have on outcome and response to therapy.^13,21,22  That is, the focus has not been on describing the severity of organ dysfunctions alone, but rather the actual type and combination of organ dysfunctions and their impact on outcomes. Thus, having a larger set of organ dysfunctions with clearer delineations between organ systems is likely to benefit this important line of research. Additionally, the number of concurrent PODIUM organ dysfunction criteria met had a favorable prognostic performance compared with the IPSCC criteria, which is also a desirable characteristic given the strong association of organ dysfunction burden and mortality in critically ill children.

In our study, we show that progressive MOD is more frequent and is associated with higher mortality than new MOD, similar to previous analyses.^23–25  Expanding on the MOD-based outcome literature, we also present the prevalence and associated mortality of persistent MOD, which captures patients with MOD on day 1 who, regardless of whether they develop progressive organ dysfunctions or not, are persistently affected by MOD a week after their presentation. Among all PICU patients in our study, almost 1 in 5 developed new, progressive, or persistent MOD during the first 7 days of stay, and those patients were almost 50 times more likely to die than those who either never had MOD or those who had nonprogressive, nonpersistent MOD. Using new, progressive, or persistent MOD based on the PODIUM criteria as an outcome in pediatric clinical trials appears promising, similar to what has been proposed and used previously.^5,24,26 

Our study has several strengths and limitations. Study data were derived from EHR databases corresponding to 2 large, urban, academic PICUs. Although the size and heterogeneity of the patient population is an advantage from a generalizability standpoint, our cohort may not be representative of all PICU populations, particularly in smaller, less urban, or nonacademic settings. Additionally, our aim was to evaluate the face validity, prognostic performance, and generalizability of the PODIUM criteria using structured EHR data, which we were able to achieve for a majority of the criteria. However, we did not evaluate a number of criteria, either because of the low number of patients with recorded values (eg, CD4+ T lymphocyte counts or total thyroxine levels) or the absence of the criteria as structured variables in the EHR (eg, electroencephalography results or type of respiratory failure). Some of these criteria may only be useful for studies with prospective data and blood specimen collection and should be further studied. However, we do highlight some of the variables that could be processed and included in EHRs in structured formats (eg, left ventricular ejection fraction from echocardiographic studies) to allow their integration with data-driven systems. As the development of data-driven systems continues to expand, work in this area is needed.

We present the first evaluation of the PODIUM organ dysfunction criteria in 2 large EHR databases. The prevalence and associated in-hospital mortality of the different PODIUM criteria is relatively similar across age groups and between the 2 PICUs in the study. The number of concurrent PODIUM organ dysfunction criteria demonstrated good to excellent discrimination for in-hospital mortality and compared favorably to the IPSCC organ dysfunction criteria. Similarly, MOD on day 1 and new, progressive, or persistent MOD by day 7 based on the PODIUM criteria demonstrated higher sensitivity and higher odds for in-hospital mortality than the IPSCC criteria. The use of the PODIUM organ dysfunction criteria appears promising for epidemiologic and clinical research studies using EHR data. More studies are needed to evaluate the PODIUM criteria that are not routinely collected in structured format or are infrequently recorded in EHR databases.

AUC

area under the curve

CI

confidence interval

EHR

electronic health record

IPSCC

International Pediatric Sepsis Consensus Conference

MOD

multiple organ dysfunction

NPMOD

new or progressive MOD

OR

odds ratio

PODIUM

Pediatric Organ Dysfunction Information Update Mandate