OBJECTIVES:

To develop predictive models for death or neurodevelopmental impairment (NDI) after neonatal hypoxic-ischemic encephalopathy (HIE) from data readily available at the time of NICU admission (“early”) or discharge (“cumulative”).

METHODS:

In this retrospective cohort analysis, we used data from the Children’s Hospitals Neonatal Consortium Database (2010–2016). Infants born at ≥35 weeks’ gestation and treated with therapeutic hypothermia for HIE at 11 participating sites were included; infants without Bayley Scales of Infant Development scores documented after 11 months of age were excluded. The primary outcome was death or NDI. Multivariable models were generated with 80% of the cohort; validation was performed in the remaining 20%.

RESULTS:

The primary outcome occurred in 242 of 486 infants; 180 died and 62 infants surviving to follow-up had NDI. HIE severity, epinephrine administration in the delivery room, and respiratory support and fraction of inspired oxygen of 0.21 at admission were significant in the early model. Severity of EEG findings was combined with HIE severity for the cumulative model, and additional significant variables included the use of steroids for blood pressure management and significant brain injury on MRI. Discovery models revealed areas under the curve of 0.852 for the early model and of 0.861 for the cumulative model, and both models performed well in the validation cohort (goodness-of-fit χ2: P = .24 and .06, respectively).

CONCLUSIONS:

Establishing reliable predictive models will enable clinicians to more accurately evaluate HIE severity and may allow for more targeted early therapies for those at highest risk of death or NDI.

What’s Known on This Subject:

Neonates with hypoxic-ischemic encephalopathy are at high risk for death or neurodevelopmental impairment, but accurate biomarkers for the identification of infants at the highest risk for adverse outcomes within the first days of life remain elusive.

What This Study Adds:

Using the largest data set to date of developmental outcomes collected in infants after hypothermia for neonatal hypoxic-ischemic encephalopathy, we provide two models for predicting death or neurodevelopmental impairment in as early as the first few hours of life.

Neonatal encephalopathy is a devastating clinical condition with multiple etiologies that affects ∼3 per 1000 live births in high-income countries. In approximately half of those infants, encephalopathy results from acute or chronic hypoxia-ischemia.1  Now a standard of care, therapeutic hypothermia (TH) significantly decreases death and neurodevelopmental impairment (NDI) after hypoxic-ischemic encephalopathy (HIE).2  Despite these improvements, however, nearly half of infants with moderate to severe HIE die or develop significant NDI. To date, the search for a single risk factor or widely available biomarker to predict death or NDI in this population has provided mixed results.

During the current era of TH, the greatest successes in predicting death or NDI have been achieved with electrophysiological techniques and brain MRI.35  Presently, the best quantitative imaging predictor for neurodevelopmental outcome is magnetic resonance spectroscopy (MRS).6,7  Although promising, translating these studies to the world outside the confines of clinical trials requires significant resources, including technology such as EEG, amplitude-integrated electroencephalography (aEEG), MRI, and MRS as well as specialized technicians and consultants. In the current study, we sought to develop predictive models for death or NDI after neonatal HIE using data readily available to any center providing TH. The Children’s Hospitals Neonatal Consortium Database (CHND) was established to assess disease-specific outcomes across participating NICUs within children’s hospitals in North America8  and provides a robust contemporary repository for demographic and clinical data, including advanced neurodiagnostic testing results.9 

We aimed to evaluate prediction of death or NDI in infants with HIE at two time points: (1) NICU admission using “early” clinical variables (ie, details of delivery, resuscitation, and initial transport) and (2) hospital discharge using “cumulative” data from throughout the entire hospitalization. These pragmatically staged models provide information early, when significant management decisions must often be made, as well as later in the hospitalization, allowing for predischarge prognostication. Establishing reliable predictive models will enable clinicians to more accurately identify infants at high risk for NDI and can be a major step toward establishing valuable algorithms for early surveillance and support services.

The CHND prospectively captures detailed clinical data from all infants admitted to 34 participating NICUs.

The CHND was queried to identify newborns with HIE treated with TH between January 2010 and December 2016. Inclusion criteria for the study were diagnosis of HIE, treatment with TH, admission at <4 days of life, and birth at ≥35 weeks’ gestational age. Neonates with major congenital anomalies and those with nonperinatal HIE were excluded. The institutional review board (IRB) at each institution approved participation in the CHND and associated research studies. Additional IRB approval was obtained from centers to collect follow-up data.

Data abstraction and collection from the CHND was performed as previously described.10  The specific battery of standardized tests obtained at developmental follow-up varied across centers, but for the analyses in this study, only Bayley Scales of Infant Development, Third Edition (BSID-III) and Gross Motor Function Classification Scale (GMFCS) data were analyzed. As described in previous neonatal database analyses,11  NDI was defined as the presence of any of the following: BSID-III composite cognitive score <85 or composite motor score <70, deafness, blindness, or gross motor delay defined as a GMFCS score ≥2. Because we were interested in follow-up at ≥1 year, the first follow-up visit occurring on or after 335 days of life (allowing for visits up to 1 month before 1 year of age) was included.

Developmental outcomes were evaluated by comparing infants with the primary outcome of death or NDI with those who survived without NDI. Study population characteristics were described by using nonparametric statistics as data departed from normality. Fisher’s exact test was used for categorical variables, and the Wilcoxon rank test was used for continuous distributions. A P value <.05 was used to determine statistical significance in the bivariate analyses.

Significant variables identified in the bivariate analyses were then subjected to logistic regression analyses. For these latter analyses, we chose to use two time points to develop clinical models: an early predictor model based on the information available around the time of NICU admission and a cumulative predictor model based on data obtained throughout the hospitalization. For the cumulative model, severity of aEEG or EEG findings at 24 hours was combined with HIE severity to generate an overall severity variable because of concerns for collinearity between the variables (association confirmed statistically, P < .001). Severely abnormal aEEG or EEG findings were defined as the presence of burst suppression, isoelectric, or flat tracings, and moderately abnormal findings were defined as a discontinuous pattern. Pattern interpretation was abstracted from neurology or neurophysiology reports in the medical record by the data abstractors, and when unclear, the classification was verified by the neonatologist site sponsor. Similarly, MRI findings that were significant in the bivariate analysis were combined into a single “severe MRI” variable based on the most predictive linear combination of individual MRI findings.

In model development, we considered a random selection of 80% of the cohort as model building data and the remaining 20% as test data for model validation. Both early and cumulative models underwent stepwise backward selection until all remaining variables were significant at an α of .05. Data are presented as odds ratios with 95% confidence intervals. For the validation model in the remaining 20% of the cohort, the Hosmer-Lemeshow goodness-of-fit test was used to validate the model equation fit, and receiver operating characteristic (ROC) curves were calculated. All data were analyzed by using SAS Enterprise Guide 7.1 (SAS Institute, Inc, Cary, NC).

Of the 34 CHND sites, 11 obtained IRB approval and collected developmental follow-up data (included sites are listed in the Acknowledgments section). From those 11 sites, 1693 infants with HIE were entered into the database over the 6 years of the study period, of whom 486 (28.7%) had outcomes, including 180 (10.6%) who died and 306 (18.1%) who had developmental follow-up visits with BSID-III scores documented after 11 months of age. In those infants with follow-up, 62 (20.3%) had NDI, of whom 59% and 45% had BSID-III cognitive scores <85 and motor scores <70, respectively; 2% had abnormal hearing; and 21% had GMFCS scores ≥2. The median time to follow-up for the entire study cohort was 723 (interquartile range [IQR] 565–851) days: 726.5 (IQR 619–828) days for those with NDI and 719 (IQR 558.5–832.5) days for those who survived without NDI.

The excluded infants who survived but did not have BSID-III scores documented were compared with those that were included in this cohort (Supplemental Table 4). In Table 1, the demographics and presenting clinical characteristics are compared between those infants who died or developed NDI at follow-up and those who survived without NDI. All infants included in this cohort were outborn. Site as a variable was significantly associated with death or NDI (P < .001) in bivariate analyses. In Table 2, early management and outcomes are compared between the two groups.

TABLE 1

Subject Demographics and Presenting Characteristics

All Infants (N = 486)Death or NDI (n = 242)Survival Without NDI (n = 244)Pa
Gestational age, wk, mean ± SD 38.8 ± 1.5 38.8 ± 1.5 38.8 ± 1.5 .87 
Birth wt, g, mean ± SD 3349 ± 616 3353 ± 622 3344 ± 612 .87 
Female sex, n (%) 223 (45.9) 107 (44.2) 116 (47.5) .47 
Race and/or ethnicity, n (%)    .10 
 White 303 (62.4) 149 (61.6) 154 (63.1) .78 
 Black 98 (20.2) 48 (20.0) 50 (20.5) .91 
 Other 72 (14.8) 34 (14.1) 38 (15.6) .70 
 Unknown 13 (2.7) 11 (4.6) 2 (0.9) .01 
Maternal antenatal conditions, n (%)     
 Diabetes 56 (11.5) 28 (11.6) 28 (11.5) .99 
 Hypertension 75 (15.4) 33 (13.6) 42 (17.2) .32 
 Chorioamnionitis 36 (7.4) 15 (6.2) 21 (8.6) .39 
Perinatal sentinel event, n (%)     
 Nuchal cord 86 (17.7) 37 (15.3) 49 (20.1) .19 
 Cord prolapse 13 (2.7) 7 (2.9) 6 (2.5) .79 
 Uterine rupture 32 (6.6) 21 (8.7) 11 (4.5) .07 
 Placental abruption 68 (14.0) 36 (14.9) 32 (13.1) .60 
Delivery type, n (%)     
 Vaginal, nonoperative 125 (25.7) 55 (22.7) 70 (28.7) .15 
 Vaginal, operative 55 (11.3) 27 (11.2) 28 (11.5) .99 
 Cesarean 306 (63.0) 160 (66.1) 146 (60.0) .16 
Apgar scores, median (IQR)     
 1-min Apgar score 1 (0–2) 1 (0–1) 1 (1–2) <.001 
 5-min Apgar score 3 (1–4) 2 (0–3) 3 (2–4) <.001 
 10-min Apgar score 3 (1–4) 3 (1–4) 4 (2–5) <.001 
Delivery room resuscitation, n (%)     
 Intubation or ventilationb 352 (72.4) 195 (80.6) 157 (64.3) <.001 
 Chest compressionc 214 (44.0) 139 (57.4) 75 (30.7) <.001 
 Epinephrine administrationd 141 (29.0) 108 (44.6) 33 (13.5) <.001 
Presenting pH <7.0, n (%) 106 (21.8) 52 (21.5) 54 (22.1) .91 
 Median (IQR) 7 (6.8–7.2) 7 (6.8–7.1) 7 (6.9–7.2) .006 
Presenting BD >16, n (%) 177 (36.4) 87 (36.0) 90 (36.9) .85 
 Median (IQR) 16 (10.2–22) 19 (12.9–24.8) 14 (8.7–20.3) .001 
HIE severity, n (%)     
 Mild 48 (9.9) 12 (5.0) 36 (14.8) <.001 
 Moderate 215 (44.2) 52 (21.5) 163 (66.8) <.001 
 Severe 199 (41.0) 171 (70.7) 28 (11.5) <.001 
 Not assigned 24 (4.9) 12 (5.0) 17 (7.0) .06 
Seizures before admission, n (%)     
 Clinical seizures, not confirmed 146 (30.0) 85 (35.1) 61 (25.0) .02 
 Confirmed by EEG or aEEG 8 (1.7) 7 (2.8) 1 (0.4) .04 
Vasoactive infusion on transport, n (%) 110 (22.6) 83 (34.3) 27 (11.1) <.001 
All Infants (N = 486)Death or NDI (n = 242)Survival Without NDI (n = 244)Pa
Gestational age, wk, mean ± SD 38.8 ± 1.5 38.8 ± 1.5 38.8 ± 1.5 .87 
Birth wt, g, mean ± SD 3349 ± 616 3353 ± 622 3344 ± 612 .87 
Female sex, n (%) 223 (45.9) 107 (44.2) 116 (47.5) .47 
Race and/or ethnicity, n (%)    .10 
 White 303 (62.4) 149 (61.6) 154 (63.1) .78 
 Black 98 (20.2) 48 (20.0) 50 (20.5) .91 
 Other 72 (14.8) 34 (14.1) 38 (15.6) .70 
 Unknown 13 (2.7) 11 (4.6) 2 (0.9) .01 
Maternal antenatal conditions, n (%)     
 Diabetes 56 (11.5) 28 (11.6) 28 (11.5) .99 
 Hypertension 75 (15.4) 33 (13.6) 42 (17.2) .32 
 Chorioamnionitis 36 (7.4) 15 (6.2) 21 (8.6) .39 
Perinatal sentinel event, n (%)     
 Nuchal cord 86 (17.7) 37 (15.3) 49 (20.1) .19 
 Cord prolapse 13 (2.7) 7 (2.9) 6 (2.5) .79 
 Uterine rupture 32 (6.6) 21 (8.7) 11 (4.5) .07 
 Placental abruption 68 (14.0) 36 (14.9) 32 (13.1) .60 
Delivery type, n (%)     
 Vaginal, nonoperative 125 (25.7) 55 (22.7) 70 (28.7) .15 
 Vaginal, operative 55 (11.3) 27 (11.2) 28 (11.5) .99 
 Cesarean 306 (63.0) 160 (66.1) 146 (60.0) .16 
Apgar scores, median (IQR)     
 1-min Apgar score 1 (0–2) 1 (0–1) 1 (1–2) <.001 
 5-min Apgar score 3 (1–4) 2 (0–3) 3 (2–4) <.001 
 10-min Apgar score 3 (1–4) 3 (1–4) 4 (2–5) <.001 
Delivery room resuscitation, n (%)     
 Intubation or ventilationb 352 (72.4) 195 (80.6) 157 (64.3) <.001 
 Chest compressionc 214 (44.0) 139 (57.4) 75 (30.7) <.001 
 Epinephrine administrationd 141 (29.0) 108 (44.6) 33 (13.5) <.001 
Presenting pH <7.0, n (%) 106 (21.8) 52 (21.5) 54 (22.1) .91 
 Median (IQR) 7 (6.8–7.2) 7 (6.8–7.1) 7 (6.9–7.2) .006 
Presenting BD >16, n (%) 177 (36.4) 87 (36.0) 90 (36.9) .85 
 Median (IQR) 16 (10.2–22) 19 (12.9–24.8) 14 (8.7–20.3) .001 
HIE severity, n (%)     
 Mild 48 (9.9) 12 (5.0) 36 (14.8) <.001 
 Moderate 215 (44.2) 52 (21.5) 163 (66.8) <.001 
 Severe 199 (41.0) 171 (70.7) 28 (11.5) <.001 
 Not assigned 24 (4.9) 12 (5.0) 17 (7.0) .06 
Seizures before admission, n (%)     
 Clinical seizures, not confirmed 146 (30.0) 85 (35.1) 61 (25.0) .02 
 Confirmed by EEG or aEEG 8 (1.7) 7 (2.8) 1 (0.4) .04 
Vasoactive infusion on transport, n (%) 110 (22.6) 83 (34.3) 27 (11.1) <.001 

BD, base deficit.

a

Comparison between death or NDI and survival without NDI.

b

Data missing in 3 patients.

c

Data missing in 3 patients.

d

Data missing in 5 patients.

TABLE 2

Early Management and Outcomes

All Infants (N = 486)Death or NDI (n = 242)Survival Without NDI (n = 244)Pa
Type of TH, n (%)b     
 Head 75 (15.4) 42 (17.4) 33 (13.5) .26 
 Whole body 414 (85.2) 203 (83.9) 211 (86.5) .45 
 Both 9 (1.9) 8 (3.3) 1 (0.4) .02 
Time of hypothermia initiation, h, median (IQR)c 5 (4–6) 5 (4–6) 5 (4–6) .33 
Time target temperature achieved, h, median (IQR)c 6 (5–8) 6 (4–8) 6 (5–8) .87 
Cooling interrupted >30 min, n (%) 45 (9.3) 32 (13.2) 13 (5.3) .003 
Comorbidities, n (%)d     
 Coagulopathy 151 (31.1) 100 (41.3) 51 (20.9) <.001 
 Bradycardia, HR <80 beats per minute 70 (14.4) 27 (11.2) 43 (17.6) <.001 
 Hyperglycemia, glucose level >200 mg/dL 112 (23.1) 85 (35.1) 27 (11.1) <.001 
 Skin necrosis 4 (0.9) 2 (0.8) 2 (0.8) .99 
 Hypothermia, >0.5°C below target 160 (32.9) 83 (34.3) 77 (31.6) .15 
Steroids for blood pressure, n (%)e 118 (24.3) 95 (39.3) 23 (9.4) <.001 
Inhaled nitric oxide, n (%)f 106 (21.8) 70 (28.9) 36 (14.8) <.001 
ECMO, n (%) 17 (3.5) 11 (4.6) 6 (2.5) .23 
Seizures ≤3 d of life, n (%)     
 Clinical seizures, not confirmed 68 (14.0) 34 (14.1) 34 (13.9) .99 
 Seizures confirmed by EEG or aEEG 128 (26.3) 89 (36.8) 39 (16.0) <.001 
EEG or aEEG seizures at 24 h, n (%) 118 (27.8) 82 (39.8) 36 (16.5) <.001 
Status epilepticus at 24 h, n (%)g 18 (4.9) 16 (9.3) 2 (1.0) <.001 
No suspected or confirmed seizures, n (%) 262 (53.9) 101 (41.7) 161 (66.0) <.001 
Total days ventilated, median (IQR) 4 (2–8) 5 (2–9) 4 (1–7) .003 
Length of stay, d, median (IQR) 10 (6–18) 7 (2–14) 13 (9–20.5) <.001 
All Infants (N = 486)Death or NDI (n = 242)Survival Without NDI (n = 244)Pa
Type of TH, n (%)b     
 Head 75 (15.4) 42 (17.4) 33 (13.5) .26 
 Whole body 414 (85.2) 203 (83.9) 211 (86.5) .45 
 Both 9 (1.9) 8 (3.3) 1 (0.4) .02 
Time of hypothermia initiation, h, median (IQR)c 5 (4–6) 5 (4–6) 5 (4–6) .33 
Time target temperature achieved, h, median (IQR)c 6 (5–8) 6 (4–8) 6 (5–8) .87 
Cooling interrupted >30 min, n (%) 45 (9.3) 32 (13.2) 13 (5.3) .003 
Comorbidities, n (%)d     
 Coagulopathy 151 (31.1) 100 (41.3) 51 (20.9) <.001 
 Bradycardia, HR <80 beats per minute 70 (14.4) 27 (11.2) 43 (17.6) <.001 
 Hyperglycemia, glucose level >200 mg/dL 112 (23.1) 85 (35.1) 27 (11.1) <.001 
 Skin necrosis 4 (0.9) 2 (0.8) 2 (0.8) .99 
 Hypothermia, >0.5°C below target 160 (32.9) 83 (34.3) 77 (31.6) .15 
Steroids for blood pressure, n (%)e 118 (24.3) 95 (39.3) 23 (9.4) <.001 
Inhaled nitric oxide, n (%)f 106 (21.8) 70 (28.9) 36 (14.8) <.001 
ECMO, n (%) 17 (3.5) 11 (4.6) 6 (2.5) .23 
Seizures ≤3 d of life, n (%)     
 Clinical seizures, not confirmed 68 (14.0) 34 (14.1) 34 (13.9) .99 
 Seizures confirmed by EEG or aEEG 128 (26.3) 89 (36.8) 39 (16.0) <.001 
EEG or aEEG seizures at 24 h, n (%) 118 (27.8) 82 (39.8) 36 (16.5) <.001 
Status epilepticus at 24 h, n (%)g 18 (4.9) 16 (9.3) 2 (1.0) <.001 
No suspected or confirmed seizures, n (%) 262 (53.9) 101 (41.7) 161 (66.0) <.001 
Total days ventilated, median (IQR) 4 (2–8) 5 (2–9) 4 (1–7) .003 
Length of stay, d, median (IQR) 10 (6–18) 7 (2–14) 13 (9–20.5) <.001 

ECMO, extracorporeal membrane oxygenation; HR, heart rate.

a

Comparison between death or NDI and survival without NDI.

b

Use of each is not mutually exclusive, so they may add up to more than the total n.

c

Data missing in 6 patients.

d

Data missing in 198 patients.

e

Data missing in 350 patients.

f

Data missing in 29 patients.

g

Data missing in 2 patients.

EEG, aEEG, and MRI interpretations were compared between groups (Fig 1). Standard EEG data were available for 353 of 399 (88.5%) infants who had background interpretations documented; the remaining infants had aEEG data only. The primary outcome group had more severely abnormal aEEG findings at 6 hours (P < .001) and severely abnormal EEG or aEEG findings at 24 hours (P < .001). Conversely, the survival without NDI group had normal aEEG background findings at 6 hours significantly more often (P < .001).

FIGURE 1

EEG, aEEG, and MRI findings stratified by cohort. A, aEEG findings at 6 hours. B, EEG or aEEG findings at 24 hours. C, MRI findings. There were more severely abnormal aEEG or EEG findings in the death or NDI group at 6 hours (P < .001) and 24 hours (P < .001). * P < .001; ** P = .003.

FIGURE 1

EEG, aEEG, and MRI findings stratified by cohort. A, aEEG findings at 6 hours. B, EEG or aEEG findings at 24 hours. C, MRI findings. There were more severely abnormal aEEG or EEG findings in the death or NDI group at 6 hours (P < .001) and 24 hours (P < .001). * P < .001; ** P = .003.

Close modal

We modeled the association between death and NDI with the early and cumulative groups of variables on 80% of the cohort (n = 388). For the early model, 11 predictors (Supplemental Table 5) were selected because of to their significance in the bivariate analysis as well as their perceived clinical significance. Among those predictors, 4 remained significant in the early model (Table 3): severity of HIE, delivery room epinephrine administration, respiratory support at admission, and a fraction of inspired oxygen (Fio2) of 0.21 at admission. The discovery model had an area under the receiver operating characteristic curve (AUC) of 0.852. The resulting regression equation and ROC analyses (Fig 2) were used to validate the model by predicting death or NDI in the remaining 20% of the cohort. After validation, the observed event rate matched the expected rate (goodness-of-fit χ2: P = .236), and the AUC for this validation cohort was 0.810. To evaluate the effect of the individual site on the model, the site variable was added to the early model, and the association with death or NDI remained significant (P = .041), although Fio2 and respiratory support were no longer significant. This indicated possible collinearity between site and the two respiratory predictors, so site was not included in the final model.

TABLE 3

Multivariable Logistic Regression Equations Predicting Death or NDI by Using Variables From Delivery and Transport (Early) or During the Entire Hospital Stay (Cumulative)

VariableOR95% CIP
Early clinical predictor model    
 Severe-grade HIE 19.13 10.29–35.56 <.001 
 Fio2 of 0.21 on admission 0.43 0.23–0.82 .01 
 No respiratory support on admission 0.45 0.24–0.84 .01 
 Delivery room epinephrine 2.00 1.05–3.78 .03 
Cumulative clinical predictor model    
 Severe-grade HIE or severely abnormal aEEG or cEEG findings at 24 ha,b 10.91 6.00–19.86 <.001 
 Fio2 of 0.21 on admissiona 0.47 0.24–0.92 .03 
 No respiratory support on admissiona 0.45 0.24–0.87 .02 
 Delivery room epinephrinea 2.34 1.22–4.48 .01 
 Steroids for blood pressure 4.71 2.28–9.73 <.001 
 Severe injury on MRIc 2.11 1.02–4.34 .04 
VariableOR95% CIP
Early clinical predictor model    
 Severe-grade HIE 19.13 10.29–35.56 <.001 
 Fio2 of 0.21 on admission 0.43 0.23–0.82 .01 
 No respiratory support on admission 0.45 0.24–0.84 .01 
 Delivery room epinephrine 2.00 1.05–3.78 .03 
Cumulative clinical predictor model    
 Severe-grade HIE or severely abnormal aEEG or cEEG findings at 24 ha,b 10.91 6.00–19.86 <.001 
 Fio2 of 0.21 on admissiona 0.47 0.24–0.92 .03 
 No respiratory support on admissiona 0.45 0.24–0.87 .02 
 Delivery room epinephrinea 2.34 1.22–4.48 .01 
 Steroids for blood pressure 4.71 2.28–9.73 <.001 
 Severe injury on MRIc 2.11 1.02–4.34 .04 

cEEG, continuous electroencephalography; CI, confidence interval; OR, odds ratio.

a

Carried over because of demonstrated significance in early model.

b

Severely abnormal aEEG or cEEG findings were defined as the presence of burst suppression, isoelectric, or flat tracings.

c

Severe injury was defined as the presence of deep gray injury, cortical injury, or both.

FIGURE 2

ROC curves for initial models before validation. A, Early predictor model. B, Late predictor model.

FIGURE 2

ROC curves for initial models before validation. A, Early predictor model. B, Late predictor model.

Close modal

The 4 significant predictors in the early model were carried over to the cumulative model, with the exception of HIE severity, which was combined with severity of aEEG or EEG findings at 24 hours to generate a combined severity variable. In addition to the 4 early variables, 16 variables were chosen for inclusion in the cumulative model (Supplemental Table 5). The final cumulative model (Table 3) had 6 variables (4 early variables and 2 cumulative variables) that remained significant. The two additional variables were the use of steroids for blood pressure management and significant brain injury on MRI, defined as either cortical injury or deep gray matter injury (white matter injury was not significant when inserted into the model). Electrographic seizures were significantly associated with the primary outcome in univariate analyses (P < .001) but were not significant in the final model because of significant collinearity with the severity variable (P < .001). When we used the same process as for the early model, the discovery model had an AUC of 0.861 (Fig 2), and the validation performed well (goodness-of-fit χ2: P = .060), with an AUC of 0.781.

Our results describe two models capable of accurately predicting death or NDI in infants after TH for neonatal HIE. The AUCs for both the early (AUC 0.85) and cumulative (AUC 0.86) models suggest an excellent (defined as AUC 0.8–0.9) ability to discriminate between the primary outcomes, and the models performed similarly to those shown in a meta-analysis of NDI prediction that revealed pooled AUCs of 0.88, 0.88, and 0.78 for MRI, EEG, and aEEG background findings, respectively.12  Both of the current models use data that are regularly collected throughout the typical hospital course of an infant after HIE. Despite not including EEG or imaging data, which are usually not available at admission, the early model performed well in predicting death or NDI. The addition of variables such as EEG, MRI, and hemodynamic data, which are collected later in the NICU course, further improved the prediction performance of the cumulative model.

Defining HIE severity remains a matter of continued debate, and there is no clear consensus in the literature over the definitions of mild, moderate, and severe injury.13  Despite the concern for subjectivity in the neonatal neurologic examination, in the current study, the injury grade was significantly associated with death or NDI. The classification of HIE severity was additionally supported in our cumulative model by the inclusion of EEG severity findings to generate an overall severity variable, which may aid in decreasing subjectivity. Similar to our findings, several other studies have revealed associations between HIE severity grading and death or disability at 6 months14,15  and 1 year of life.16  When interpreting the current data, it is important to note that the CHND records severity on the basis of the worst grade observed over the first 7 days of life on the basis of the knowledge that encephalopathy evolves and may deteriorate over the course of the first few days after injury.17 

In addition to the HIE severity, EEG is one of the more common ways that clinicians attempt to estimate brain injury severity in the first few hours to days of life. EEG has been shown to correlate moderately well with neurodevelopmental outcomes, with an overall limited sensitivity of 0.63 but a specificity of 0.82, whereas aEEG background patterns have a sensitivity of 0.90 and specificity of 0.46 for predicting unfavorable outcomes.12  In the current study, EEG findings at 24 hours were associated with death or NDI in the bivariate analyses, and the combined variable of severe aEEG or EEG findings and severe HIE grade remained significantly associated with death or NDI in the final model. Although electrographic seizures were associated with the primary outcome in univariate analyses, they were not included in the final model because of collinearity with severity. Unfortunately, the CHND does not record sleep-wake cycling or interburst interval characteristics, both of which have previously been shown to be strong predictors of adverse outcomes.18,19 

Specific patterns of injury on MRI may also be associated with NDI. In the current study, severe injury on MRI was defined as the presence of either cortical or deep gray matter injury. White matter injury was also evaluated for the cumulative model but did not reach significance, which is in line with a recent study revealing that white matter injury had the lowest specificity of the MRI biomarkers evaluated, with only 7% of the infants with white matter injury developing NDI at 2 years.20  Previous studies have revealed that deep gray matter injury, such as bilateral basal ganglia changes, has an 84% sensitivity but a 42% specificity for death or central motor deficit at 1 year,21  although diffusion imaging of the basal ganglia and thalamus has revealed much higher specificities for death or NDI of 95% and 98%, respectively.22  Additionally, MRS has been evaluated as a prognostic biomarker, and deep gray matter lactate to N-acetyl aspartate ratio is one of the most accurate quantitative magnetic resonance biomarkers in the neonatal period for predicting NDI.23  Unfortunately, the CHND does not currently collect MRS data, so we were unable to assess MRS as a predictor in the models.

Two respiratory variables, the lack of respiratory support on admission and an Fio2 of 0.21, were found to be protective against death or NDI, potentially suggesting that they function as surrogate markers of overall severity of illness. They are likely more than simply markers of illness, however, because several other variables are also associated with severity of illness, such as pH and base deficit, but were not significant in the final models. They may also be surrogates for other disease processes; for instance, an increased Fio2 requirement can be a marker of pulmonary hypertension, which has previously been associated with abnormal outcomes.24  To further support this association, in the current study, the use of inhaled nitric oxide was associated with death or NDI in the bivariate analysis. The respiratory variables may also have direct effects on death or NDI because previous studies have revealed associations of oxygen and carbon dioxide fluctuations with adverse outcomes in infants with HIE.25,26  These concerns, along with the possible collinearity between study site and these two respiratory variables, highlight the importance of improving and standardizing respiratory management of neonates with HIE.

To generate the final models, we chose to use the simple supervised machine learning algorithm, logistic regression, because of the presumptions that the relationship between the features and output was simple and because the number of model building examples was low.27  A systematic review in which logistic regression was compared to other machine learning methods, including classification trees, random forests, artificial neural networks, and support vector machines, revealed that the latter methods were only superior for binary outcomes in studies with high risk for bias.28  Thus, although investigators have used more complex modeling algorithms for other purposes, logistic regression was effective for the modeling described here.

The main strength of this study is its large sample size. These models were created from the largest collection of neurodevelopmental outcomes in infants with HIE to date. In fact, the 486 infants in this study is just short of the 495 cooled infants with developmental follow-up data from all 8 of the trials in the Cochrane review combined.29 

The study has some limitations. The use of a composite outcome (ie, death or NDI), although justified by the competing nature of the two adverse outcomes, does decrease the specificity of the results and requires a reasonable but imperfect assumption that those infants who died of withdrawal of life support30  would most likely have suffered from either later death or NDI. Additionally, although the CHND takes several proactive steps to reduce error by training abstractors and maintaining thresholds in interrater agreement at each site, as in any secondary data analysis, unknown and unmeasured variables, as well as errors in coding, could alter the observed associations.

Lastly, because this was a pragmatic study outside the confines of a clinical trial, only 11 of the 34 sites collected follow-up data for this study; therefore, our study excluded a high number of infants without follow-up data recorded or without BSID-III scores. Infants who were lost to follow-up were less likely to have severe HIE, seizures, and injury on MRI. It is possible that these infants were doing well clinically and therefore had limited follow-up. The higher severity in the study population should be taken into account when applying these results to individual populations. Given that the CHND consists of NICUs at children’s hospitals with only outborn infants, our results may not be generalizable to all populations of infants with HIE. The early model, however, consisted of data collected before transfer and can be widely applicable regardless of the ultimate site of cooling. In addition, both the early and cumulative models should be applicable to regions where these infants are primarily cared for in outborn units.31 

Accurate assessment of risk for death or NDI remains a significant challenge in the care of infants with HIE. Although neuromonitoring and imaging advancements have improved the clinician’s ability to identify infants at highest risk for NDI, most of these data are not available for hours to days after the initial injury. Most predictive models in this population have been generated from within clinical trial cohorts, in which care is more strictly prescribed across sites. In contrast, the models generated in this study used contemporary data derived from typical clinical practice of managing patients with HIE. In the manner of previously published and widely used predictive models for extremely preterm birth outcomes and early-onset neonatal sepsis, we hope that our models for neonatal HIE may allow for improved counseling for families and more targeted early therapies for those at highest risk for death or NDI.

We thank the following institutions that serve the infants and their families, and these institutions also have invested and continue to participate in the CHND. The site sponsors and contributors for the CHND include the following: Children’s Healthcare of Atlanta (Atlanta, GA): Anthony Piazza; Children’s Healthcare of Atlanta at Scottish Rite (Atlanta, GA): Gregory Sysyn; Children’s of Alabama (Birmingham, AL): Carl Coghill; Le Bonheur Children’s Hospital (Memphis, TN): Ajay Talati; Boston Children’s Hospital (Boston, MA): Anne Hansen and Tanzeema Hossain*; Ann & Robert H. Lurie Children’s Hospital of Chicago (Chicago, IL): Karna Murthy and Gustave Falciglia*; Cincinnati Children’s Hospital Medical Center (Cincinnati, OH): Beth Haberman; Nationwide Children’s Hospital (Columbus, OH): Kristina Reber*; Children’s Medical Center Dallas (Dallas, TX): Rashmin Savani; Children’s Hospital Colorado (Aurora, CO): Theresa Grover*; Children’s Hospital of Michigan (Detroit, MI): Girija Natarajan; Cook Children’s Health Care System (Fort Worth, TX): Annie Chi and Yvette Johnson*; Texas Children’s Hospital (Houston, TX): Gautham Suresh; Riley Children’s Hospital (Indianapolis, IN): William Engle; Children’s Mercy Hospitals & Clinics (Kansas City, MO): Eugenia Pallotto*; Arkansas Children’s Hospital (Little Rock, AR): Robert Lyle and Becky Rogers; Children’s Hospital Los Angeles (Los Angeles, CA): Rachel Chapman; American Family Children’s Hospital (Madison, WI): Jamie Limjoco; University of California, San Francisco Benioff Children’s Hospital (Oakland, CA): Priscilla Joe*; Children’s Hospital of Philadelphia (Philadelphia, PA): Jacquelyn Evans, Michael Padula, and David Munson*; St Christopher’s Hospital for Children (Philadelphia, PA): Suzanne Touch; University of Pittsburgh Medical Center Children’s Hospital of Pittsburgh (Pittsburgh, PA): Beverly Brozanski; St Louis Children’s Hospital (St Louis, MO): Rakesh Rao and Amit Mathur*; Johns Hopkins All Children’s Hospital (St Petersburg, FL): Victor McKay; Rady Children’s Hospital–San Diego (San Diego, CA): Mark Speziale and Laurel Moyer*; Children’s National Medical Center (Washington, DC): Billie Short*; Nemours/Alfred I. duPont Hospital for Children (Wilmington, DE): Kevin Sullivan; Primary Children’s Medical Center (Salt Lake City, UT): Con Yee Ling; Children’s Wisconsin (Milwaukee, WI): Michael Uhing and Ankur Datta; Children’s Hospital and Medical Center (Omaha, NE): Nicole Birge; Florida Hospital for Children (Orlando, FL): Rajan Wadhawan; Seattle Children’s Hospital (Seattle, WA): Elizabeth Jacobsen-Misbe and Robert DiGeronimo; Hospital for Sick Children (Toronto, Canada): Kyong-Soon Lee; and Children’s Hospital of Orange County (Los Angeles, CA): Michel Mikhael.

Jeanette Asselin, Beverly Brozanski, David Durand (ex officio), Francine Dykes (ex officio), Jacquelyn Evans (executive director), Theresa Grover, Karna Murthy (chair), Michael Padula, Eugenia Pallotto, Anthony Piazza, Kristina Reber, and Billie Short are members of the Children’s Hospitals Neonatal Consortium, Inc. For more information, please contact support@thechnc.org.

*Those sites with follow-up data included in the current study.

Dr Peeples contributed to the design, analysis, and interpretation of the data, drafted the initial manuscript, and reviewed and critically revised the manuscript; Drs Dizon, and Rao contributed to the design, analysis, collection, and interpretation of the data, drafted the initial manuscript, and reviewed and critically revised the manuscript; Drs Johnson, Joe, Flibotte, Hossain, Smith, and Massaro contributed to the design, analysis, collection, and interpretation of the data and reviewed and critically revised the manuscript; Drs Hamrick, DiGeronimo, Natarajan, Lee, Yanowitz, Mietzsch, Wu, Mathur, and Zaniletti contributed to the design, analysis, and interpretation of the data and reviewed and critically revised the manuscript; Drs Maitre, Pallotto, and Speziale contributed substantially to the acquisition of the data and reviewed and revised the manuscript critically for important intellectual content; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

     
  • aEEG

    amplitude-integrated electroencephalography

  •  
  • AUC

    area under the receiver operating characteristic curve

  •  
  • BSID-III

    Bayley Scales of Infant Development, Third Edition

  •  
  • CHND

    Children’s Hospitals Neonatal Consortium Database

  •  
  • Fio2

    fraction of inspired oxygen

  •  
  • GMFCS

    Gross Motor Function Classification Scale

  •  
  • HIE

    hypoxic-ischemic encephalopathy

  •  
  • IQR

    interquartile range

  •  
  • IRB

    institutional review board

  •  
  • MRS

    magnetic resonance spectroscopy

  •  
  • NDI

    neurodevelopmental impairment

  •  
  • ROC

    receiver operating characteristic

  •  
  • TH

    therapeutic hypothermia

1
Kurinczuk
JJ
,
White-Koning
M
,
Badawi
N
.
Epidemiology of neonatal encephalopathy and hypoxic-ischaemic encephalopathy
.
Early Hum Dev
.
2010
;
86
(
6
):
329
338
2
Edwards
AD
,
Brocklehurst
P
,
Gunn
AJ
, et al
.
Neurological outcomes at 18 months of age after moderate hypothermia for perinatal hypoxic ischaemic encephalopathy: synthesis and meta-analysis of trial data
.
BMJ
.
2010
;
340
:
c363
3
Fitzgerald
MP
,
Massey
SL
,
Fung
FW
,
Kessler
SK
,
Abend
NS
.
High electroencephalographic seizure exposure is associated with unfavorable outcomes in neonates with hypoxic-ischemic encephalopathy
.
Seizure
.
2018
;
61
:
221
226
4
Skranes
JH
,
Løhaugen
G
,
Schumacher
EM
, et al
.
Amplitude-integrated electroencephalography improves the identification of infants with encephalopathy for therapeutic hypothermia and predicts neurodevelopmental outcomes at 2 years of age
.
J Pediatr
.
2017
;
187
:
34
42
5
Twomey
E
,
Twomey
A
,
Ryan
S
,
Murphy
J
,
Donoghue
VB
.
MR imaging of term infants with hypoxic-ischaemic encephalopathy as a predictor of neurodevelopmental outcome and late MRI appearances
.
Pediatr Radiol
.
2010
;
40
(
9
):
1526
1535
6
Barta
H
,
Jermendy
A
,
Kolossvary
M
, et al
.
Prognostic value of early, conventional proton magnetic resonance spectroscopy in cooled asphyxiated infants
.
BMC Pediatr
.
2018
;
18
(
1
):
302
7
Lally
PJ
,
Montaldo
P
,
Oliveira
V
, et al.;
MARBLE Consortium
.
Magnetic resonance spectroscopy assessment of brain injury after moderate hypothermia in neonatal encephalopathy: a prospective multicentre cohort study
.
Lancet Neurol
.
2019
;
18
(
1
):
35
45
8
Murthy
K
,
Dykes
FD
,
Padula
MA
, et al
.
The Children’s Hospitals Neonatal Database: an overview of patient complexity, outcomes and variation in care
.
J Perinatol
.
2014
;
34
(
8
):
582
586
9
Massaro
AN
,
Murthy
K
,
Zaniletti
I
, et al
.
Short-term outcomes after perinatal hypoxic ischemic encephalopathy: a report from the Children’s Hospitals Neonatal Consortium HIE focus group
.
J Perinatol
.
2015
;
35
(
4
):
290
296
10
Murthy
K
,
Dykes
FD
,
Padula
MA
, et al
.
The Children’s Hospitals Neonatal Database: an overview of patient complexity, outcomes and variation in care
.
J Perinatol
.
2014
;
34
(
8
):
582
586
11
Adams-Chapman
I
,
Heyne
RJ
,
DeMauro
SB
, et al.;
Follow-Up Study of the Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network
.
Neurodevelopmental impairment among extremely preterm infants in the neonatal research network
.
Pediatrics
.
2018
;
141
(
5
):
e20173091
12
Liu
W
,
Yang
Q
,
Wei
H
,
Dong
W
,
Fan
Y
,
Hua
Z
.
Prognostic value of clinical tests in neonates with hypoxic-ischemic encephalopathy treated with therapeutic hypothermia: a systematic review and meta-analysis
.
Front Neurol
.
2020
;
11
:
133
13
Leviton
A
,
Nelson
KB
.
Problems with definitions and classifications of newborn encephalopathy
.
Pediatr Neurol
.
1992
;
8
(
2
):
85
90
14
Jyoti
R
,
O’Neil
R
,
Hurrion
E
.
Predicting outcome in term neonates with hypoxic-ischaemic encephalopathy using simplified MR criteria
.
Pediatr Radiol
.
2006
;
36
(
1
):
38
42
15
Mwakyusa
SD
,
Manji
KP
,
Massawe
AW
.
The hypoxic ischaemic encephalopathy score in predicting neurodevelopmental outcomes among infants with birth asphyxia at the Muhimbili National Hospital, Dar-es-Salaam, Tanzania
.
J Trop Pediatr
.
2009
;
55
(
1
):
8
14
16
Abdulqawi
K
,
Al-Zohairy
YZ
,
Karam
K
.
Early predictors of neurodevelopmental adverse outcome in term infants with postasphyxial hypoxic ischemic encephalopathy
.
Int J Collab Res Intern Med Public Health
.
2011
;
3
(
11
):
822
837
17
Volpe
JJ
.
Perinatal hypoxic-ischemic brain injury
.
Pediatr Clin North Am
.
1976
;
23
(
3
):
383
397
18
Osredkar
D
,
Toet
MC
,
van Rooij
LG
,
van Huffelen
AC
,
Groenendaal
F
,
de Vries
LS
.
Sleep-wake cycling on amplitude-integrated electroencephalography in term newborns with hypoxic-ischemic encephalopathy
.
Pediatrics
.
2005
;
115
(
2
):
327
332
19
Dunne
JM
,
Wertheim
D
,
Clarke
P
, et al
.
Automated electroencephalographic discontinuity in cooled newborns predicts cerebral MRI and neurodevelopmental outcome
.
Arch Dis Child Fetal Neonatal Ed
.
2017
;
102
(
1
):
F58
F64
20
Montaldo
P
,
Ivain
P
,
Lally
P
, et al
.
White matter injury after neonatal encephalopathy is associated with thalamic metabolite perturbations
.
EBioMedicine
.
2020
;
52
:
102663
21
Rutherford
MA
,
Pennock
JM
,
Dubowitz
LM
.
Cranial ultrasound and magnetic resonance imaging in hypoxic-ischaemic encephalopathy: a comparison with outcome
.
Dev Med Child Neurol
.
1994
;
36
(
9
):
813
825
22
Alderliesten
T
,
de Vries
LS
,
Benders
MJNL
,
Koopman
C
,
Groenendaal
F
.
MR imaging and outcome of term neonates with perinatal asphyxia: value of diffusion-weighted MR imaging and 1H MR spectroscopy
.
Radiology
.
2011
;
261
(
1
):
235
242
23
Thayyil
S
,
Chandrasekaran
M
,
Taylor
A
, et al
.
Cerebral magnetic resonance biomarkers in neonatal encephalopathy: a meta-analysis
.
Pediatrics
.
2010
;
125
(
2
). Available at: https://pediatrics.aappublications.org/content/125/2/e382
24
Lakshminrusimha
S
,
Shankaran
S
,
Laptook
A
, et al
.
Pulmonary hypertension associated with hypoxic-ischemic encephalopathy-antecedent characteristics and comorbidities
.
J Pediatr
.
2018
;
196
:
45
51.e3
25
Klinger
G
,
Beyene
J
,
Shah
P
,
Perlman
M
.
Do hyperoxaemia and hypocapnia add to the risk of brain injury after intrapartum asphyxia?
Arch Dis Child Fetal Neonatal Ed
.
2005
;
90
(
1
):
F49
F52
26
Sabir
H
,
Jary
S
,
Tooley
J
,
Liu
X
,
Thoresen
M
.
Increased inspired oxygen in the first hours of life is associated with adverse outcome in newborns treated for perinatal asphyxia with therapeutic hypothermia
.
J Pediatr
.
2012
;
161
(
3
):
409
416
27
Doupe
P
,
Faghmous
J
,
Basu
S
.
Machine learning for health services researchers
.
Value Health
.
2019
;
22
(
7
):
808
815
28
Christodoulou
E
,
Ma
J
,
Collins
GS
,
Steyerberg
EW
,
Verbakel
JY
,
Van Calster
B
.
A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models
.
J Clin Epidemiol
.
2019
;
110
:
12
22
29
Jacobs
SE
,
Berg
M
,
Hunt
R
,
Tarnow-Mordi
WO
,
Inder
TE
,
Davis
PG
.
Cooling for newborns with hypoxic ischaemic encephalopathy
.
Cochrane Database Syst Rev
.
2013
;(
1
):
CD003311
30
Natarajan
G
,
Mathur
A
,
Zaniletti
I
, et al.;
Children’s Hospitals Neonatal Consortium (CHNC)
.
Withdrawal of life-support in neonatal hypoxic-ischemic encephalopathy
.
Pediatr Neurol
.
2019
;
91
:
20
26
31
Goswami
IR
,
Whyte
H
,
Wintermark
P
, et al.;
Canadian Neonatal Network Investigators
.
Characteristics and short-term outcomes of neonates with mild hypoxic-ischemic encephalopathy treated with hypothermia
.
J Perinatol
.
2020
;
40
(
2
):
275
283

Competing Interests

POTENTIAL CONFLICT OF INTEREST: Dr Zaniletti is an employee of Children’s Hospital Association, Inc (Overland Park, KS); the other authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data