Emergency Department Pediatric Readiness and Mortality in Critically Ill Children

We performed a retrospective cohort study using 2013 data from the Agency for Healthcare Research and Quality’s Healthcare Cost and Utilization Project (HCUP). We used 2 HCUP databases: the State Emergency Department Databases (SEDD) and the State Inpatient Databases (SID). The SEDD contains administrative records from ED encounters that do not result in a hospitalization at the associated hospital, whereas the SID contains administrative records from all hospitalizations, including those originating in the ED of the associated hospital. For this study, we used HCUP data from Florida, Iowa, Massachusetts, Nebraska, and New York. These states represent a geographically diverse population, provide both SEDD and SID data, and contain unique patient and hospital identifiers, allowing patients to be tracked across hospitals over time and enabling a complete picture of episodes of emergency care, including patients transferred between hospitals. These states were specifically chosen for our analysis because the unique patient identifier is not available in all states that provide data to the HCUP.

We augmented HCUP data with data from 3 ancillary hospital-level sources: the 2013 Centers for Medicare and Medicaid Services Healthcare Cost Report Information System, which contains data on hospital size, teaching status, and location; the 2011 American Hospital Association Annual Survey, which contains data on the presence of a pediatric ED, pediatric inpatient unit, and PICU; and the 2013 NPRP assessment, which contains data on ED pediatric readiness.⁹  The NPRP assessment data were collected between January 1, 2013, and August 31, 2013, and reflect hospital pediatric readiness and capabilities experienced by patients in the 2013 HCUP databases.

Briefly, the NPRP assessment is a 55-question online questionnaire based on the 2009 multisociety guidelines for the care of children in the ED.^8,9  The assessment evaluates hospital pediatric readiness in the following domains: administration and coordination; staffing; quality-improvement efforts; pediatric patient safety, policies, or protocols; and equipment, supplies, and medications. One-third of points are related to equipment and supplies. Full weights for each item assessed are available in Supplemental Table 4. The assessment was completed online via a hyperlink emailed to ED nurse managers at 5017 US hospitals with a 24-hours-per-day ED. The result was an overall measure of pediatric readiness known as the weighted pediatric readiness score (WPRS), a summary score derived from a modified Delphi process in which 24 of 55 questions were weighted and normalized to a 100-point scale. A WPRS of 100 indicates the ED meets all the essential elements for pediatric readiness on the basis of published guidelines.⁸ 

Our primary population of interest was pediatric patients presenting to an ED with critical illness. We focused on critically ill children because they require timely treatment and specialized care, potentially making their outcomes more sensitive to ED pediatric readiness.^11,12  To identify this population, we created unique episodes of care by linking temporally adjacent records from the SEDD and SID for individual patients (ie, records in which the discharge date of the first record was equal to either the admission date or the day before the admission date of the next record). Episodes began either in the SEDD or SID and ended with hospitalization, discharge, or death.

After constructing unique episodes of care, we only included episodes involving general short-stay adult and children’s hospitals. We further restricted the analysis to patients aged <18 years and those with critical illness, defined as either ICU admission or death during the episode. We included decedents in our definition of critical illness to capture patients who died in the ED before ICU admission. Finally, we excluded episodes not originating in an ED, episodes containing 3 or more administrative records because the complex trajectories of these patients may obscure the relationship between the readiness of the presenting hospital and patient outcomes, episodes originating in a hospital that did not respond to the NPRP assessment, and episodes with missing data necessary for the analysis.

The primary exposure variable was the pediatric readiness of the initial ED for each episode, defined as the hospital-specific WPRS from the 2013 NPRP assessment.⁹  If patients were transferred between hospitals, only the WPRS of the initial hospital was evaluated. The primary outcome variable was mortality during the episode. Patient-level characteristics included age, race, sex, the presence of complex chronic conditions, and severity of illness. The presence of complex chronic conditions was based on International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnosis codes by using a published algorithm.¹³  Severity of illness was defined by using the Pediatric Emergency Care Applied Research Network Severity Classification System, a validated score ranging from 1 (least severe) to 5 (most severe) based on the initial primary diagnosis.¹⁴ 

Hospital-level characteristics included hospital size, teaching status, geographic location, pediatric emergency volume, and the availability of a dedicated pediatric ED, pediatric inpatient unit, or PICU. We defined hospital size using total number of beds, categorized as small (<100), medium (100–250), and large (>250). We defined teaching status using the ratio of resident full-time equivalents to beds, categorized as nonteaching (no residents), small teaching (ratio <0.2), and large teaching (ratio 0.2 or higher). We defined geographic location on the basis of metropolitan statistical area population, categorized as small (<100 000), medium (100 000 to <1 million), and large (≥1 million). We defined annual pediatric emergency volume using criteria from the NPRP assessment, categorized as low volume (<1800 annual pediatric visits), medium volume (1800–4999 annual visits), medium-high volume (5000–9999 annual visits), and high volume (≥10 000 annual visits). We identified the availability of a dedicated pediatric ED, pediatric inpatient unit, or PICU directly from American Hospital Association data.

For our analysis we categorized hospitals by quartiles of pediatric readiness. We compared hospital and patient characteristics across quartiles using χ² tests for categorical data and Kruskal-Wallis tests for continuous data. We examined the independent relationship between pediatric readiness and episode mortality by fitting a multivariable logistic regression model in which the dependent variable was mortality and the independent variables were WPRS quartile (modeled as an indicator covariate) as well as a priori–specified patient-level confounders, including age, modeled as a continuous variable; complex chronic conditions,^13,15  modeled as individual indicator covariates; and severity of illness, modeled as an ordered categorical variable. We accounted for clustering by hospital using robust SEs. Results from this model are presented as adjusted odds ratios, with hospitals in the lowest quartile of readiness being the reference category.

In addition to the quartile analysis, we modeled the relationship between mortality and WPRS as a continuous variable using fractional polynomials. Fractional polynomials are a method of operationalizing continuous variables without a priori specifying their functional form, allowing the data to determine the best-fitting polynomial function and avoiding the use of arbitrary cut points.¹⁶  Our goal with this modeling was to provide a more nuanced understanding of the relationship between the WPRS and mortality. This model was fit similarly to the regression model described above with the exception of using fractional polynomials for the WPRS. The results are shown in a figure of the relationship between the predicted mortality rate (on the y-axis) and the WPRS (on the x-axis).

We also performed analyses on 3 prespecified diagnostic conditions: cardiac arrest, sepsis, and traumatic brain injury (TBI), defined by using previously used ICD-9-CM diagnosis and procedure codes (Supplemental Table 5).^17–19  We chose these conditions because these patients are likely to benefit from timely, specialized care in the ED. We also considered examining asthma as a fourth diagnostic condition,²⁰  but preliminary analyses revealed patients with asthma experienced low mortality, precluding further analysis. For each diagnostic condition, we fit separate logistic regression models among patients with each condition. As in the primary analysis, we analyzed WPRS both as quartiles (using the same quartile cut points as the primary analysis) and as fractional polynomials.

Statistical analyses were performed by using Stata version 15.1 (Stata Corp, College Station, TX). All statistical tests were 2 sided, and significance was set at P < .05. This project was deemed to be exempt human subjects research by the University of Pittsburgh Human Research Protections Office.

Of 37 304 episodes of pediatric critical illness in the study sample, 20 483 episodes in 426 hospitals met all inclusion criteria (Fig 1). Most exclusions were for inpatient episodes not originating in an ED (13 775 episodes), whereas a smaller number (2698 episodes in 104 hospitals) were excluded because the hospital did not respond to the NPRP assessment. A comparison of hospital characteristics between NPRP assessment responders and nonresponders is shown in Supplemental Table 6. Compared with nonresponding hospitals, responding hospitals tended to be smaller and located in smaller communities.

Among hospitals in the final sample, the WPRS ranged from 29.6 to 100.0 with a median of 74.8 (interquartile range: 59.3–88.2). Hospital characteristics by quartile of the WPRS of the presenting hospital are shown in Table 1. Compared with hospitals with higher scores, hospitals with lower scores were more often small hospitals, nonteaching hospitals, and located in smaller communities. Hospitals with lower scores were also less likely to have a dedicated pediatric ED, pediatric inpatient unit, and/or PICU. Patient characteristics by quartile of the WPRS of the presenting hospital are shown in Table 2. Less than 5% (4.3%; n = 879) of patients presented to hospitals in the lowest quartile, whereas 68.6% (n = 14 059) presented to hospitals in the highest quartile. Compared with patients presenting to hospitals with a higher WPRS, patients presenting to hospitals with a lower WPRS were older and had fewer complex chronic conditions. Unadjusted mortality was 3.4% of those patients presenting to hospitals in the highest quartile of WPRS compared with 11.1% of those presenting to hospitals in the lowest quartile of WPRS (P < .001 for trend). Among decedents, most died during the initial ED encounter rather than as an inpatient or after transfer to another hospital (Supplemental Table 7).

In our regression analysis adjusting for patient characteristics, we found that presentation to a hospital in a quartile with higher readiness scores was associated with lower odds of death (Table 3). This relationship was dose dependent, with mortality decreasing as the quartile of pediatric readiness score increased.

The relationship between WPRS modeled as a continuous variable and predicted mortality rate is shown in Fig 2. As in the quartile analysis, we found that mortality generally decreased as WPRS increased. An exception was for hospitals with a WPRS <50, for which an increasing score was associated with increasing mortality. Investigating this finding further, we found that among the hospitals in our cohort, only 61 (14.4%) had scores at or below this inflection point. These hospitals had an average of 5.6 ± 7.9 episodes (range 1–49), whereas hospitals above the inflection point had an average of 51.0 ± 113.7 episodes (range 1–851).

The results of the condition-specific analyses are presented in Table 3 and Fig 3. These results are consistent with our primary analysis and show decreasing risk-adjusted mortality with an increasing readiness score, although because of small sample sizes, only the results for TBI were statistically significant.

In a multistate cohort study of pediatric ED encounters, we found that critically ill children presenting to hospitals with a higher WPRS had lower odds of death compared with those presenting to hospitals with a lower WPRS. Similar findingswere also observed when we examined condition-specific groups of critically ill children. The results of our analysis using fractional polynomials suggested that mortality also decreased in hospitals with low readiness scores. However, the smaller numbers of hospitals and episodes below the inflection point and the associated wide confidence intervals indicate that this finding should be interpreted with caution.

Although several previous studies demonstrate that hospitals vary widely in their ability to care for pediatric emergencies,^6,21,22  to date, few studies have evaluated the effect of pediatric readiness on patient outcomes. Previous data suggest that care at specialized pediatric trauma centers is associated with lower mortality after severe traumatic injury,^4,23  and management in high-volume PICUs is associated with lower mortality in pediatric critical illness.^24,25  Our study expands on this literature by demonstrating an association between hospital-level capabilities for pediatric emergencies and mortality for critically ill children.

Our findings have important implications for pediatric emergency and critical care delivery in the United States. Primarily, our findings suggest that patient outcomes may be improved by increasing the readiness of hospitals to care for pediatric emergencies. As detailed in previous work, the most common reasons for low readiness scores are lack of implementation of ED policies dedicated to children, lack of quality-improvement efforts, and absence of a dedicated pediatric emergency care coordinator,^9,26  all of which are plausibly related to patient outcomes. Other factors likely to impact outcomes may include availability of pediatric resuscitation equipment, use of a medication chart or length-based tape for medication dosing, and written interfacility guidelines for transfer to a center with more pediatric capabilities.

Efforts to address these deficits might include local and state collaborations designed to provide shared resources, such as educational materials, pediatric-focused policies, collaborative quality-improvement activities, and appointment of pediatric emergency care coordinators.^27–29  Although these efforts are difficult and resource intensive, our findings suggest that they have the potential to improve clinical outcomes, particularly for children at high risk.

Our study also provides support for alternative approaches to expanding access to high-quality pediatric emergency care, such as regionalization and telemedicine. Under a system of regionalized pediatric emergency care, selected children at high risk would be systematically triaged and transferred to designated centers of pediatric readiness.³⁰  With pediatric emergency telemedicine, the expertise of regional referral centers is delivered to community hospitals via a real-time audiovisual link.³¹  Early data suggest a potential role for both regionalization and telemedicine as strategies for improving the outcomes of severely ill children presenting to small, rural, community hospitals.^32–34  More work is needed to identify the barriers to adopting these approaches and to develop strategies to effectively incorporate them into the pediatric emergency care landscape.^35–37 

There are limitations to our study. First, we used administrative data, which lack granular clinical risk-adjustment variables, such as vital signs and laboratory results, which could result in unmeasured confounding. However, high-readiness hospitals would be expected to see patients with greater complexity and severity of illness than other hospitals,²²  meaning our results are a conservative estimate of the benefit of presentation to a high-readiness hospital. In addition, there were baseline imbalances in ethnicity and severity of illness across quartiles of readiness scores, although not to such a degree that the multivariable models could not address confounding by these variables. Also, the use of mortality as an outcome measure may not be a sensitive marker of quality care in many pediatric conditions for which mortality rates are low. Future condition-specific analyses for conditions with low mortality rates should focus on other outcomes, such as length of stay or development of new morbidities. In addition, our definition of critical illness may have missed patients who were dead on arrival to the ED or critically ill patients admitted to hospitals without an ICU. Our study was further limited by the availability and quality of NPRP assessment data, which are available for 82.7% of hospitals in the United States and 80.8% of hospitals in our study. It is possible NPRP responders differed from nonresponders in ways that could bias our results, making it difficult to draw conclusions about hospitals that did not participate. In addition, the assessment is a self-reported reflection of hospital pediatric capabilities, and no validation was performed to confirm the accuracy of responses.

Presentation to a hospital with lower pediatric readiness is associated with increased risk of death for children with critical illness. Efforts are needed to improve the pediatric readiness of EDs that care for children, ensuring critically ill children have access to timely, well-resourced, and effective emergency care.

ED

emergency department

HCUP

Healthcare Cost and Utilization Project

ICD-9-CM

International Classification of Diseases

9th Revision

Clinical Modification

NPRP

National Pediatric Readiness Project

SEDD

State Emergency Department Databases

SID

State Inpatient Databases

TBI

traumatic brain injury

WPRS

weighted pediatric readiness score