Prediction Model for Croup Admission Need Free

Data were collected retrospectively from the electronic medical record for patients presenting with croup to the ED (ED discharge International Classification of Diseases, 10^th Revision diagnostic code for croup, J05.0) between the ages of 1 month and 17 years at 2 tertiary pediatric hospitals and 1 community hospital between October 2019 and September 2020. The study was approved by the institutional review board of each participating hospital.

The patient characteristics collected were age, sex, race, and preexisting conditions defined as history of prematurity, intubation, tracheal anomalies, asthma, or croup. The ED visit characteristics collected were ED length of stay (LOS), any dexamethasone administration before ED presentation in the previous 24 hours, time to dexamethasone administration in the ED, the number of racemic epinephrine (RE) doses received in the ED, respiratory syncytial virus and influenza testing results when done, and any ED revisit or readmission within 72 hours of the initial ED encounter. Finally, the Westley croup score¹⁰  (WCS) at presentation and the WCS at the time of ED disposition were calculated on the basis of the ED vital signs, nursing, and provider notes as a numerically quantifiable measure of croup severity ranging from 0 to 17. The clinical factors comprising the WCS are level of consciousness, cyanosis, stridor, air entry, and retractions. If the clinical notes did not comment on any single WCS factor, then that factor was scored as a 0 with the assumption that abnormal findings will be adequately documented. It is important to note that all the EDs participating in the current study follow an institutional croup management protocol with only minor variations between each institution. For the subset of patients admitted from the ED (observation/inpatient encounter), the number of additional RE doses administered, intubation, oxygen or Heliox administration, and PICU transfer were also recorded, representing inpatient interventions. Dexamethasone administration in the inpatient setting was not considered an inpatient intervention, given the fact that dexamethasone could also be administered at the time of discharge or postdischarge, if at all, particularly if the LOS was <24hrs. In addition, the practice of administering a second dose of dexamethasone in the inpatient setting after the initial ED dose varies significantly between institutions and is not strongly supported by the current literature. Patients were excluded if there were missing data elements, the WCS could not be reasonably abstracted, or if admission was based on a diagnosis other than croup.

Descriptive statistics were used to summarize the demographic characteristics and characteristics of the ED visits as well as any related inpatient stays of the study population.

The risk prediction model used the patient and ED visit characteristics outlined above (with the exceptions of race and respiratory syncytial virus and influenza testing results) to predict the likelihood of needing admission and interventions. A modified WCS was also included as a potential predictor, in addition to the nonmodified WCS, as in the previously referenced pilot study⁵  and defined as follows: if the original score was 0, 1, or 2, the score was 0; otherwise, the modified score was the original score minus 2. This modification captures observations in our pilot study that the probability of the outcome was estimated to be the same if the original croup score was <3 and increased steadily for larger score values. L1-penalized (lasso) logistic regression¹¹  was applied, which adaptively performs variable selection to exclude predictors from the model if they were not useful in predicting whether a patient needed admission. Patients were defined as needing admission if they were (1) admitted and needed an inpatient intervention, as defined above, or (2) not admitted but revisited the ED within 72 hours of the initial ED encounter as a composite outcome. The tuning parameter for the lasso model was chosen by using 10-fold cross-validation to avoid overfitting the data.

The discriminative performance of the risk model was summarized by using the receiver operating characteristic (ROC) curve and its associated area under the curve (AUC). The calibration of the risk model was summarized by calculating the observed proportion needing admission for categories of predicted risk. Ten-fold cross-validation was used to obtain predicted probabilities from the model for each patient to account for using the same data to both develop and internally validate the prediction model.⁷  All analyses were performed by using R version 4.0.2.¹² 

A total of 2951 children were included in the study; 2745 (93%) were evaluated at the tertiary centers and 206 (7%) were evaluated at the community hospital, with 212 (7%) being admitted to the inpatient unit. The median age was 24 months, 64% were male, and 49% were non-Hispanic white. There were 68 patients (2.3%) who met the “need for inpatient admission” criteria; 49 patients were admitted to the inpatient unit and received an inpatient respiratory intervention and 19 patients were discharged from the hospital but revisited the ED within 24 hours, well within the 72 hour window defined by the study. Of those who revisited the ED, all were discharged from the ED and were not admitted. The remaining 2883 patients (98%) who did not meet the “need for inpatient admission” criteria included 163 patients who were admitted to the inpatient unit but did not receive any inpatient respiratory interventions and 2720 patients who were discharged from the hospital and did not revisit the ED within 72 hours (Fig 1). The most common inpatient intervention was RE administration. Of the 49 patients who were admitted and met the “need for inpatient admission” criteria, 5 were directly admitted or transferred to the PICU, 38 received RE only, 2 received RE and supplemental oxygen, and 4 received supplemental oxygen only. Patients who needed admission were less likely to be female (24% vs 37%) and more likely to have histories of prematurity (16% vs 5.8%), intubation (15% vs 2.7%), and tracheal anomalies (5.9% vs 1.5%) than those not needing admission (Table 1). The ED visit characteristics differed significantly between the 2 groups in ED LOS, presentation WCS, ED disposition WCS, dexamethasone administration before ED presentation, time to dexamethasone administration in the ED, and the number of ED RE doses administered (Table 2).

The selected predictive model for croup admission included 4 predictors: ED disposition (nonmodified) WCS, number of ED RE doses, previous dexamethasone administration, and history of intubation. The probability of needing admission for croup was equal to: e^w/(1 + e^w) where: w = −4.199 + (1.001 * [ED disposition WCS]) + (0.185 * [number of ED RE]) + (1.387 * [previous dexamethasone administration]) + (0.325 * [history of intubation]) with previous dexamethasone administration and history of intubation being equal to 1 when present and 0 otherwise. The positive weights associated with the predictors indicate that those with higher ED disposition WCS, more doses of ED RE, previous dexamethasone administration, or a history of intubation are more likely to need admission for croup.

An admission risk stratification was proposed based on the model’s calculated probabilities with the following risk categories defined (Fig 2): extremely low risk (0% to 1.95%), very low risk (>1.95% to 5%), low risk (>5% to 20%), moderate risk (>20% to 50%) and high risk (>50% to 100%). The risk categories were chosen to be increasingly wide as risk increased (eg, >5% to 20%, >20% to 50%, >50% to 100%) because of the limited number of patients who had >5% predicted risk of needing admission. The threshold of 1.95% was chosen to split the extremely low risk and very low risk categories because this threshold corresponds to the model performance that best approximates provider judgment (Fig 3).

The AUC for the predictive model’s ROC curve was 0.80 (Fig 3). The sensitivity and specificity of physician judgment were 72% (49/68) and 94% (2720/2883), respectively, with a positive predictive value (PPV) of 23% (49/212) and a negative predictive value (NPV) of 99% (2720/2739). The cut-point for the predictive model that most closely matched the performance of provider judgment was 0.0195, which translated into a sensitivity of 66%, specificity of 91%, PPV of 15%, and NPV of 99%. This cut-point is equivalent to admitting all patients except those who have all of the following: (1) an ED disposition WCS of 0, (2) no previous dexamethasone administration, (3) no history of intubation, and (4) at most 1 dose of ED RE. Figure 3 illustrates how our model improves on the best single predictor, ED disposition WCS, by identifying those with a WCS of 0 who still may be at risk for needing hospital admission. The calibration of the risk model is shown in Supplemental Fig 4.

The main objective of this study was to derive a reliable predictive model which could help reduce the number of avoidable admissions for croup while mitigating the risk of respiratory deterioration post-ED discharge. The study’s selected model appeared to strongly address the latter risk with its high NPV (99%), but its predictive performance as to whom qualifies for admission was low given its PPV (15%) and remained similar to provider judgment.

The authors of several previous studies have looked at the ED characteristics of patients presenting with croup in an attempt to identify reliable predictors for the need for inpatient interventions,^13–16  but all studies were limited to a single health care setting, were smaller in scope, and had a more limited focus. Consequently, no unified model has emerged for croup, and the issue of significant overutilization of inpatient facilities with its associated costs, both financial and in terms of health care human resources has remained. In addition, because most croup admissions fall under observation care,¹⁷  there is an added financial burden imposed on the families due to the increased out-of-pocket expenses, particularly if observation care takes place on inpatient units.¹⁸  This is also compounded by the potential loss of income secondary to reduced work hours while caring for their child, as well as an unquantifiable emotional cost stemming from the hospitalization experience.

With this current work, we attempted to address these limitations by including 3 different institutions with 2 tertiary centers and a community hospital, evaluating a broad range of potential predictors and relying on the subsequent statistical modeling to select the relevant ones. As a result, several predictors were identified, including, in order of decreasing weight, the administration of dexamethasone at an outside facility before ED presentation, the ED disposition WCS, history of intubation, and the number of ED RE doses administered. Interestingly, all these predictors had been individually identified by the authors of the previous studies referenced above except for previous dexamethasone administration, which is novel. In retrospect, this makes sense because the patients who have been administered dexamethasone before presenting to the ED were likely previously diagnosed with croup but had failed to respond to the gold standard dexamethasone pharmacological management of croup outside of the hospital setting¹⁹  and, therefore, presumably had a more severe case, an assumption supported by the larger weight assigned to that predictor in the derived model. Similarly, a worse disposition WCS would reasonably point to higher croup severity, as revealed in the previously referenced studies. As to the history of intubation predictor, its association with subglottic stenosis²⁰  could potentially exacerbate croup symptomatology in affected patients and hence contribute to the croup severity level. Although the number of ED RE doses is represented in the risk model, it does, however, carry the lowest weight and its overall effect on the probability of needing admission is small, and yet current ED croup management protocols frequently recommend admission after ≥2 RE doses despite the lack of supporting evidence in the literature.^4,5,13,21 

The strength of our predictive model is to identify those patients likely not to require admission for croup and subsequent respiratory interventions given its high specificity and NPV, providing a measure of reassurance for ED providers and caregivers to the decision to discharge a child with croup. On the other hand, the model’s moderate sensitivity and low PPV provide limited guidance as to the need for admission and do not exceed provider judgment in the ED. This latter finding may, however, reflect the fact that the majority of patients in the study were evaluated in the tertiary centers by experienced pediatric ED providers, and thus may not hold up in the setting of community or rural EDs in which experienced pediatric providers are less common.²² 

Such a model, particularly if made available as an online or app-based calculator with limited manual data input, could offer an automated starting point for the ED provider, particularly in the rural setting, to make a disposition decision for the croup patient. Whereas the extremely low, very low, and low risk categories would suggest a safe discharge to home disposition and the high-risk category would indicate a probable need for admission, it is the moderate risk category that would likely present the most difficulty. In this latter case, a shared decision-making process with the caregivers would be highly advised, taking into consideration caregiver comfort with home care and barriers to returning to the ED. In addition, the model could serve as a basis for ED croup protocol modification, particularly regarding the undue importance given to the number of ED RE doses administered when an admission decision is made.

There are several limitations to this study. Only 0.3% of patients (n = 8) fell into the high-risk category, which made its range necessarily broad. Because of the small number of patients with higher ED disposition WCS and a larger number of RE doses, we limited the modeling of these predictors to linear fits. Future larger studies could explore modeling these risk factors categorically to provide more flexibility, which could improve the discrimination and calibration of the model. In addition, given the retrospective design of this work, the WCS had to be derived from the clinical notes, a fact which may have introduced inaccuracies in the score determination and adversely affected the model’s characteristics. In addition, although an ED revisit within 72 hours of the initial encounter was one of the criteria for the need for admission, we did not track the management of the ED revisit encounters. Presumably, if the majority of the ED revisit encounters required only minimal interventions, then the validity of using ED revisits as one of the criteria of admission need could be challenged. Finally, the study population was heavily tilted toward the tertiary centers’ EDs and is likely not fully representative of the outcomes in community and rural centers.

These limitations notwithstanding, this predictive croup admission risk model could be an additional tool in the ED provider’s clinical decision-making armamentarium by facilitating the identification of patients at low risk of requiring inpatient respiratory interventions, which may in turn potentially help reduce the number of avoidable croup admissions, along with the associated health care expenditures, hospital utilization rates, and caregiver hardships. Future prospective studies are planned to validate this predictive model.