Risk Prediction After a Brief Resolved Unexplained Event

This was a clinical prediction model derivation and validation study. This study was reported according to the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) statement.¹⁰ 

This study was nested within a previously described research study and quality improvement collaborative^2,11  and included infants presenting to 15 emergency departments (EDs) across the United States between October 1, 2015, and September 30, 2018, and April 1, 2019, and June 30, 2020.

Data were retrospectively collected at these hospitals as part of the BRUE Research and Quality Improvement Network.¹⁰  Research ethics approval was obtained from each participating center. Potentially eligible patients were identified using the Pediatric Health Information System.¹¹  Hospitals not participating in the Pediatric Health Information System completed a similar query of their internal databases.¹⁰  Administrative data were validated and supplemented with a review of medical records by local trained investigators.

We identified potential BRUE cases by reviewing medical records for infants (<1 year of age) with a discharge diagnosis of an ALTE or BRUE, a diagnosis used to describe characteristics of the event, or a diagnosis to explain the event.¹²  Details on the identification of eligible patients have been published previously.¹³  Patients were included if they met the AAP guidelines criteria for a BRUE.⁵  Patients were excluded if they: (1) presented for care for an unrelated reason; (2) were transferred from another institution (to ensure the reliability of collected data); (3) had a known comorbid condition that was a potential cause of the event (eg, known genetic disorder); or (4) had objective symptoms or signs by history or on examination that precluded a diagnosis of BRUE (eg, fever).

The AAP guidelines risk factors and BRUE characteristics were evaluated as candidate predictors. These included age (≤60 days), prematurity (defined as <32 weeks’ gestation, or born between 32 to 38 weeks and corrected to <45 weeks), indicated use of CPR, recurrent events, event clusters, an event lasting ≥1 minute, and abnormal medical history such as previous hospitalization or underlying medical condition.⁵  Additionally, sex and event characteristics were evaluated (eg, color change, abnormal respiratory pattern, and change in tone). We did not assess family and social history because these historical risk factors were inconsistently documented.

The primary outcome of the study was the presence of a serious underlying illness, defined as 1 or more conditions that could explain the presenting events, requiring timely diagnosis, and in which a delay could potentially cause significant morbidity or mortality. Examples include epilepsy requiring treatment with antiepileptics, serious bacterial infections requiring antimicrobial agents, airway abnormalities requiring surgery, and child abuse.¹²  These diagnoses could have been made during the first presentation or at a later encounter. The timeline for monitoring differed between the research study and the quality improvement collaborative.^2,10  For the first cohort (2015 to 2018), patients were monitored for this outcome until age 1 year. For the second cohort (2019 to 2020), patients were monitored up to age 1 year or for 3 months, whichever came first.

As a secondary outcome, we assessed for event recurrence after assessment in the ED for the index BRUE event. We limited the monitoring for recurrent events to the duration of observation in the ED or during hospitalization for the index event.

Unadjusted odds ratios (OR) were calculated for identified risk factors and each of the outcomes.

Given the number of potential predictors of interest (13) and the limited number of patients with the primary outcome (n = 150), we opted to maximize the number of patients in the derivation set. As such, the cohort was randomly split into a training set (80%) and a validation set (20%) while balancing for the lowest prevalent outcome, an underlying serious diagnosis (Supplemental Table 6). Using Classification And Regression Trees,¹⁴  we identified potentially relevant interaction terms to include in the multivariable model. The Classification And Regression Trees model also provided optimal split cut-offs to transform continuous variables into binary ones to simplify the final prediction rule (eg, age and gestational age). Variables that were not documented were assumed to be absent (eg, previous event) or normal (eg, term birth).

All potential predictors and relevant interactions were initially included in the multivariable logistic prediction model. Using a stepwise approach, the model with the lowest Akaike Information Criterion was selected.¹⁵  This generated a more parsimonious model and avoided overfitting by penalizing additional predictors. A similar prediction model was generated to assess the secondary outcomes.

The selected model was validated internally using the validation set. The goodness of fit was assessed using the Hosmer–Lemeshow test.¹⁶  We generated a receiver operating characteristic curve to evaluate discrimination. This was compared with the receiver operating characteristic of the current standard of care (the AAP guidelines higher-risk criteria).

All statistical analyses were performed by using SAS version 9.4 (SAS Institute, Inc, Cary, NC). Figures were generated with GraphPad Prism (version 9.3; GraphPad Software, La Jolla, CA). Median and interquartile range (IQR) were used to describe nonnormally distributed variables. Patients and BRUE characteristics were compared between patients with and without each outcome of interest using χ² tests and Wilcoxon rank test for categorical and continuous variables respectively. All statistical tests were 2-sided, and a P < .05 was considered significant.

A total of 8655 patients were identified using the search strategy. After weighted selection and chart review, 3283 patients were deemed eligible (Supplemental Information). The median age of the patients was 48 days (IQR: 18–116), and 51.9% (n = 1705) were female (Table 1). The majority were born at term (74.0%, n = 2429) and with no other abnormal medical history (69.8%, n = 2290). A total of 91.5% (n = 3005) of patients met the AAP higher-risk criteria, most commonly for age ≤60 days (57.1%, n = 1874), history of a similar event (36.3%, n = 1192), or history of event clusters (30.3%, n = 996). Most patients presented with breathing changes (69.4%, n = 2278), and 50.1% (n = 1645) had a color change (cyanosis or pallor).

Among this cohort, 2063 patients (62.8%) were hospitalized, 150 (4.6%) were diagnosed with a serious underlying condition, and 469 (14.3%) had a recurrent event during the index visit. The most commonly identified serious diagnosis was seizure or epilepsy treated with antiepileptics in 44 patients (29%), followed by airway abnormalities treated with surgery in 19 (13%), and severe dysphagia or gastroesophageal reflux disease treated with nasogastric tube feeding in 17 (11%) (Supplemental Table 4). Only 1 death was recorded during the monitoring period in this cohort. The patient was discharged from the hospital after the first BRUE and then re-presented to the index hospital 1 week later after cardiopulmonary arrest at home. No cause identified after autopsy.

In the cohort, 150 patients had a serious underlying diagnosis. Presence of at least 1 higher-risk AAP criteria was not significantly associated with a serious diagnosis (OR: 1.93, 95% confidence interval [CI]: 0.94–4.18, P = .09) (Table 1). The AAP criteria demonstrated a sensitivity of 95.3%, specificity of 8.6%, and area under the curve (AUC) of 0.52 (Table 2). Among these criteria, the following were associated with the outcome of a serious diagnosis: history of a similar event (OR: 1.86, 95% CI: 1.34–2.57, P < .001), history of event clusters (OR: 1.66, 95% CI: 1.19–2.30, P = .003), the need for CPR (OR: 2.38, 95% CI: 1.09–5.53, P = .04), and abnormal medical history (OR: 1.92, 95% CI: 1.37–2.66, P < .001). In contrast to the AAP criteria, patients >60 days of age were more likely to have a serious underlying diagnosis (OR: 1.43, 95% CI: 1.03–1.98, P = .03).

Patients with a serious underlying diagnosis were more likely to exhibit a color change (60.7% vs 49.6%, P = .008) or altered responsiveness (46.0% vs 35.0%, P = .006) and were less likely to have an abnormal breathing pattern (59.3% vs 69.9%, P = .006).

In the cohort, 469 patients had a recurrent event (14.3%). Regarding this secondary outcome, the AAP guidelines demonstrated a sensitivity of 98.9%, a specificity of 9.7%, and AUC of 0.54 (Table 2). Presence of at least 1 of the higher-risk AAP criteria increased the odds of event recurrence (OR: 5.00, 95% CI: 2.80–9.00, P < .001) (Supplemental Table 5). The following AAP higher-risk criteria were associated with event recurrence: prematurity (<32 weeks, or between 32–38 weeks and corrected to <45 weeks) (OR: 1.53, 95% CI: 1.22–1.92, P < .001), history of a similar event (OR: 2.34, 95% CI: 1.92–2.86, P < .001), event clusters (OR: 3.70, 95% CI: 3.02–4.53, P < .001), and abnormal medical history (OR: 1.58, 95% CI: 1.29–1.93, P < .001). Event recurrence was more common in those ≤60 days of age (OR: 1.40, 95% CI: 1.14–1.71, P < .001).

As for BRUE characteristics, patients with event recurrence were more likely to exhibit a color change (57.1% vs 48.9%, P < .001) and less likely to have an abnormal breathing pattern (62.8% vs 70.8%, P < .001) or change in tone (40.1% vs 45.9%, P = .02).

Using a training set of 2627 patients (80% of the cohort), we derived a multivariable logistic regression model with the primary outcome as the dependent variable. The following variables were retained by the model as predictors (Table 3, Supplemental Table 7): age as a continuous variable, history of similar events, and abnormal medical history. The derived model was internally validated using a validation set of 656 patients (20.0%). The model had an AUC of 0.64 (95% CI: 0.59–0.70) in the training set and 0.61 (95% CI: 0.49–0.72) on the validation set, which was higher but not significantly greater than the AAP guidelines value of 0.51 (95% CI: 0.46–0.56) (P = .10) (Fig 1A).

A similar model was developed for the secondary outcome of event recurrence (Table 3 and Fig 1B). For this model, the following variables were retained: prematurity (<32 weeks or 32–38 weeks and corrected to <45 weeks), history of a similar event, history of event clusters, color change, tone change, and abnormal respiratory pattern. Both abnormal respiratory pattern and tone change had a negative association. The model had an AUC of 0.68 (95% CI: 0.62–0.74), compared with 0.54 (95% CI: 0.52–0.57) for the AAP guidelines in the same cohort (P < .001; Table 2, Fig 1B).

Using the largest cohort of BRUE patients described to date, we derived and validated clinical prediction rules that considered patient and BRUE characteristics to determine the risk of patient- and family-centered outcomes, event recurrence, or a serious underlying diagnosis. Compared to the current AAP guidelines, the derived rules showed better discrimination. Rather than rely on arbitrary cut-offs for defining lower- and higher-risk groups, we have incorporated these calculations into a shared decision-making tool to guide clinicians and families (https://www.mdcalc.com/calc/10400/brief-resolved-unexplained-events-2.0-brue-2.0-criteria-infants [Supplemental Fig 3]).

Among 3283 patients with a BRUE and no ongoing symptoms at the time of assessment, a diagnosis requiring prompt recognition, in which delays could cause morbidity and mortality, was only identified in a small proportion of patients (4%). This group was very heterogeneous in terms of etiologies, leading to a low yield of testing as discussed in other reports.^3,8,9  Furthermore, only 14% of patients experienced a recurrent event during the index visit. The low risk of a serious underlying diagnosis or event recurrence was similar to those reported previously.^7,17  Death remains an uncommon outcome post-BRUE. One patient died during follow-up, which was not significantly higher than the risk in a general infant population.^18,19  Although a serious diagnosis may not be identified in the acute setting, a clinical prediction rule may help heighten awareness of the need for closer follow-up and further investigations in the outpatient setting.^12,17 

Identifying the minority of infants in whom these brief events represent a symptom of a serious underlying pathology has been an ongoing challenge over the past decades. The current AAP guidelines were developed with the aim of identifying very low-risk patients, in which admission or invasive investigations are unlikely to be beneficial. This current AAP framework was based on the ALTE literature available at the time. However, recent evidence suggests that the ALTE and BRUE populations are not comparable when considering elevated risk.²⁰  This study and other reports published specifically on BRUE cohorts have shown that patients who meet the higher-risk criteria do have increased odds of having an adverse outcome.⁷  However, the majority of our cohort fit this higher-risk definition (92%), similar to previous reports (67% to 95%).^2,6,7,17  Overall, this translated into high sensitivity of the AAP guidelines for both event recurrence and serious diagnosis and low specificity and positive predictive value.

Our study attempted to improve the AAP higher-risk criteria and quantify the risk associated with individual risk factors, specifically looking at BRUE and patient characteristics. Earlier reports have suggested a lack of association between younger age and a serious diagnosis.²  Interestingly, we found that age ≤60 days was positively associated with event recurrence, but negatively associated with a serious diagnosis. This might be because of BRUE more often being caused by normal immature physiology in young neonates, leading to event recurrence without a serious underlying pathology.²¹  Alternatively, young infants are more likely to be hospitalized, leading to increased recognition and reporting of event recurrence.

Our clinical prediction rules outperformed the AAP guidelines on both the training and internal validation sets. A major advantage of these rules is that they can be incorporated into shared decision-making tools for physicians and families.²²  The AAP has highlighted the importance of family-centered care and its benefits, including a stronger alliance with the family in promoting their child’s health, as well as improved follow-up on recommendations.^23–28  It has been shown that providers appreciate the benefits of shared decision-making, specifically in a context like BRUE in which >1 valid approach exists, to avoid low-diagnostic yield and potentially harmful testing.^27,29  The risk tolerance between parents and providers is variable.^30,31  Research on families’ experience in the context of BRUEs supports wide variability. Even among those labeled as low-risk, some parents felt uncomfortable returning home without monitoring in the hospital or further investigations.^30,32,33  This further highlights the need for shared decision-making.^12,22  Using this tool, providers can emphasize that the majority of patients do not have an acute life-threatening illness and a low risk of recurrence when involving the family in discussions around the benefits or harms of admission or further investigations.^22,33–36  A similar tool developed by PECARN (Head CT Choice Decision Aid) educates caregivers on concussion, potential risks, and the harms and benefits of close observation versus head imaging.³⁷  In a randomized controlled trial assessing the utility of the head computed tomography tool, parents reported greater knowledge, less decisional conflict, and greater physician trust. Using the tool required only an additional 2 minutes from providers and was associated with lower health care utilization costs.

Our study has some limitations given its retrospective nature and reliance on chart review. First, we may have underestimated the association with certain risk factors because they were not accurately documented in the medical record. Specific risk factors that were inconsistently documented (family and social history) were excluded from the models. Second, our outcome of event recurrence was limited to the duration of observation in hospital. The true risk of recurrence is likely higher. Additionally, we may have overestimated the risk in young patients or those with multiple events because they were more likely to undergo hospital admission and further work-up. Third, the heterogeneity of etiologies identified in the subset of patients with an underlying disorder might have precluded the identification of risk factors associated with a specific disorder (eg, epilepsy and altered responsiveness). Fourth, we have limited our primary outcome to conditions in which a delay could potentially cause significant morbidity or mortality. Patients at lower-risk might have a diagnosis not requiring prompt management and may benefit from closer follow-up. The primary outcome was also limited to diagnoses that can potentially explain the BRUE. Nonetheless, it is possible that the BRUE presentation and the identified diagnosis were unrelated in some patients and that we have overestimated the risk of diagnoses specifically causing a BRUE. Finally, as we aimed to provide a prediction risk calculator rather than specify cut-offs to define risk groups, we are unable to provide a measured sensitivity and specificity.

Most infants presenting with a BRUE do not have a serious underlying pathology requiring prompt diagnosis and treatment. Using the largest BRUE cohort described to date, we assessed specific risk factors related to patient and BRUE characteristics and created clinical prediction rules to assist pediatricians and families in communicating about BRUE. Future research should focus on implementing these tools prospectively, confirming their prediction validity for patients with a serious underlying diagnosis and assessing their benefits to families’ understanding and comfort.

We thank all the members of the BRUE Research and Quality Improvement Network and the MDCalc team for their invaluable contributions.