Impact of the AAP Guideline on Management of Brief Resolved Unexplained Events

This was a retrospective, observational, cohort study using the Pediatric Health Information System (PHIS; Children's Hospital Association, Lenexa, KS) database. The PHIS database contains deidentified administrative and billing data from 49 tertiary care children’s hospitals and accounts for roughly one fifth of the pediatric hospitalizations in the United States. Data quality and integrity are ensured through a joint effort between the Children’s Hospital Association and participating hospitals.

Case ascertainment is challenging in BRUE. There are known inaccuracies in using International Classification of Diseases-10 (ICD-10) codes to identify patients presenting with a BRUE. The use of a single code for ascertainment can result in underclassifying BRUE cases. For optimal case ascertainment, we used a previously validated sampling strategy.⁶  As described in that study, use of a principal diagnosis of BRUE for identification can be appropriate for lower-risk BRUE evaluated in the emergency department (ED). On the other hand, among hospitalized patients, clinical characteristics of BRUE may be used to maximize specificity and identify the group of patients presenting with BRUE who may or may not retain the diagnosis at discharge. We separated the 2 groups for investigative purposes on the basis of this previously reported strategy: BRUE and clinical features of BRUE. The ICD-10 codes used to identify ALTE/BRUE and clinical features of BRUE are listed in Supplemental Table 6. For simplicity, we present findings for the patients with the ICD-10 codes for ALTE/BRUE in the main manuscript and the findings for the patients with the codes for the clinical features of BRUE in the supplement.

For patients aged 2 days to ≤365 days, we included all ED encounters and inpatient/observational encounters admitted via the ED between January 1, 2012, and December 2019 with a primary or secondary ICD-10 diagnostic code for ALTE/BRUE or a clinical feature of BRUE.⁷  We excluded patients with complex chronic conditions, transferred from an outside hospital, nonindex encounters, and from hospitals that inconsistently participated in PHIS and had incomplete data during the study period.⁸ 

The AAP BRUE CPG was originally published in May 2016. We grouped encounters into 2-time cohorts on the basis of hospital discharge date: preguideline (January 1, 2012–January 31, 2016) and postguideline (July 1, 2016–December 31, 2019). The 6-month period between January 31, 2016, and July 1, 2016, was not included because this was considered a wash-out period. Four-months before CPG publication was selected for the wash-out period because the AAP began communicating the recommendations at that time. The 2 months after CPG publication was selected to allow for distribution and assimilation of the CPG.

The primary outcomes were rates of diagnostic tests and treatments that the AAP BRUE CPG recommended against in the management of patients with BRUE. The tests included cardiopulmonary (echocardiography, chest radiograph, polysomnogram), neurologic (EEG, neuroimaging (computed tomography, MRI, or ultrasonography), infectious disease (CBC, blood culture, CSF analysis or culture, urinalysis or culture, pertussis testing, viral respiratory screen), GI (upper GI series, pH study, endoscopy, swallow studies), and metabolic (blood gas, electrolyte studies, serum ammonia, blood lactate, serum amino acid, serum acylcarnitine). Of these diagnostic tests, 16 were strongly recommended against and 4 were weakly recommended against in the BRUE clinical practice guideline. We also evaluated the change in electrocardiogram and pertussis testing because the CPG weakly recommended that these be performed in lower-risk patients. We evaluated the change in use of acid suppression therapy, the medication recommended against in the BRUE CPG. The acid suppressor medications included were omeprazole, lansoprazole, pantoprazole, and ranitidine. It was on the basis of any use in the hospital.

Secondarily, we evaluated other measures of health care utilization, including admission rate, LOS, and costs. Costs were estimated from charges using hospital/year specific ratios of cost to charge and adjusted for inflation to 2019 US dollars using the medical component of the Consumer Price Index.⁹ 

Patient characteristics were summarized by using geometric means (for continuous variables) and percentages (for categorical variables) and compared across time cohorts using Wilcoxon Rank-Sum tests and χ² tests as appropriate. Segmented regression analysis with interrupted time series was used to evaluate the outcomes over time, with the publication of the 2016 CPG as the event point.¹⁰  The analysis measured rates of change in outcomes (slope) during the 2 time periods, the change in slope between time periods, and the level of change at the time of the CPG. All models were adjusted for age, gender, payer, region, and severity using the Hospitalization Resource Intensity Score for Kids and included a random intercept for each hospital to account for clustering of patients within hospitals.¹¹  The denominator included the total number of ED encounters. They are expressed as rates. All statistical analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC), and because we performed modeling on 26 different outcomes, we Bonferroni corrected the significance level to be P values < (.05/26) = .002 when considering statistical significance. The institutional review board approved this cohort study upon determination that it did not constitute human subjects’ research.

Over the study period, 36 hospitals contributed data for 112 029 encounters meeting initial inclusion criteria. Of these, 40 355 were excluded because of nonindex encounters, complex chronic conditions, transfers from outside EDs, and nonconsistent hospitals. Of the remaining 71 674 encounters, 27 941 were identified using the ICD-10 codes for BRUE and 43 733 were identified using the codes for clinical features of BRUE. The patient characteristics and outcomes of the study cohort identified using the codes for the clinical features of BRUE are presented in Supplemental Tables 4 and 5.

Characteristics of the final 27 941 encounters for BRUE are included in Table 1. The preguideline period included 13 948 encounters and the postguideline included 13 993 encounters. The overall median age was 5 months.

The admission rates were decreased from 87.9% in the preguideline period compared with 74.2% in the postguideline post-CPG (Tables 2 and 3). The quarterly admission rate continued to decrease over the postguideline period (slope of −0.41%, P = .072), but this did not reach statistical significance and was not different from preguideline quarterly changes (Table 3).

Chest radiograph use decreased by −5.12% (P < .001) after the washout period, with a continued quarterly decrease in the postguideline period (−0.4%, P < .001) (Fig 1). Echocardiography use decreased (−0.4%, P = .004) after the washout period. Electrocardiogram use was not associated with an early change after the washout period and continued to increase (0.36%, P = .016) quarterly over the postguideline period. There was no discernable change in polysomnogram use.

EEG use decreased by −0.58% (P = .016) after the washout period but there was no significant decrease early after the washout period or change in trend in head imaging evaluation.

There was an early decrease in CBC use (−3.69%, P < .001) after the washout period and the declining trajectory slowed over the postguideline period. There was no early change or change in trend for blood culture use. CSF analyses decreased −0.39% (P = .004) and the decrease slowed over the postguideline period. Urinalysis testing decreased by −1.49% (P = .003) early after the washout period. There was an early decrease in viral respiratory screening by −2.84% (P = .002) at the time of CPG release. Pertussis testing decreased by −1.44% (P < .001) without further changes in the postguideline period.

After the washout period, there was an early decrease in swallow evaluations by −0.78% (P < .001).

There was a decrease in electrolyte studies (−3.48%), blood lactate (−0.18%), and serum acyl carnitine (−0.01%) after the washout period. During the postguideline period, there was a continued decrease in electrolyte studies (−0.39% quarterly, P < .001).

There was no early decrease in use of acid suppression therapy after the washout period but there was a significant decrease in use in the postguideline period (−0.16% quarterly, P < .001). Cost adjusted for inflation decreased from $1850 to $1342 (P < .001). There was a significant decrease in costs in all categories (clinical, pharmacy, laboratory, imaging, supplies) as shown in Table 2. Over the study period, LOS decreased from 1.4 days to 1.3 days (P < .001). There was no increase in 7-, 30-, and 60-day revisits (Fig 2).

Findings for patients identified using ICD-10 codes for clinical symptoms of BRUE are shown in Supplemental Figs 3 and 4. Compared with the BRUE group, patients with clinical features of BRUE did not show early decreases in admission rates and utilization rates of tests and treatments.

Our study evaluates the association of 2016 AAP CPG on admission rate, diagnostic testing, treatment, cost, LOS, and revisits in patients with BRUE. We found a decrease in resource use that aligns with the recommendations in the CPG. Because the pattern of these findings is consistent with the CPG recommendations, this likely indicates some degree of adherence to the AAP recommendations.

Similar to other studies, we found a lower rate of admissions and testing in patients with BRUE or ALTE, even after evaluating more patients, a longer time, and using a more accurate case ascertainment strategy.^1,6  Our study further demonstrated decrease in cost and LOS after CPG publication.

CPGs are increasingly used to standardize care and optimize value of care in specific clinical conditions. There are potential benefits, harms, and limitations of using CPGs.^12,13  A systematic review of the quality of CPGs across a range of clinical conditions using the Appraisal of Guidelines, Research and Evaluation instrument showed the wide range of quality scores of the CPGs.¹⁴  Rigorously developed, evidence-based guidelines improve quality of patient care.¹⁵  Various factors such as applicability, clarity, editorial independence, rigor of development, scope, and stakeholder involvement have been described to affect the uptake and adherence to CPGs.¹⁴  In the context of BRUE CPG, having a clear executive summary, narrow focus using a specific definition and risk stratification, multidisciplinary involvement, rigorously developed key action statements using the AAP rating of evidence, and recommendations may have contributed to the good uptake and assimilation. The change of terminology from a vague, broad, and “life-threatening” ALTE definition to a more precise, clear, and narrow definition may have resulted in deimplementation of unnecessary, low-value nonevidence-based tests and resource use. The clarity of the scope and purpose of the BRUE CPG for the diagnosis, risk classification, and management recommendations may have helped clinicians. We observed a decrease in resources use as recommended in the CPG but before the CPG publication. Since there was not a previous guideline, the trend was likely because of a practice change based on emerging evidence from single and multicenter studies on the low yield of diagnostic testing.¹⁶  It may also be partly because of the rapid specialization of pediatric hospital medicine. The publication of CPG was associated with a more rapid deceleration in the use of resources commiserate with the CPG recommendations.

It is interesting to note the decrease in resource use for so many patients, particularly considering the AAP CPG recommendations would not apply to patients meeting higher-risk criteria, which we now know to be >85% of patients presenting to the ED. This is likely because the criteria for higher risk in the 2016 guidelines have since been demonstrated to overestimate risk for the vast majority of patients.¹⁷  In other words, we now know that many higher-risk patients according to the AAP criteria are not at an elevated risk. Although the AAP CPG recommendations were meant to apply to lower-risk criteria patients, it is likely that clinicians have begun to apply the recommendations to a subset of the higher-risk patients.

Some tests did not seem to be impacted by the recommendations in the CPG. This may be because of barriers to deimplementation of common but nonevidence-based practices (eg, antacid medications for gastroesophageal reflux) that were not measured in this study. These factors include clinician, patient, and family preferences and risk tolerance. The benefits of reducing unnecessary tests may need to be balanced with rare, missed diagnosis, as well as caregiver preferences on testing and hospitalization.

Our study had several limitations. First, it was a retrospective, observational cohort study using administrative data. Discharge codes may not accurately reflect the clinical encounter and may under and overclassify cases.⁵  Next, coding practices may vary across hospitals and time. This is especially true for BRUE, which is a clinical diagnosis based on history and physical examination.¹  A recent study showed that the ICD-10 ALTE/BRUE code alone was optimal for case ascertainment among ED patients, whereas clinical characteristics of BRUE are preferred for hospitalized patients.⁶  Our study used both the cohorts and they showed similar findings. Third, ICD-9 was transitioned to ICD-10 in October 2015 and may have led to additional misclassification of cases. ICD-9 codes for ALTE have not been validated. However, studies have validated the ICD-9 to ICD-10 conversions.¹⁸  The precohort and postcohort are likely different case mixes, which may have contributed to some amount of the observed differences in utilization.

Fourth, we were unable to determine what proportion of the patients qualified as lower-risk BRUE, the focus of the BRUE CPG. However, recent research indicates that most patients qualifying for the AAP higher risk are still at low risk for recurrence or a serious diagnosis. Fifth, the results of this study may not be generalizable to community hospitals.¹⁹  Finally, the study cannot establish a causal relationship of publication of the CPG and decreased resource utilization; however, use of interrupted time series helps account for secular trends.¹⁰ 

The 2016 AAP BRUE CPG was associated with a significant reduction in hospitalizations, diagnostic testing, and medication use among patients cared for in a large representative sample of children’s hospitals. Future prospective studies in a diverse group of hospitals will help identify factors associated with deimplementation and guideline adherence, and better understand associations with improved clinical outcomes.

This project was done under the mentorship of the Health Services Research Academy of the Children’s Hospital Association.