Associations Between Quality Measures and Outcomes for Children Hospitalized With Bronchiolitis

We conducted a prospective cohort study of children admitted with bronchiolitis to one of five geographically dispersed US children’s hospitals belonging to the Pediatric Research in Inpatient Settings Network between July 1, 2014, and June 30, 2016. Children were eligible for enrollment if (1) they were admitted with a diagnosis of bronchiolitis, (2) they were 2 weeks to 2 years old, and (3) their family spoke either English or Spanish. Children were excluded if they had any contraindications to routine care for bronchiolitis, including history of cardiac disease requiring baseline medication, anatomic airway abnormalities, reactive airway disease or asthma at ≥12 months of age, developmental delay, cystic fibrosis, neuromuscular disease, premature birth <32 weeks, bronchopulmonary dysplasia, chronic lung disease, or immunodeficiency. Subjects were excluded if they were admitted to the ICU at any point during their hospitalization. We excluded subjects admitted to the ICU to control for severity of illness so that the study population represented children who should receive routine care for bronchiolitis. If the child’s final discharge diagnosis was not bronchiolitis, the case was excluded from analyses. Detailed descriptions of recruitment and enrollment procedures are included in a previous publication.²⁴ 

This study was approved by the institutional review boards of the participating hospitals and the Western Institutional Review Board.

Adherence to the bronchiolitis PRIMES indicators was determined through medical record abstraction by two trained research team members at each hospital using the previously developed and validated PRIMES tool.²⁴  For each case, the PRIMES tool was used to determine if the participant met eligibility criteria for each PRIMES bronchiolitis indicator (denominator) and whether the patient received the indicated care (numerator). Eligibility criteria varied by indicator, for example, children <8 weeks were excluded from indicators related to complete blood cell (CBC) counts and blood cultures (Supplemental Table 3), because evaluation for neonatal fever in this age group could confound adherence to these indicators for the child’s bronchiolitis care.

The tool was used to calculate adherence to each of the 16 individual PRIMES indicators for each participant.²⁴  To enable an assessment of interrater reliability at each hospital, abstraction was performed on a 10% random sample from the other abstractor’s cases. Two agreement measures were used to evaluate interrater reliability, one for scoring eligibility (whether a patient was eligible for a PRIMES indicator) and one of scoring of the indicator (whether compliance for the indicator was met). Because of the low prevalence of indicator ineligibility (<5% of cases), the prevalence-adjusted, bias-adjusted κ statistic²⁹  was used to determine level of agreement for scoring eligibility, which was 0.98 (95% confidence interval: 0.96 to 0.99). For indicator scoring, the interrater agreement was 88%, with a Cohen κ³⁰  of 0.53 (95% confidence interval: 0.49 to 0.57), representing a moderate level of agreement.

During development, each PRIMES indicator was classified by its function related to patient care (diagnosis, treatment, and follow-up) and by the modality by which care is delivered (eg, laboratory, medication, referrals).²⁴  Each of the 16 indicators was assigned a single function and a single modality. The three functions were diagnosis (8 indicators), treatment (7 indicators), and follow-up (1 indicator). The five modalities included history (2 indicators), physical examination (3 indicators), laboratory and radiology (4 indicators), medication (5 indicators), ancillary therapy (1 indicator), and referral (1 indicator).²⁴  For this study, we also classified the PRIMES indicators into “overuse” and “underuse” indicators. The overuse indicators were 9 indicators for care that should not be routinely provided (eg, chest radiographs for bronchiolitis). Underuse indicators were the indicators for care that should be provided in most cases (eg, supplemental fluids for children who had difficulty feeding safely) and included 7 indicators²⁶  (Supplemental Table 3).

LOS was calculated in hours by using admission and discharge times from the medical records for each hospital. Cost of care was obtained from the Children’s Hospital Association Pediatric Health Information System (PHIS) database. The PHIS database contains total charges for each child in the study. Charges were converted to costs by using institutional cost-to-charge ratios, and inflation was adjusted to 2018 dollars by using the medical care services component of the Consumer Price Index.²⁸  Costs were winsorized at the fifth and 95th percentile to reduce the skewness in the cost distribution and to minimize the impact of extreme cost outliers on estimation.³¹ 

Demographic data collected from the PHIS included child age, child sex, level of medical complexity, insurance type, admit season, and hospital. Level of medical complexity was determined by using the Pediatric Medical Complexity Algorithm (PMCA) to classify children into 3 categories of medical complexity: (1) without chronic disease, (2) noncomplex chronic disease, or (3) complex chronic disease.³²  We conducted a medical record review of all subjects who were classified as PMCA 3 (complex chronic disease) to ensure that they met inclusion criteria. Admission season was defined as fall and winter if the child was admitted between November and March. Additional demographics data were collected through parent questionnaires completed at the time of hospitalization and included race and ethnicity and caregiver education level.

A priori, we generated hypotheses as to whether each individual indicator would be associated with decreased LOS and decreased cost (Supplemental Table 3). We examined univariate descriptive statistics for all predictor and outcome variables. We applied mixed effects linear regression models to examine the association between adherence to each individual indicator and each independent outcomes (LOS and cost). We calculated the adjusted mean difference between participants who received care that was adherent versus nonadherent to each individual indicator. For LOS, the adjusted mean difference was calculated in hours, and for cost, it was calculated in 2018 dollars. We included a priori identified covariates (child age, child sex, child race and ethnicity, PMCA category, caregiver education, insurance type, admission season, and hospital site) in the adjusted model. Cost models were additionally adjusted for LOS, which is usually the strongest driver for cost. All models adjusted for within-hospital clustering by including a hospital-specific random effect. Last, we plotted the adjusted mean differences for LOS and cost for each indicator to identify which indicators were associated with both shorter LOS and lower cost of care.

All analyses were conducted by using R 3.5.0.³³ 

There were 699 participants enrolled in this study. The mean age of participants was 8.2 months (SD: 6.2). Fifty-six percent of the sample was male, 38% was white, and 63% had public insurance. The mean PRIMES score for all participants was 87 (SD: 8) (Table 1). Individual PRIMES scores ranged from 57 to 103. The mean LOS was 2.3 days (SD: 1.6) and the mean cost per hospitalization was $5051 (SD: $2821).

There were four indicators for which adherence to the indicator was associated with a significantly shorter adjusted mean LOS (Table 2, Fig 1). These four indicators were indicator 3: children with bronchiolitis should not have blood cultures performed; indicator 5: children with bronchiolitis should not have a CBC count performed; indicator 6: children with bronchiolitis should not have a test performed for respiratory syncytial virus (RSV); and indicator 7: children with bronchiolitis should not have a chest radiograph performed. All 4 of these indicators were in the overuse category, had a function for diagnosis, and were included in the modality “laboratory and radiology” (Supplemental Table 3). None of the PRIMES indicators were associated with a significantly longer LOS.

There were 5 indicators for which adherence to the indicator was associated with a significantly different adjusted mean cost after controlling for LOS (Table 2, Fig 2). Three indicators that were associated with shorter adjusted LOS were also independently associated with a lower adjusted mean cost: indicator 3: children with bronchiolitis should not have blood cultures performed; indicator 5: children with bronchiolitis should not have a CBC count performed; and indicator 6: children with bronchiolitis should not have a test performed for RSV (Fig 3). Adherence to 2 indicators was associated with a higher adjusted mean cost adjusted for LOS: indicator 17: children admitted with bronchiolitis should have discharge criteria documented on feedings at a level to prevent dehydration; and indicator 18: parents of children admitted with bronchiolitis should be instructed to schedule a follow-up appointment with the child’s primary care provider. Both of these indicators were in the underuse category (Supplemental Table 3).

In a prospective cohort of children admitted with bronchiolitis to 5 geographically diverse children’s hospitals, we identified several quality indicators that were strongly associated with outcomes, LOS, and cost. Importantly, all 3 quality indicators that were significantly associated with better outcomes (ie, shorter LOS and lower cost) were overuse indicators and were within the modality of laboratory and radiology testing. To adhere to these indicators, children did not receive tests that are generally unnecessary in bronchiolitis: blood cultures, CBC counts, and RSV testing. With these findings, we highlight the importance that decreasing overuse in inpatient care for bronchiolitis can have on outcomes.^{19,22,34–36} 

Hospital admissions for bronchiolitis are common and costly, which is why improving care for bronchiolitis has been a focus of considerable attention over the past 20 years.^{2,3,5,15,19,22,37}  Although LOS is the main contributor to hospital-based costs of care, this study additionally included cost as an independent outcome by controlling for LOS in the regression models. This strategy allowed us to identify indicators that were strongly associated with adjusted cost. Because “less is more” for bronchiolitis, it follows that adherence to overuse quality measures related to less testing is associated with lower cost.¹¹  In addition to the direct costs from unnecessary tests contributing to the association between process measures and cost outcomes, we hypothesize that there may be a “cascade of care” that occurs after unnecessary testing that leads to higher cost in cases of lower adherence to the overuse quality measures identified.^38,39  As deimplementation efforts continue for bronchiolitis, it is important to include cost as an outcome, particularly given our finding that adherence to two quality indicators was associated with a higher cost of care.^19,22  It is unclear why adherence to two of the underuse quality measures related to documentation at time of discharge for oral intake and follow-up was associated with higher cost. We hypothesize that oral intake and follow-up may be preferentially documented for children with a higher severity of illness and thus associated with a higher cost of care. Although we adjusted for patient-level demographic variables, level of medical complexity using PMCA, and hospital site and excluded children requiring ICU-level care, there remains potential for unmeasured confounding, including illness severity or unmeasured patient characteristics contributing to this relationship. A more in-depth understanding of the effect of quality improvement efforts on cost will help to improve delivery of high-value care to hospitalized children with bronchiolitis.

Unlike previous studies, PRIMES allows for a more detailed analysis of adherence to specific recommendations by using medical records to identify individual patient characteristics that may contribute to appropriate clinical differences in adherence to quality measures in bronchiolitis. PRIMES was developed so that hospitals can have an effective tool to measure the quality of care provided to children hospitalized with respiratory illnesses. This regression analysis has identified that there are certain indicators that are closely associated with favorable outcomes, including shorter LOS and lower cost. Importantly, in this study, we also found that none of the PRIMES indicators were associated with a significantly longer LOS. An important next step in the use of PRIMES would be to examine whether improving adherence to the identified quality indicators over time by using quality improvement methods is associated with a favorable impact on patient-level outcomes. By longitudinally evaluating adherence to these quality indicators over time, a hospital could measure the association of their quality improvement efforts using standardized validated indicators.

Nationally, bronchiolitis remains a key target for quality improvement by the Society of Hospital Medicine, which has focused 4 of its 5 pediatric metrics in the Choosing Wisely campaign on respiratory illnesses such as bronchiolitis.^40,41  The strong association of adherence to PRIMES bronchiolitis quality measures with lower cost and better health care use outcomes make them a useful tool for quality improvement at the hospital level.²⁵  We demonstrate in this study that hospitals can use PRIMES as a tool to identify opportunities for improvement and measure their progress in meeting these improvement goals.

There were several limitations to our study. All of the hospitals in this study were children’s hospitals. Although geographically dispersed, they may not be representative of all children’s hospitals or community-based settings where many children are hospitalized for respiratory illnesses.⁴²  Expanding the focus of this research and the use of quality measures for bronchiolitis to community-based settings should be a priority.

We examined adherence to the inpatient quality indicators for children with bronchiolitis. Previous studies have revealed significant variation in care for bronchiolitis in ED settings and that adherence to quality indicators across both settings is associated with higher value care.^25,43  Further studies should be focused on quality improvement efforts for bronchiolitis across health care settings, including the ED, where recent evidence examining ED care nationally has revealed little improvement in the rate of unnecessary medication use and chest radiographs for bronchiolitis despite national guidelines.^44,45 

Adherence to PRIMES indicators in this study was determined by using medical record abstraction. Although this method allows for a detailed evaluation at the patient level, there may be different degrees of difficulty in abstracting data for different indicators and across different hospitals. The interrater reliability for indicator scoring in this study was moderate. Finally, we included factors, such as PMCA, as a proxy for severity of illness and excluded children who were admitted to the ICU. However, these factors may not be fully reflective of illness severity during the hospitalization or between hospitals because, for example, each hospital may have different criteria for ICU admission. Additionally, there may have been additional unmeasured patient characteristics that contributed to the associations identified.

Finally, the findings in this study represent associations between individual PRIMES indicators and outcomes. Future prospective quality improvement work is needed to examine whether increasing adherence to these quality indicators can lead to improved outcomes.

We identified several PRIMES quality indicators that were strongly associated with both shorter LOS and cost for children admitted with bronchiolitis. The three quality indicators associated with both favorable outcomes were overuse of laboratory tests: blood cultures, CBC counts, and RSV testing. By using quality measures with strong associations with improved outcomes as specific individualized targets for quality improvement, hospitals can focus efforts to provide higher value care for children admitted with bronchiolitis.