Pediatric Respiratory Illnesses: An Update on Achievable Benchmarks of Care Free

The original driver for the PRI report was to update and reconcile several existing standard reports for respiratory conditions from PHIS, the data set of the CHA (Lenexa, KC).⁵²  PHIS is an administrative database that contains outpatients, ED, inpatient, observation, and ambulatory surgery encounter-level data from 49 affiliated pediatric hospitals in North America. These hospitals provide care for ∼20% of all pediatric hospitalizations and ∼10% of all pediatric ED visits in the United States annually. A joint effort between CHA and participating institutions ensures data quality.⁵⁷  Hospitals submit discharge and encounter-level data, including demographics, diagnoses, and procedures. Hospitals also submit resource utilization data, including pharmaceuticals, imaging, and laboratory.

The process for development of the PRI report followed a methodology outlined by the National Quality Forum⁵⁸  and principles applied by the Agency for Healthcare Research and Quality.^59,60  These agencies have advocated assessing high-impact conditions, reducing measurement burden through automatic data extraction, and promoting the implementation of consistent measures across all stakeholders of care delivery.

Similar to previous work in pediatric emergency medicine⁶¹  and pediatric hospital medicine,⁵⁵  we engaged a working group of 10 subject-matter experts from the fields of pediatric hospital medicine (M.R., V.E., R.M.S., S.K., K.P.) and health services research from CHA (C.H., P.D., A.D., and M.H.). All had health services research and/or QI experience. The subject-matter experts mapped and executed the process of identification or development of QM and further operationalization in PHIS. To identify candidate QM for PRI from different repositories, the authors reviewed the literature independently (Table 1). We discussed each identified potential QM in virtual meetings until reaching a consensus for inclusion in the report. If changes in definitions were required for operationalization in the data set, the PHIS experts’ team moderated the discussion to find a consensus on the best way to define the metric.

This research received exemption from approval by the institutional review board from Nicklaus Children’s Hospital.

We developed bundles of International Classification of Diseases, 10th Revision (ICD-10), Clinical Modification,⁶²  codes to identify encounters with each PRI. We reconciled code lists used in PHIS report cards with coding strategies applied in previous QI work.^{10,26–29,31,46,51,52,63}  They comprehensively and accurately define each condition, reflecting current documentation and coding practices in children’s hospitals (Supplemental Table 6).

Encounters were eligible if the corresponding ICD-10, Clinical Modification, code was listed as the principal diagnosis. To avoid misclassification, the presence of any other PRI as a secondary diagnosis was an exclusion criteria. However, this exclusion was not applied to influenza, because influenza viruses can trigger any respiratory illness (eg, influenza-triggered asthma). Similarly, the influenza cohort did not exclude those with a secondary diagnosis of any other PRI. Pneumonia is a clinically diverse condition, complex to categorize in administrative data sets,⁶⁰  and we added special considerations for this cohort (Table 2).

We further defined each condition-specific cohort using age to reflect the unique pathophysiology and clinical characteristics:

Bronchiolitis: ≥1 month and <24 months (aligned with American Academy of Pediatrics [AAP] practice guidelines).²¹ 

Asthma: ≥24 months to <18 years old because the diagnosis of asthma in children <24 months is challenging, and wheezing is more commonly because of other diagnoses.^19,24 

Pneumonia: ≥3 months to <18 years old (aligned with Infectious Diseases Society of America guidelines).²⁰ 

Croup:<6 years old, reflecting the population specified in previously published practice recommendations and research.^10,11,28 

Influenza: ≥1 month to <18 years old, the age limits for this study.

We targeted a cohort of “uncomplicated” children aimed at higher specificity by excluding children with complex chronic conditions (Feudtner’s algorithm)⁶⁴  if present 12 months before or during the encounter. We also excluded severe illness (All Patient Refined Diagnosis-Related Group; 3M), “extreme” severity of illness, and ICU admission. (Table 2).

Measures were identified from varied sources: PHIS standard reports,⁵²  relevant national QI efforts (ie, the Pediatric Respiratory Illness Inpatient Measurement System^11,28 ), Value Inpatient Pediatrics (VIP) projects,^{30,33–35,48,49}  the Agency for Healthcare Research and Quality,⁵⁹  the National Quality Forum,⁶⁵  or peer-review clinical practice guidelines (Table 1).^10,19–23 

The structured process for measure prioritization included:

1. Face validity: Expert consensus.

2. Actionability: How much is the indicator amenable to be influenced by the care team and clinical processes.

3. Feasibility for operationalization in PHIS.

4. Exclusions: Duplicated measures or indicators with limited room for improvement (performance <5 or >95% in PHIS, depending on the desire trend).

5. Potential applicability to other administrative data sets and electronic health records.

Candidate measures were iteratively discussed in >20 monthly videoconference meetings between 2019 and 2021 until a consensus was reached. Indicators that met prioritization criteria were included in the final roster of the measure set. We submitted the set of candidate measures for public input, including ED experts, to the leadership of VIP and PRIS, the AAP Section on Hospital Medicine, and the CHA Quality & Safety and ED networking forums. We integrated this input into the final selection.

When indicated, the QM are estimated as ratios: Percentage (%) of encounters with the intervention of interest (therapies, procedures) divided by absolute number of eligible encounters. Clinical Translation Codes are a proprietary resource of the CHA that identify therapies and procedures in PHIS. Supplemental Table 7 includes these metrics with specifications and definitions (numerator/denominator) for operationalization in administrative data sets.

All cohorts include patient demographics, volume, length of stay (days), case-mix index, excess hospital days (based on the case-mix index), standardized costs (US dollars), and readmission/ED revisits. Supplemental Table 8 includes the complete PRI report card measure set.

We conducted a retrospective, observational cohort study applying the PRI report to the PHIS database. The study included children aged between 1 month and 18 years presenting to the hospital with PRI from January 1, 2017, to December 31, 2019. Hospitals’ data were included only if they consistently contributed data during this period. QM were estimated for 2 cohorts: (1) encounters resulting in discharge from the ED, and (2) encounters of patients hospitalized (IP), including ED care before admission. Multiple ED visits or hospitalizations by the same patient in the same or separate calendar years were counted independently. Encounters were eligible if they met all inclusion and exclusion criteria (Table 2).

We determined ABC and performance gaps (PG) for pertinent measures. We estimated ABC by applying a previously described methodology.^29,53,54  Hospitals were ranked from best to worst performance, and the highest-performing sample represented at least 10% of the overall population. The ABC was calculated as the performance of this pooled patient-level sample. PGs are the absolute differences between the median hospital performance and the ABC, and assess how far the actual performance is from the ABC. Analysis of the PG serves to quantify the amount of variation in performance on a measure and help prioritize QI efforts.⁵⁶ 

Categorical variables were summarized with counts and percentages. For continuous variables, we estimated the median and interquartile range of the aggregated measure performance across the hospitals, both for ED and hospitalizations. ABC with the number of hospitals that contributed to the calculation are reported along with the PG. All statistical analyses were performed by using SAS version 9.4 (SAS Institute, Cary, NC).

The PRI report includes a total of 94 quality measures: asthma (18), bronchiolitis (20), croup (18), influenza (16), and uncomplicated pneumonia (22). Several metrics are common to ED and hospitalized cohorts (Supplemental Table 8).

The study included 984 337 eligible encounters from 49 tertiary children’s hospitals. The most common primary diagnosis was asthma (n = 285 292), and the least common was pneumonia (n = 102 291). Almost two-thirds were children aged <5 years (64.8%). Most encounters were for children with government insurance (63.2%) (Table 3).

The performance analysis of measures, with the corresponding ABC and PG, are included in Supplemental Table 7.

The most common reason for ED visits was asthma (n = 225 079) and the least frequent was pneumonia (n = 76 315).

Table 4 displays the performance of selected measures. Measures with high performance included median values of 0.4% and 4.1% for antibiotics and systemic corticosteroids, respectively, in bronchiolitis. In pneumonia: 5.3% for the use of blood cultures and <4% for all inflammatory markers. In these measures, the performance was close to the ABC with a small PG. Also, the median value for the use of dexamethasone in croup was 92.7%.

Other indicators showed less optimal performance: Chest x-ray (CXR) use in bronchiolitis and asthma had median values of 13.7% and 14.4%, with PG of 7.8% and 6.5%, respectively.

The median estimated cost per ED encounter ranged from $301 in croup to $531 in pneumonia. The PG in standardized costs ranged from in $119 in bronchiolitis to $236 in pneumonia.

The most common conditions for hospitalization were for bronchiolitis (n = 63 444) and the least common influenza (n = 10 523).

Table 5 displays the performance of selected measures. The median value of the ALOS was <2 days for all conditions, longest for bronchiolitis and pneumonia (1.9 days) and shortest for croup (1.06 days). Measures with high performance included median values of 97.5% for use of systemic corticosteroids in asthma and 7.8% for antibiotics in bronchiolitis. In both cases, performance was close to the ABC, with a resulting PG <5%.

Measures with low performance included median values of use of bronchodilators (34.6%), CXR (14.4%) in bronchiolitis, and 32.7% use of CXR in asthma. In the pneumonia cohort, there was a 57.3% use of narrow spectrum antibiotics, including a 23% use of macrolides, and 28.1% of collected blood culture. The PG was larger for these indicators.

The estimated cost per hospitalization ranged from $2592 in croup to $5144 in pneumonia. The PG in standardized costs ranged from $670 in croup to $1324 in pneumonia. Both cost and PG were the lowest in croup and highest for pneumonia.

We developed the PRI report card to guide quality measurement, benchmarking, and QI efforts in children’s hospitals. Potentially, the report can be broadly operationalized in other administrative and electronic health record data sets. Thus, this report can drive high-value care for children hospitalized with these common conditions.

Key trends in ABC for bronchiolitis, asthma, and pneumonia: (decrease in ABC suggest performance improvement for overuse measures where decreased use is the desired practice).

Bronchiolitis: The 2014 AAP bronchiolitis guidelines recommended against the routine use of CXR, steroids, antibiotics, or bronchodilators.²¹  The 2012 Society of Hospital Medicine Choosing Wisely Campaign recommendations also discouraged the routine use of CXR and bronchodilators.⁶⁶  There was a downward trend in the ABC for CXR in inpatients: 32.4% in 2012,⁵³  28.8% from 2014 to 2019,⁵⁶  and 15.3% in our study. There was also a downward trend in the ABC for bronchodilator use in inpatients (19.9% in 2012,⁵³  22.8% from 2014 to 2019,⁵⁶  and 13.5% in our study) and for antibiotic use (18.5% in 2012,⁵³  to <1% in the ED and 3% in the IP in our study, respectively). This trend may reflect the impact and sustainability of multiple local and national QI projects undertaken over the last decade. The optimal performance of these measures cannot be 0, because some cases of bronchiolitis have complications requiring these interventions. Close monitoring of the measures both in academic and community hospitals can help identify those that have achieved optimal performance and guide QI prioritization.

Asthma: The use of CXR for uncomplicated asthma was recently highlighted as a low-value service.⁶⁷  It is encouraging to see a decrease in the ABC for the use of CXR in asthma from 24.5% (median of 46.1%) in 2012⁵³  to 19.8% (median of 32.7%) in the current study. It is important to note the downtrend in the ABC and median, reflecting decreased utilization across more hospitals, not just the top performers used in the ABC calculation. This trend may also represent the impact of multiple local and national QI work. For example, clinical pathways and peer coaching were interventions used in the multicenter “Pathways for Improving Inpatient Pediatric Asthma Care” collaborative and may be important tools for improving care.^48,49 

Pneumonia: The ABC for narrow-spectrum antibiotic use demonstrated promising improvements. When compared with the 2012 cohort,⁵³  the ABC increased by 10% (from 60.7% to 70.1%), and the median increased by 30% (from 27.3% to 57.3%). These results may be attributable to the development of national guidelines,²⁰  antibiotic stewardship recommendations,⁶⁸  and/or QI collaborative projects.³⁵  The VIP multicenter collaborative to improve care of community-acquired pneumonia project was able to increase the use of narrow-spectrum antibiotics from 14% to 44% in the ED, and 36% to 63% in the IP using low-resource strategies, including a virtual collaborative, practice guidelines, informational sessions, and coaching.³⁵ 

This study provides updated ABC for using steroids in croup in the ED. Compared with a previous study,²⁹  this metric increased from 92% to 94%. The use of racemic epinephrine in the ED was low: Median of 19.5%, ABC 35.8%, but this therapy applies only to moderate and severe croup,¹⁰  and the measure should be interpreted cautiously. We provide the first known set of ABC for croup in IP and influenza. The use of racemic epinephrine in inpatients was also low, with the ABC at 35.8%. In Influenza, the use of oral antiviral agents was low in both settings, with a large PG: 23% in the ED and 18.5% in inpatients. The use of antiviral agents in influenza was recently highlighted as a target for improvement.⁶⁹  (Tables 4 and 5).

Four key features characterize this new PRI report: First, developed through expert consensus, this is an updated set of QM drawn from multiple existing repositories. Second, the report allows for automated data extraction of QM for benchmarking that significantly reduces measurement burden, leading to greater usability and implementation. Automation could eventually facilitate predictive analytics and application of artificial intelligence tools. Third, the report integrates ABC for several measures, critical for benchmarking aimed at driving local improvements. Lastly, the report facilitates the analysis of trending of both performance and ABC over time. This allows for assessing longitudinal changes in local and national care patterns, and the potential retirement of measures with sustained optimal performance.

The study period is before the coronavirus disease 2019 (COVID-19). The pandemic dramatically affected patient volumes and hospital operations.⁷⁰  New evidence suggests lapses in quality monitoring and potential quality declines during this time.^71,72  However, our cohort analysis was meant for testing and internal validation of the report, so we purposefully focused on a time interval before COVID-19. The results represent a “baseline,” prepandemic performance. Thus, it can be used to understand and drive improvements in health care quality and inequities existing before and exacerbated during the pandemic. Cohorts can be further stratified by readily available data on race, ethnicity, primary language, and socioeconomic status markers to understand and address these inequities.

This study has several limitations. Only patients hospitalized at freestanding, tertiary children’s hospitals were included, and the results may not be applicable to other institutions. Administrative databases lack clinically detailed information on illness severity. The use of ICD-10 codes to identify cohorts of respiratory illnesses and coding practices may vary among institutions. The bundle codes applied are frequently used in clinical research and QI efforts; however, they have been validated only for pneumonia.⁶³  In addition, there was no validation of pertinent QM by actual chart review. Reports from administrative databases lagged for some time after care was provided, limiting their use as real-time data for QI projects requiring rapid cycle tests of change. Changes in cohort definitions, specifically excluding codiagnoses, could affect the analysis of the evolution of the benchmarks when compared with some of the previously published benchmarks. For example, excluding the diagnosis of pneumonia in the bronchiolitis cohort may have resulted in a lower estimate of antibiotic use. Finally, there is lack of evidence for the impact of some process QM in the report on meaningful outcomes. Further work is needed to explore this association.

The PRI report card is designed to assess care quality for children with the most common causes of hospitalization and ED visits for respiratory illnesses. It defines new benchmarks and identifies performance gaps. This information is critical for prioritizing local and multicenter QI projects, health services, and comparative effectiveness research. Future directions include dissemination, examining health inequities, and understanding and addressing the effects of the COVID-19 pandemic on care quality.

AAP

American Academy of Pediatrics

ABC

achievable benchmarks of care

CHA

Children’s Hospital Association

COVID-19

coronavirus disease 2019

CXR

chest x-ray

ED

emergency department

ICD-10

International Classification of Diseases, 10th Revision

IP

inpatient settings

PG

performance gaps

PHIS

Pediatric Health Information System

PRI

pediatric respiratory illnesses

QI

quality improvement

QM

quality measures

VIP

Value Inpatient Pediatrics