OBJECTIVES:

In 2013, the Society of Hospital Medicine (SHM) released 5 pediatric recommendations for the Choosing Wisely Campaign (CWC). Our goals were to develop a report card on the basis of those recommendations, calculate achievable benchmarks of care (ABCs), and analyze performance among hospitals participating in the Pediatric Health Information System.

METHODS:

Children hospitalized between January 2013 and September 2015 from 32 Pediatric Health Information System hospitals were studied. The quality metrics in the report card included the use of chest radiograph (CXR) in asthma and bronchiolitis, bronchodilators in bronchiolitis, systemic corticosteroids in lower respiratory tract infections (LRTI), and acid suppression therapy in gastroesophageal reflux (GER). ABCs were calculated for each metric.

RESULTS:

Calculated ABCs were 22.3% of patients with asthma and 19.8% of patients with bronchiolitis having a CXR, 17.9% of patients with bronchiolitis receiving bronchodilators, 5.5% of patients with LRTIs treated with systemic corticosteroids, and 32.2% of patients with GER treated with acid suppressors. We found variation among hospitals in the use of CXR in asthma (median: 34.7%, interquartile range [IQR]: 28.5%–45.9%), CXR in bronchiolitis (median: 34.4%, IQR: 27.9%–49%), bronchodilators in bronchiolitis (median: 55.4%, IQR: 32.3%–64.9%), and acid suppressors in GER (median: 59.4%, IQR: 49.9%–71.2%). Less variation was noted in the use of systemic corticosteroids in LRTIs (median: 13.5%, IQR: 11.1%–17.9%).

CONCLUSIONS:

A novel report card was developed on the basis of the SHM-CWC pediatric recommendations, including ABCs. We found variance in practices among institutions and gaps between hospital performances and ABCs. These findings represent a roadmap for improvement.

In 2012, the American Board of Internal Medicine Foundation launched the Choosing Wisely Campaign (CWC) with the goal of avoiding wasteful or unnecessary medical tests, treatments, and procedures. The campaign encouraged medical societies to develop a list of 5 therapies or tests that physicians and patients should question. In 2013, the Society of Hospital Medicine (SHM) Pediatric Committee published a list of 5 pediatric recommendations and the methodology for its development.1  These recommendations include the following:

  1. do not order chest radiographs (CXRs) for children with uncomplicated asthma or bronchiolitis;

  2. do not routinely use bronchodilators in children with bronchiolitis;

  3. do not use systemic corticosteroids in children <2 years of age with an uncomplicated lower respiratory tract infection (LRTI);

  4. do not treat gastroesophageal reflux (GER) in infants routinely with acid suppression therapy; and

  5. do not use continuous pulse oximetry routinely in children with acute respiratory illness unless they are on supplemental oxygen.

The authors encouraged the use of the SHM-CWC pediatrics list in quality improvement (QI) efforts aimed at improving the care of hospitalized children. Indeed, since the publication of these recommendations, at least 1 institution has undertaken and published a QI project inspired by them.2  However, a comprehensive examination regarding variation across hospitals as to the adherence to the SHM-CWC pediatric recommendations is lacking.

Hospital dashboards are automated systems that report on a discrete number of key metrics. They are easy to use and displayed graphically, which allows stakeholders to have an up-to-date glance at institutional performance on a variety of metrics.3  Institution members of the Children’s Hospital Association (CHA) who submit data to the Pediatric Health Information System (PHIS) have Web-based access to several clinical report cards. The existing report cards measure institutional and group practice performance in the care of asthma, bronchiolitis, appendicitis, tonsillectomy, diabetic ketoacidosis, hospital acquired conditions, Agency for Healthcare Research & Quality Pediatric Quality Indicators, and in emergency departments.4 

The primary goal of this article is to describe the development of a new report card based on the SHM-CWC pediatric recommendations from PHIS administrative data. This dashboard will help institutions and group practices to measure and benchmark individual performance on adherence to these recommendations.

However, the recommendations are not expected to achieve 100% compliance. An additional purpose of this article is providing realistic goals for benchmarking by calculating achievable benchmarks of care (ABCs) by using a previously published method.57 

By compiling the performance of the tertiary children’s hospitals included in this study, we expect to have a glimpse at the compliance with the 2013 SHM-CWC pediatric recommendations in these centers.

From 2014 to 2016, a multicenter working group from 4 children’s hospital members of CHA and analytic support specialists worked on a collaborative project. The PHIS database (CHA, Lenexa, KS) was queried to develop a report card for the SHM-CWC pediatric recommendations. This database includes clinical and daily billing data from 48 tertiary care children’s hospitals and accounts for ∼20% of all US pediatric hospitalizations every year. Participating hospitals submit de-identifying data. A joint effort between CHA and participating hospitals ensures the quality of the data submitted, as has been previously described.8  These data are subjected to a routine quality check with each submission and within each report. In total, 32 PHIS hospitals met the strict quality standards for inclusion of submitted data. The rest of the hospitals did not meet the strict data quality requirements from PHIS and were excluded from this study.

This research was exempt from approval by the institutional review board of Nicklaus Children’s Hospital.

The development of a hospital-specific report card required multiple conference calls among the authors and included the following steps:

  1. assembly of content (all authors) and design of the report card (development experts);

  2. selection and definition of measures, population inclusion, and exclusion criteria;

  3. selection of ABCs as a method for establishing targets in the report card;

  4. selection of the established CHA report card format for consistency in presentation and ease of use; and

  5. education regarding content and purpose provided to users through Webcasts and newsletters.

Four of the 5 SHM-CWC pediatric recommendations were able to be translated into 5 quality metrics. The first recommendation (avoiding the use of CXR in asthma and bronchiolitis) was converted into 2 separate metrics, 1 for each condition. The fifth SHM-CWC pediatric recommendation, limiting the use of continuous pulse oximetry for acute respiratory illness unless the patient is receiving supplemental oxygen, could not be translated into a metric by using PHIS data because of inconsistent reporting of the use of these resources. Consequently, the report card has 5 metrics:

  1. percentage of patients with uncomplicated asthma receiving CXR;

  2. percentage of patients with uncomplicated bronchiolitis receiving CXR;

  3. percentage of patients with uncomplicated bronchiolitis receiving bronchodilators;

  4. percentage of patients with LRTI receiving systemic corticosteroids; and

  5. percentage of patients with uncomplicated GER receiving acid suppressor therapy.

These metrics have the study population selected as the denominator and the number of patients who received the contraindicated therapy or intervention as a numerator. Thus, the percentages displayed on the report card represent nonadherence to the recommendations.

Children were eligible for inclusion if they were hospitalized (inpatient or observation status) between January 1, 2013 and September 31, 2015. The SHM-CWC pediatric recommendations refer to “uncomplicated” conditions. Some of the original recommendations do not specify age groups.1  We defined different age groups for each condition as well as inclusion and exclusion criteria aimed at identifying only uncomplicated cases, as follows:

  • Asthma: 3 to 17 years. Considering that the diagnosis of asthma in infants and children younger than 3 years old is challenging and it may overlap with other causes of wheezing. The upper limit of 17 years of age will capture most children with this condition.

  • Bronchiolitis: ≥1 month to <1 year. This age group was selected to exclude newborn infants who tend to have a more severe disease and patients older than 1-year-old who may have other causes of recurrent wheezing that may require inhaled bronchodilators and CXR.

  • LRTI: ≥1 month to <2 years. This age group closely matches the age group of the SHM-CWC pediatric recommendation on avoiding systemic corticosteroids. Newborn infants are excluded considering the likelihood of more severe disease. This group includes mostly cases of bronchiolitis and pneumonia who do not benefit from this intervention.

  • GER: ≤1 year. This age group matches the age group of infants, for whom the specific SHM-CWC pediatric recommendation is intended. In this age group, the presence of physiologic, uncomplicated GER is more likely, and the routine use of acid suppressors is not recommended.

As inclusion criteria, the working group developed comprehensive lists of International Classification of Diseases, Ninth Revision (ICD-9) codes9  that identify bronchiolitis, asthma, LRTI and GER. Exclusion criteria comprised a detailed list of ICD-9 codes9  corresponding to multiple conditions and comorbidities that may complicate the course or treatment of these conditions. Additional exclusion criteria also included were length of stay (LOS) of more than 5 days, hospitalizations with charges for intensive care services, and extreme severity of illness (level 4) from the All Patient Refined Diagnosis Related Group (3M Corp, Wallingford, CT). This way, the cases selected for the study have a short LOS, no complications requiring intensive care, and no other chronic or comorbid conditions.

Additionally, to minimize misclassification, for each category of respiratory conditions, we excluded cases with any of the other conditions if they were listed as a comorbidity because they may require different management. For example, patients with an ICD-9 code indicating asthma are excluded from the population of patients with bronchiolitis and LRTI, and vice versa.

The resulting cohorts represent uniform populations of uncomplicated conditions, as explicitly targeted in the SHM-CWC pediatric recommendations.

The interventions and therapies not routinely recommended are CXR, bronchodilators, systemic corticosteroids, and acid suppressor therapy. Development experts define these interventions through a mapping process in which PHIS hospitals’ data are translated from their charge codes into clinical translation classification codes. These codes are a meaningful tool to make hospital billing comparable among institutions, and they are updated yearly.

Table 1 shows the age groups for each condition, a summary of the inclusion and exclusion criteria, and the 5 metrics selected. Readers can obtain the complete lists of inclusion and exclusion criteria and the most current group of clinical translation classification codes identifying interventions not recommended in the SHM-CWC pediatric recommendations by contacting Carla Hronek (carla.hronek@childrenshospitals.org). The detailed ICD-9 lists of inclusion and exclusion criteria for each condition and the lists of codes for procedures and interventions are available in the Supplemental Fig 4, which details the exclusion list for respiratory conditions).

TABLE 1

SHM-CWC Pediatric Recommendations in 2013

ConditionInclusion CriteriaExclusion CriteriaMetric, %
Asthma Age: 3–17 y ICU admission, APR-DRG SOI, and 4 ICD-9 codes, including Receiving CXR 
ICD-9 codes: asthma  Pneumonia 
LOS ≤5 d  Malnutrition 
Patient type: inpatient and observation  Neurologically impaired children 
 CLD 
 Congenital lung anomalies 
 Congenital heart defects 
 Respiratory failure and/or acidosis 
 Orofacial anomalies 
 Presence of tracheostomy tube 
Bronchiolitis Age: ≥1 m and <1y ICD-9 codes: bronchiolitis ICU admission APR-DRG SOI, and 4 ICD-9 codes, including Receiving CXR 
LOS ≤5 d  Asthma Receiving bronchodilator 
Patient type: inpatient and observation  Malnutrition 
 Neurologically impaired children 
 CLD 
 Congenital lung anomalies 
 Congenital heart defects 
 Respiratory failure and/or acidosis 
 Orofacial anomalies 
 Presence of tracheostomy tube 
LRTI Age: ≥1 m and <2 y ICU admission APR-DRG SOI, and 4 ICD-9 codes, including Receiving systemic corticosteroid 
ICD-9 codes: bronchiolitis, pneumonia, other LRTI  Asthma 
 Malnutrition 
Patient type: inpatient and observation  Neurologically impaired children 
 CLD 
 Congenital lung anomalies 
 Congenital heart defects 
 Respiratory failure and/or acidosis 
 Orofacial anomalies 
 Presence of tracheostomy tube 
GER Age: ≤1 y ICU Admission, APR-DRG SOI, 4 ICD-9 codes, including Receiving acid suppression therapy 
ICD-9 codes: esophageal reflux  Malnutrition and/or failure to thrive 
Patient type: inpatient and observation  Apnea and/or ALTE 
 Esophagitis 
 Peptic ulcer disease 
 Gastritis and/or duodenitis 
ConditionInclusion CriteriaExclusion CriteriaMetric, %
Asthma Age: 3–17 y ICU admission, APR-DRG SOI, and 4 ICD-9 codes, including Receiving CXR 
ICD-9 codes: asthma  Pneumonia 
LOS ≤5 d  Malnutrition 
Patient type: inpatient and observation  Neurologically impaired children 
 CLD 
 Congenital lung anomalies 
 Congenital heart defects 
 Respiratory failure and/or acidosis 
 Orofacial anomalies 
 Presence of tracheostomy tube 
Bronchiolitis Age: ≥1 m and <1y ICD-9 codes: bronchiolitis ICU admission APR-DRG SOI, and 4 ICD-9 codes, including Receiving CXR 
LOS ≤5 d  Asthma Receiving bronchodilator 
Patient type: inpatient and observation  Malnutrition 
 Neurologically impaired children 
 CLD 
 Congenital lung anomalies 
 Congenital heart defects 
 Respiratory failure and/or acidosis 
 Orofacial anomalies 
 Presence of tracheostomy tube 
LRTI Age: ≥1 m and <2 y ICU admission APR-DRG SOI, and 4 ICD-9 codes, including Receiving systemic corticosteroid 
ICD-9 codes: bronchiolitis, pneumonia, other LRTI  Asthma 
 Malnutrition 
Patient type: inpatient and observation  Neurologically impaired children 
 CLD 
 Congenital lung anomalies 
 Congenital heart defects 
 Respiratory failure and/or acidosis 
 Orofacial anomalies 
 Presence of tracheostomy tube 
GER Age: ≤1 y ICU Admission, APR-DRG SOI, 4 ICD-9 codes, including Receiving acid suppression therapy 
ICD-9 codes: esophageal reflux  Malnutrition and/or failure to thrive 
Patient type: inpatient and observation  Apnea and/or ALTE 
 Esophagitis 
 Peptic ulcer disease 
 Gastritis and/or duodenitis 

CHA-PHIS report card: definitions of uncomplicated conditions and metrics. A complete list of inclusion and exclusion codes is available on request by contacting analytics@childrenshospitals.org or see online Supplemental Materials. ALTE, apparent life threatening event; CLD, chronic lung disease; SOI, severity of illness.

We compiled the combined hospital performances on each of the quality metrics included in the report card. Categorical variables are summarized with counts and percentages, whereas medians and interquartile ranges (IQRs) summarized continuous variables.

By using a previously published method,57  ABCs for the 5 SHM-CWC pediatric metrics included in this study were estimated. ABCs are calculated by first ranking the hospitals on the basis of their performance on a specific metric. Then, the best-performing hospitals that comprise 10% of the total patient population are selected. Finally, calculated ABCs represent the average from those top-performing centers.5 

Results are presented in tables and graphics. All statistical analyses were performed by using SAS version 9.3 (SAS Institute Inc, Cary, NC).

Table 2 shows the demographic information and the APR-DRG distribution of the cohorts for each condition included in the SHM-CWC pediatric recommendations. The study included 32 487 cases of asthma, 23 531 cases of bronchiolitis, 5963 cases of LRTI, and 2946 cases of GER. A slight predominance of boys in all categories was noted: 61.9% in asthma, 59.6% in bronchiolitis, 55% in LRTI, and 53.6% in GER. We also observed a higher prevalence of non-Hispanic African American patients with asthma (43%). Inversely, non-Hispanic white patients were more prevalent in the bronchiolitis, LRTI, and GER groups, mostly in the GER cohort (53.7%). The “minor” (severity level 1) APR-DGR severity of illness for each condition represents more than half of the patients in each of the diagnostic groups, ranging from 52.7% in LRTI to 76.4% in asthma.

TABLE 2

Cohort Demographics

AsthmaBronchiolitisLRTIGER
N 32 487 23 531 5963 2946 
Age, y     
 <1 — 23 531 (100) 2858 (47.9) 2719 (92.3) 
 1–4 9520 (29.3) — 3105 (52.1) 227 (7.7) 
 5–9 15 088 (46.4) — — — 
 10–18 7879 (24.3) — — — 
Sex     
 Male 20 124 (61.9) 14 036 (59.6) 3281 (55) 1578 (53.6) 
 Female 12 363 (38.1) 9495 (40.4) 2681 (45) 1368 (46.4) 
Race     
 Non-Hispanic white 8081 (24.9) 10 055 (42.7) 2209 (37) 1582 (53.7) 
 Non-Hispanic African American 13 980 (43) 4400 (18.7) 987 (16.6) 464 (15.8) 
 Hispanic 7218 (22.2) 6202 (26.4) 1835 (30.8) 580 (19.7) 
 Asian American 679 (2.1) 516 (2.2) 241 (4) 34 (1.2) 
 Other 2529 (7.8) 2358 (10) 691 (11.6) 286 (9.7) 
APR-DRG SOI     
 Minor 24 829 (76.4) 16 644 (70.7) 3141 (52.7) 1697 (57.6) 
 Moderate 7385 (22.7) 6636 (28.2) 2562 (43) 957 (32.5) 
 Major 273 (0.8) 251 (1.1) 260 (4.4) 292 (9.9) 
AsthmaBronchiolitisLRTIGER
N 32 487 23 531 5963 2946 
Age, y     
 <1 — 23 531 (100) 2858 (47.9) 2719 (92.3) 
 1–4 9520 (29.3) — 3105 (52.1) 227 (7.7) 
 5–9 15 088 (46.4) — — — 
 10–18 7879 (24.3) — — — 
Sex     
 Male 20 124 (61.9) 14 036 (59.6) 3281 (55) 1578 (53.6) 
 Female 12 363 (38.1) 9495 (40.4) 2681 (45) 1368 (46.4) 
Race     
 Non-Hispanic white 8081 (24.9) 10 055 (42.7) 2209 (37) 1582 (53.7) 
 Non-Hispanic African American 13 980 (43) 4400 (18.7) 987 (16.6) 464 (15.8) 
 Hispanic 7218 (22.2) 6202 (26.4) 1835 (30.8) 580 (19.7) 
 Asian American 679 (2.1) 516 (2.2) 241 (4) 34 (1.2) 
 Other 2529 (7.8) 2358 (10) 691 (11.6) 286 (9.7) 
APR-DRG SOI     
 Minor 24 829 (76.4) 16 644 (70.7) 3141 (52.7) 1697 (57.6) 
 Moderate 7385 (22.7) 6636 (28.2) 2562 (43) 957 (32.5) 
 Major 273 (0.8) 251 (1.1) 260 (4.4) 292 (9.9) 

Q1 of 2013 to Q3 of 2015. N = patients from 32 hospitals. Source: PHIS database.10  SOI: severity of illness; —, not applicable.

Table 3 shows the calculated ABCs for each of study populations, along with the number of hospitals used to create these benchmarks. These calculated ABCs are 22.3% of patients with asthma and 19.8% of patients with bronchiolitis having a CXR performed, 17.9% of patients with bronchiolitis receiving bronchodilators, 5.5% of patients with LRTI treated with systemic corticosteroids, and 32.2% of patients with GER treated with acid suppressor therapy. According to the methodology described above, only 3 to 4 hospitals were included in the estimation of the ABCs, representing ∼10% of the population for each cohort.

TABLE 3

ABCs for Conditions Included in the SHM-CWC Pediatric Recommendations

Condition and TreatmentNo. of Hospitals in ABCaABC, %
Asthma 
 Received CXR 22.3 
Bronchiolitis 
 Received CXR 19.8 
 Received reliever 17.9 
LRTI 
 Received systemic steroid 5.5 
GER 
 Received acid suppressor 32.2 
Condition and TreatmentNo. of Hospitals in ABCaABC, %
Asthma 
 Received CXR 22.3 
Bronchiolitis 
 Received CXR 19.8 
 Received reliever 17.9 
LRTI 
 Received systemic steroid 5.5 
GER 
 Received acid suppressor 32.2 

Q1 of 2013 to Q3 of 2015. Source: PHIS database.10 

a

Number of sample hospitals that contributed to the calculation of the ABC by measure.

Figure 1 represents the variation in performance among the participating hospitals for the 5 metrics included in the study in box plots. Each measure, except steroid use in LRTIs, shows significant variability among institutions, as depicted by the whiskers on each box plot: CXR in asthma (median: 34.7%, IQR: 28.5%–45.9%), CXR use in bronchiolitis (median: 34.4%, IQR: 27.9%–49%), bronchodilators use in bronchiolitis (median: 55.4%, IQR: 32.3%–64.9%), and use of acid suppressor therapy in GER (median: 59.4%, IQR: 49.9%–71.2%). The use of systemic corticosteroids in LRTI shows less variance (median: 13.5%, IQR: 11.1%–17.9%). Additionally, we observed a clear gap between hospital performances and ABCs, as represented by the black circles, in all conditions.

FIGURE 1

SHM-CWC pediatric recommendations. Variation among 32 children’s hospitals that are CHA and/or PHIS members from 2013 Q1 to 2015 Q3 is shown. In a box plot, the central rectangle spans the 25th percentile to the 75th percentile. The line inside the rectangle displays the median. The horizontal lines extending from the box indicate the 25th percentile − (1.5 × IQR) and the 75th percentile + (1.5 × IQR). Source: PHIS database.10 

FIGURE 1

SHM-CWC pediatric recommendations. Variation among 32 children’s hospitals that are CHA and/or PHIS members from 2013 Q1 to 2015 Q3 is shown. In a box plot, the central rectangle spans the 25th percentile to the 75th percentile. The line inside the rectangle displays the median. The horizontal lines extending from the box indicate the 25th percentile − (1.5 × IQR) and the 75th percentile + (1.5 × IQR). Source: PHIS database.10 

Close modal

Figure 2 shows the performance of each hospital, benchmarked against the ABCs for each of the SHM-CWC pediatric recommendations in a heat map. In general, more hospitals performed further from the ABC in the use of relievers for bronchiolitis and the use of acid suppressor therapy in GER, represented by the darkest gray boxes (>20 percentage points from benchmark) and medium gray boxes (11–20 percentage points from benchmark). On the other hand, participating hospitals were closer to or met the ABC regarding the avoidance of steroids for LRTI, represented by the more frequent lighter gray (0–10 percentage points from benchmark) and clear boxes (meet benchmark). Few hospitals were closer to or met benchmarks (ABCs) on most metrics, and none were able to meet benchmarks or to reach <10% variability from the benchmark on all 5 metrics.

FIGURE 2

Children’s hospitals’ performance on 5 quality measures on the SHM-CWC pediatric recommendations from 2013 Q1 to 2015 Q3. Source: PHIS database.10 

FIGURE 2

Children’s hospitals’ performance on 5 quality measures on the SHM-CWC pediatric recommendations from 2013 Q1 to 2015 Q3. Source: PHIS database.10 

Close modal

Figure 3 displays bar graphics that show the variation in performance on the 5 metrics among the 32 hospitals included in the study. The shorter bars, closer to the ABC line, represent the best-performing centers, whereas the tallest bars correspond to institutions performing further away from the benchmark. The ranges of variation from the best-performing hospital (lower percentage in each metric) to the worst-performing center (higher percentage) are as follows: CXR in asthma (6.8%–71.2%); CXR in bronchiolitis (7.5%–69.8%); relievers in bronchiolitis (14.3%–82.6%); systemic corticosteroids in LRTI (4.8%–28%); and acid suppressor therapy in GER (26.2%–100%).

FIGURE 3

Variation in Metrics from the SHM-CWC pediatric recommendations from 32 children’s hospitals in the United States from 2013 Q1 to 2015 Q3. The interrupted horizontal line indicates ABCs. Best-performing hospitals (shorter bars) are closer to the ABC line. Source: PHIS database.10  A, CXR for asthma. B, CXR for bronchiolitis. C, Relievers for bronchiolitis. D, Acid suppression for GER. E, Steroids for LRTIs.

FIGURE 3

Variation in Metrics from the SHM-CWC pediatric recommendations from 32 children’s hospitals in the United States from 2013 Q1 to 2015 Q3. The interrupted horizontal line indicates ABCs. Best-performing hospitals (shorter bars) are closer to the ABC line. Source: PHIS database.10  A, CXR for asthma. B, CXR for bronchiolitis. C, Relievers for bronchiolitis. D, Acid suppression for GER. E, Steroids for LRTIs.

Close modal

This report card is displayed in the standard format used by CHA in other clinical report cards. This format allows for a visual and numeric comparison of a given institution or physician group with both the peer group of participating hospitals and with the ABC for each metric. The report card allows customization by selecting specific timeframes and provider groups for benchmarking purposes. CHA- and PHIS-participating institutions can generate their customized, landscape version of the report card by accessing www.childrenshospitals.org.10 

The SHM-CWC pediatric recommendations report card is the latest available for PHIS members. This is the first clinical report card that includes calculated ABCs as benchmarks of excellence and has a focus on overuse. Institutions can use these data to identify areas in which they are significant outliers for local QI projects.

The aggregated results presented here represent the first available data set of the performance of children’s hospital members of CHA after the recommendations were released.

As previously described, the use of ABCs is an original method intended to help individual clinician groups and institutions benchmark their performance. ABCs represent (1) a measurable level of excellence, (2) attainable goals, and (3) a benchmark derived from data in an objective and reproducible manner.57  The calculated ABCs in our study compare well with the ABCs published by Parikh et al in 20145  in 4 of our 5 metrics. The 2 studies used the same database (PHIS), but their study included data from the calendar year 2012, whereas our study included larger cohorts corresponding to 11 quarters (January 2013–September 2015). Furthermore, we used more restrictive criteria to define uncomplicated conditions, adding extreme APR-DRG, intensive care services, and any LOS of more than 5 days as exclusion criteria, in addition to including multiple chronic conditions or comorbidities. Despite some differences in defining uncomplicated cases, the finding of similar ABCs obtained in 2 different periods may suggest that no significant changes were present in practices among the included children’s hospitals since 2012, when the calculation of ABCs took place for the first time.

Peer-reviewed, society-endorsed guidelines for the conditions included in this study are widely available.1113  Clinical guidelines are designed to provide guidance while still preserving individual physician autonomy, modification on the basis of the clinical scenario, and accommodation for patient/family preferences. Although full adherence to clinical guidelines is not expected, reasonable and attainable goals of compliance are. However, as shown in Figs 13, significant variation in care still exists among the tertiary centers included in this study, as well as a clear gap between ABCs and actual performance by most of the institutions. These facts suggest an overuse of resources with an associated increase in costs related to hospital care. These findings have been previously reported by others.5,1417 

Recent evidence suggests some progress in the care of hospitalized children with bronchiolitis since the publication of American Academy of Pediatrics guidelines for this common inpatient condition.1821  However, as we reaffirm in this study, much unwarranted variation is still the norm, specifically in the excessive use of bronchodilators. Improvement will require concerted local efforts aimed at specific conditions. Recent evidence demonstrates that it is possible to change the culture and create meaningful and sustainable improvement through local efforts18  or multicenter QI collaboratives,21  both in tertiary children hospitals and community settings. Similar collaborative efforts to decrease unnecessary care in children with other lower respiratory tract illnesses, including asthma, are lacking. The significant improvements noted through such efforts addressing bronchiolitis, and the evidence of room for improvement demonstrated through our scorecard, could serve as the impetus for such work.

This work adds a newly calculated an achievable benchmark for the use of acid suppressor therapy in children with uncomplicated GER (32.2%), as shown in Table 2. Several studies, including systematic reviews, have described the ineffectiveness of acid-reducing medications in the treatment of children with GER.22,23  Despite this, since the year 2000, a threefold increase in diagnosis (3.4%–12.3%) and a concomitant doubling of the use of proton pump inhibitors (31.5%–62.6%) have occurred in infants.24  A substantial growing body of evidence suggests that this practice may also lead to significant harm.25  Our study suggests that wide variation still exists in the use of acid suppressor therapy in children with GER across institutions (Figs 13). The fact that most institutions are far from meeting this benchmark of excellence, coupled with the strong evidence of ineffectiveness and potential harm of this therapy, makes this topic ripe for QI efforts at local and national levels.

The SHM-CWC pediatric recommendations report card could be specifically tailored by each institution member of the CHA consortium and has been readily available to use for internal QI prioritization since 2016. As shown in Figs 2 and 3, most hospitals tended to underperform on the use of relievers in bronchiolitis and prescription of acid suppressor therapy in GER. On the other hand, the use of steroids in LRTI represents the indicator with the lowest performance gap from the benchmark. Thus, prioritization of QI efforts could involve decreasing bronchodilator use in children with bronchiolitis and prescribing acid suppressor therapy in infants with uncomplicated, physiologic GER.

This report card represents an initial, valid tool to assess ongoing performance on the metrics selected for this study and their comparison with “best practices.” The inclusion of attainable benchmarks, newly and previously described for these specific conditions, makes the report card a much more meaningful instrument for benchmarking, by providing realistic goals on the basis of peer performance.

This study has several limitations, including those limitations associated with administrative and billing database research. Variation in documentation and coding practices may result in the presence of misidentified children in the cohorts and may hinder the risk adjustment for meaningful benchmarking. Additionally, the extensive list of inclusion and exclusion criteria based on chronic and/or comorbid conditions may exclude some uncomplicated cases for which the recommendations were intended.

Future versions of this report card based on data from the more granular International Classification of Diseases, Tenth Revision classification may be more revealing. Also, this study does not include information on whether a hospital implemented any changes or interventions in an attempt to follow the SHM-CWC pediatric recommendations.

Another limitation is the lack of data on hospital care provided at institutions that are not members of CHA. It is estimated that the majority (∼75%) of pediatric hospitalizations in the United States take place in hospitals that are not children’s hospitals.20  Both the ABCs and the aggregated results may be different in that population.

The study does not include directions or suggestions on how to change practices through particular interventions or QI efforts. One final limitation is that the report card is available only to PHIS member institutions. However, by using the methodology and definitions described here and the information provided in the online supplement, other institutions could reproduce the report card and use the calculated ABCs as a benchmarking reference.

A report card for the purpose of assessing and benchmarking 5 metrics derived from the SHM-CWC pediatric recommendations is currently available for CHA and/or PHIS member institutions. In this study, we describe the development of the report card and the performance of these institutions from the first quarter (Q1) of 2013 to the third quarter (Q3) of 2015, along with calculated ABCs for each metric. In this study, we primarily demonstrate variation in practices and provide an initial pathway for change. In the future, individual institutions and collaborative projects can use these dashboards to track their performance and as a roadmap for improvement. QI efforts derived from the report card findings may lead to decreased variation in the care that hospitalized children receive for these common inpatient pediatric illnesses.

We acknowledge the Research Institute and the Medical Library from Nicklaus Children’s Hospital in Miami, Florida, for their assistance and involvement in the preparation and submission of this article.

Dr Reyes and Mr Paulus conceptualized and designed the report card and the study, made a substantial contribution to the analysis and interpretation of data, and drafted, critically reviewed, and revised the manuscript; Ms Hronek made a substantial contribution to the design of the report card, coordinated data collection and analysis and drafted, critically reviewed, and revised the manuscript; Dr Etinger participated in data analysis and interpretation, made significant contributions to the initial draft, and reviewed and revised the manuscript for final submission; Mr Hall participated in the conceptualization and design of the report card, coordinated acquisition of the study data, conducted the statistical analysis, and drafted, critically reviewed, and revised the manuscript; Drs Vachani and Lusk participated in data analysis and interpretation, made significant contributions to the initial draft, and reviewed and revised the manuscript; Mr Emerson participated in the conceptualization and development of the report card and reviewed the manuscript; Ms Huddleson assisted in data collection and analysis and in the design of the report card; Dr Quinonez participated in the conceptualization and design of the initial study draft, made a substantial contribution to the analysis and interpretation of data, and critically drafted, reviewed, and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

1
Quinonez
RA
,
Garber
MD
,
Schroeder
AR
, et al
.
Choosing wisely in pediatric hospital medicine: five opportunities for improved healthcare value
.
J Hosp Med
.
2013
;
8
(
9
):
479
485
2
Schondelmeyer
AC
,
Simmons
JM
,
Statile
AM
, et al
.
Using quality improvement to reduce continuous pulse oximetry use in children with wheezing
.
Pediatrics
.
2015
;
135
(
4
).
3
Serb
C
.
Effective dashboards: what to measure and how to show it
.
Hosp Health Netw
.
2011
;
85
(
6
):
8 p following 40, 2
5
Parikh
K
,
Hall
M
,
Mittal
V
, et al
.
Establishing benchmarks for the hospitalized care of children with asthma, bronchiolitis, and pneumonia
.
Pediatrics
.
2014
;
134
(
3
):
555
562
6
Kiefe
CI
,
Weissman
NW
,
Allison
JJ
,
Farmer
R
,
Weaver
M
,
Williams
OD
.
Identifying achievable benchmarks of care: concepts and methodology
.
Int J Qual Health Care
.
1998
;
10
(
5
):
443
447
7
Weissman
NW
,
Allison
JJ
,
Kiefe
CI
, et al
.
Achievable benchmarks of care: the ABCs of benchmarking
.
J Eval Clin Pract
.
1999
;
5
(
3
):
269
281
8
Mongelluzzo
J
,
Mohamad
Z
,
Ten Have
TR
,
Shah
SS
.
Corticosteroids and mortality in children with bacterial meningitis
.
JAMA
.
2008
;
299
(
17
):
2048
2055
9
Centers for Disease Control and Prevention
.
Classification of diseases, functioning, and disability
.
2013
.
Available at: www.cdc.gov/nchs/icd.htm. Accessed April 1, 2017
10
.
Accessed September 14, 2017
11
Bradley
JS
,
Byington
CL
,
Shah
SS
, et al
;
Pediatric Infectious Diseases Society and the Infectious Diseases Society of America
.
The management of community-acquired pneumonia in infants and children older than 3 months of age: clinical practice guidelines by the Pediatric Infectious Diseases Society and the Infectious Diseases Society of America
.
Clin Infect Dis
.
2011
;
53
(
7
):
e25
e76
12
National Asthma Education and Prevention Program; National Heart, Lung, and Blood Institute
.
Expert panel report III: guidelines for the diagnosis and management of asthma. 2007
.
13
Ralston
S
,
Lieberthal
A
,
Meissner
H
, et al
.
Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis AAP gateway
.
Pediatrics
.
2014
;
134
(
5
):
e1474
e1502
14
Macias
CG
,
Mansbach
JM
,
Fisher
ES
, et al
.
Variability in inpatient management of children hospitalized with bronchiolitis
.
Acad Pediatr
.
2015
;
15
(
1
):
69
76
15
Christakis
DA
,
Cowan
CA
,
Garrison
MM
,
Molteni
R
,
Marcuse
E
,
Zerr
DM
.
Variation in inpatient diagnostic testing and management of bronchiolitis
.
Pediatrics
.
2005
;
115
(
4
):
878
884
16
Bratton
SL
,
Odetola
FO
,
McCollegan
J
,
Cabana
MD
,
Levy
FH
,
Keenan
HT
.
Regional variation in ICU care for pediatric patients with asthma
.
J Pediatr
.
2005
;
147
(
3
):
355
361
17
Florin
TA
,
Byczkowski
T
,
Ruddy
RM
,
Zorc
JJ
,
Test
M
,
Shah
SS
.
Variation in the management of infants hospitalized for bronchiolitis persists after the 2006 American Academy of Pediatrics bronchiolitis guidelines
.
J Pediatr
.
2014
;
165
(
4
):
786
792.e1
18
Mittal
V
,
Hall
M
,
Morse
R
, et al
.
Impact of inpatient bronchiolitis clinical practice guideline implementation on testing and treatment
.
J Pediatr
.
2014
;
165
(
3
):
570
576.e3
19
Parikh
K
,
Hall
M
,
Teach
SJ
.
Bronchiolitis management before and after the AAP guidelines
.
Pediatrics
.
2014
;
133
(
1
).
20
McCulloh
RJ
,
Smitherman
SE
,
Koehn
KL
,
Alverson
BK
.
Assessing the impact of national guidelines on the management of children hospitalized for acute bronchiolitis
.
Pediatr Pulmonol
.
2014
;
49
(
7
):
688
694
21
Ralston
SL
,
Garber
MD
,
Rice-Conboy
E
, et al
;
Value in Inpatient Pediatrics Network Quality Collaborative for Improving Hospital Compliance with AAP Bronchiolitis Guideline (BQIP)
.
A Multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis
.
Pediatrics
.
2016
;
137
(
1
):
e20150851
22
van der Pol
RJ
,
Smits
MJ
,
van Wijk
MP
,
Omari
TI
,
Tabbers
MM
,
Benninga
MA
.
Efficacy of proton-pump inhibitors in children with gastroesophageal reflux disease: a systematic review
.
Pediatrics
.
2011
;
127
(
5
):
925
935
23
Higginbotham
TW
.
Effectiveness and safety of proton pump inhibitors in infantile gastroesophageal reflux disease
.
Ann Pharmacother
.
2010
;
44
(
3
):
572
576
24
Nelson
SP
,
Kothari
S
,
Wu
EQ
,
Beaulieu
N
,
McHale
JM
,
Dabbous
OH
.
Pediatric gastroesophageal reflux disease and acid-related conditions: trends in incidence of diagnosis and acid suppression therapy
.
J Med Econ
.
2009
;
12
(
4
):
348
355
25
Chung
EY
,
Yardley
J
.
Are there risks associated with empiric acid suppression treatment of infants and children suspected of having gastroesophageal reflux disease?
Hosp Pediatr
.
2013
;
3
(
1
):
16
23

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data