BACKGROUND:

The Pediatric Infectious Diseases Society and Infectious Diseases Society of America national childhood community-acquired pneumonia (CAP) guideline encouraged the standard evaluation and treatment of children who were managed as outpatients. Our objectives were to (1) increase adherence to guideline-recommended diagnostics and antibiotic treatment of CAP at 5 pediatric primary care practices (PPCPs) by using quality-improvement methods and (2) evaluate the association between guideline adherence and unscheduled follow-up visits.

METHODS:

Immunocompetent children >3 months of age with no complex chronic conditions and who were diagnosed with CAP were eligible for inclusion in this stepped-wedge study. Interventions were focused on education, knowledge of colleagues’ prescribing practices, and feedback sessions. Statistical process control charts were used to assess changes in recommendations and antibiotic treatment. Unscheduled follow-up visits were compared across time by using generalized estimating equations that were clustered by PPCP.

RESULTS:

CAP was diagnosed in 1906 children. Guideline recommended therapy and pulse oximetry use increased from a mean baseline of 24.9% to a mean of 68.0% and from 4.3% to 85.0%, respectively, over the study period. Among children >5 years of age, but not among those who were younger, the receipt of guideline recommended antibiotics, as compared with nonguideline therapy, was associated with the increased likelihood of unscheduled follow-up (adjusted odds ratio, 2.12; 95% confidence interval: 1.31–3.43). Chest radiographs and complete blood cell counts were rarely performed at baseline.

CONCLUSIONS:

Recommendations for limited use of chest radiographs and complete blood cell counts and standardized antibiotic therapy in children is supported at PPCPs. However, the guideline may need to include macrolide monotherapy as appropriate antibiotic therapy for older children.

Community-acquired pneumonia (CAP) is diagnosed in >1.2 million children in outpatient settings and emergency departments (EDs) each year in the United States.1,2 Most antibiotic prescribing for CAP occurs in the outpatient setting. In 2011, members of the Pediatric Infectious Diseases Society and Infectious Diseases Society of America published an evidence-based guideline for the management of CAP in children. The authors of the recommendations encourage prescribing narrow spectrum antibiotics, increasing reliance on vital sign measurements (including pulse oximetry) for clinical decision-making, and reducing the routine performance of laboratory testing and radiography. Antibiotic therapy recommendations included β-lactam monotherapy (eg, amoxicillin) as first-line therapy for children with CAP because β-lactams are effective against the most common bacterial cause of childhood CAP, Streptococcus pneumoniae,3 and macrolides in combination with β-lactams when atypical pneumonia (eg, Mycoplasma pneumoniae) is suspected.

Pulse oximetry is recommended to be performed in children with suspected CAP because the presence of hypoxemia should guide treatment decisions. However, it is unclear if performing pulse oximetry on all children in an outpatient setting is feasible. The recommendations to forgo complete blood cell (CBC) counts and chest radiographs (CXRs) are among the most controversial recommendations and are based on a limited number of studies that were conducted on children who were hospitalized.3,8 Authors of 1 study demonstrated that more than one-third of children who were undergoing a CXR for suspicion of bacterial CAP did not have radiologic findings to support the diagnosis.9 The authors concluded that the recommendation to forgo a routine CXR would lead to unnecessary prescribing of antibiotics. However, it remains unclear how often CXRs are ordered in the outpatient setting and whether the ordering of the CXR can influence treatment decisions. Antibiotics are typically prescribed before the results from the CXR are available, with the potential exception for CXRs that are ordered in the ED or urgent care settings in which the time between obtaining the CXR and subsequent interpretation is shorter.

The passive adoption of a guideline is inefficient and often leads to incomplete adoption.10 Thus, our primary objective was to proactively implement and evaluate the effect of adopting the national guideline in the management of children who were diagnosed with CAP in the outpatient setting. However, the guideline is based on sparse data in the outpatient setting; thus, our secondary objective was to evaluate the unintended consequences, such as an increase in unscheduled follow-up visits, that may occur after the adoption of the guideline.

In this quasi-experimental study, we used a stepped-wedge design to test and implement interventions in each of 5 outpatient pediatric primary care practices (PPCPs). This design was used to allow for the sequential delivery of interventions to each clinic over a period of 21 months because it was not feasible to deliver the same intervention simultaneously to all PPCPs.11,12 This study design was also used to allow the assessment of the intervention within a practice by comparing information to their own baseline data and comparing information between practices in a set time period. Quality improvement methods were followed to implement and evaluate the adoption of the guideline. First, a process map was created that depicted the current practice for the process of care variables at each community practice (Figs 1 and 2). Each process map was developed by the PPCP per their flow for a typical respiratory-related patient; thus, no 2 PPCPs had the same process map. Second, a key driver diagram was created, and a modified failure mode and effects analysis was conducted to identify potential failures within the process.13,14 The specific, measurable, achievable, relevant, and time-oriented aims were to increase the proportion of children who were diagnosed with CAP at a PCPP who received (1) recommended antibiotics from 24.9% to 80%, (2) pulse oximetry measures from 4.3% to 80%, and (3) the number of CXR or CBC ordered from 9.7% and 6.4%, respectively, to 5% within 2 years. Third, multiple interventions were designed and tested to address the key drivers within the individual practices. Although the implementation of some interventions was less dependent on PPCP process (eg, peer review of charts), others (eg, performing pulse oximetry) were dependent on the patient flow of the practice. The key drivers included the accurate knowledge of guidelines, provider buy-in, effective communication between care providers, and preoccupation with failures. Changes to the interventions were tested through multiple Plan-Do-Study-Act cycles.15 Interventions were rolled out on the basis of the availability of the PPCP for the initial education bundle intervention. Each additional intervention was rolled out within 3 to 6 months of the previous intervention.

FIGURE 1

Example of process map from 1 of the participating PPCPs. Process map example 1. CC, chronic condition; HPI, history of present illness.

FIGURE 1

Example of process map from 1 of the participating PPCPs. Process map example 1. CC, chronic condition; HPI, history of present illness.

Close modal
FIGURE 2

Example of process map from 1 of the participating PPCPs. Process map example 2. LRTI, lower respiratory tract infection; MA, medical assistant; MD, doctor of medicine; NP, nurse practitioner; RN, registered nurse.

FIGURE 2

Example of process map from 1 of the participating PPCPs. Process map example 2. LRTI, lower respiratory tract infection; MA, medical assistant; MD, doctor of medicine; NP, nurse practitioner; RN, registered nurse.

Close modal

PPCPs providers had flexibility as to the best method for implementation of the intervention within their systems. Four interventions were implemented: (1) an educational, in-person meeting with each of the PPCPs; (2) peer chart review; (3) electronic medical record (EMR) reminder; and (4) identification and mitigation.

Education

During the in-person meeting, the guideline was reviewed, the 4 recommendations of interest were emphasized, and pens with the recommendations printed on them were distributed (Fig 3). The providers received 20 continuing medical education credits and Maintenance of Certification Part IV credit from the American Board of Pediatrics for their complete participation in the study.16 

FIGURE 3

Pens with printed recommendations. BID, twice daily; LRTI, lower respiratory tract infection; MA, medical assistant; NP, nurse practitioner, PNA, pneumonia; PO, per os; TID, thrice daily.

FIGURE 3

Pens with printed recommendations. BID, twice daily; LRTI, lower respiratory tract infection; MA, medical assistant; NP, nurse practitioner, PNA, pneumonia; PO, per os; TID, thrice daily.

Close modal

Peer Chart Review

Peer chart review was developed and modified by each PPCP. Providers within a practice reviewed the charts of patients with CAP that a colleague had treated in the previous month (Fig 4). This intervention was used to address multiple key drivers because it increased communication between providers and increased the standardization of care among providers within a PPCP. This intervention was also used to identify situations in which recommended antibiotic therapy was not prescribed for reasons other than the patient’s drug allergy (eg, previous diagnosis of otitis media).

FIGURE 4

Pneumonia chart review. Amox, amoxicillin; DX, diagnosis; F/U, follow-up; ID, identification; LRTI, lower respiratory tract infection; MMR, measles-mumps-rubella; OM, otitis media; pna, pneumonia; PVC7, pneumococcal conjugate vaccine 7; PVC13, pneumococcal conjugate vaccine 13.

FIGURE 4

Pneumonia chart review. Amox, amoxicillin; DX, diagnosis; F/U, follow-up; ID, identification; LRTI, lower respiratory tract infection; MMR, measles-mumps-rubella; OM, otitis media; pna, pneumonia; PVC7, pneumococcal conjugate vaccine 7; PVC13, pneumococcal conjugate vaccine 13.

Close modal

EMRs

A reminder within the EMR was displayed to deliver real-time guidance on the guideline when a provider identified the patient as having CAP. This reminder was modified by the providers of the PPCPs because there were 4 different EMR systems among the 5 PPCPs.

Identification and Mitigation

The final intervention occurred through a self-report of barriers to implementation within each PPCP. Examples of barriers included the onboarding of new providers within the PPCP, appropriate dosing of amoxicillin based on children with “adult-weights” and/or based on documented penicillin-resistant S pneumoniae. Control charts were provided monthly for each practice and for each provider within a practice. The study lead at each site provided feedback to individual providers whose performance was not within the normal variation of the measure (eg, antibiotic prescribing) of the PPCPs. This feedback was provided via e-mails, one-on-one conversations, or through regular PPCP meetings.

Baseline data were collected retrospectively for children who were 3 months to 18 years of age who visited the PPCP between July 1, 2010, and June 30, 2012. Children during this period were identified by using the International Classification of Diseases, Ninth Revision diagnosis codes for pneumonia (ie, 480, 482.3, 482.8, 482.9, 483, and 486). Children with an immunocompromising or chronic medical condition that predisposed them to severe or recurrent CAP were excluded.12 EMR systems were implemented at each PPCP at different times during the baseline period; thus, data for the baseline period were collected either from the EMR or paper records as available. Data for the intervention period were collected between July 1, 2013, and March 30, 2015. All data from the intervention period were obtained via chart review by physicians from the PPCPs by using a structured case report form (Fig 4). Age was stratified a priori for the analysis because children who are >5 years of age and children who are ≤5 years of age as school-aged children are more likely to be infected with an atypical bacterium, and younger children are likely to be infected with viruses.17 

There were 4 process measures that included the following: (1) receiving recommended antibiotic therapy, (2) performing pulse oximetry, (3) ordering of a CXR, and (4) not performing CBC count. Guideline antibiotic therapy was defined as the empirical receipt of the following: (1) amoxicillin; or (2) second or third generation cephalosporin for children with a penicillin allergy; or (3) azithromycin on day 1 in combination with either antibiotic 1 or 2 above if atypical bacteria were suspected clinically; or (4) cefdinir, cefuroxime, or amoxicillin-clavulanate if the child was unimmunized or under-immunized. Performing pulse oximetry was defined as the documentation of the percutaneous oxygen saturation in the chart. Hypoxia was defined as a pulse oximetry reading of <92%. Although the ordering of CXRs was defined as a failure, the authors of the guideline recommend ordering CXRs for patients who were seen in the PPCP to verify the presence or absence of pneumonia-related complications, specifically those patients with documented hypoxia; thus, the goal was 10% acknowledging that there are circumstances in which ordering a CXR would have been recommended. The guideline does not recommend the ordering of CBC count in the PPCP and thus an order of a CBC count was considered a failure.

The balancing measure was an unscheduled follow-up visit (ie, in the PPCP, ED, or as a hospital admission) within 14 days of the initial diagnosis of CAP with documentation that the follow-up was related to the previous diagnosis of CAP.

Demographic variables that were obtained included age, sex, race, and ethnicity. Clinical variables were based on documentation by the provider and included fever, asthma, wheezing, up-to-date immunizations, receipt of current influenza vaccination, and antibiotic allergy.

Separate p-charts were created to depict the weekly proportions of eligible patients for each process while evaluating the effects of different interventions over time. Special causes were identified by using established control chart rules.18 The rules included a single point outside of the control limits, 8 or more consecutive points above or below the centerline, or 6 consecutive points trending up or down.18 

To ensure that changes in the balancing measure were due to the implementation of the interventions and not due to a shift in the study population over time, characteristics were compared from the baseline to the intervention period by using a Fisher’s exact test, χ2 test, and Wilcoxon-rank sum test as appropriate. The association between the 4 process measures of interest and an unscheduled follow-up visit was assessed by using a generalized estimating equation (GEE) clustering on PPCP and time of the intervention. The GEE model was specified by using a compound symmetry correlation structure and included all 4 process measures in addition to fever, asthma, and wheezing on presentation. An interaction term between age category and recommended antibiotic therapy was assessed in the multivariable GEE model and remained in the model because it was statistically significant. Analyses were performed by using SAS software 9.4 (SAS Institute, Inc, Cary, NC).

This study was reviewed by the Institutional Review Board at Cincinnati Children’s Hospital Medical Center and was determined not to be human subjects research.

A total of 1906 episodes of CAP occurred during the study period, with 50.3% (n = 959) of episodes occurring during the preintervention period. Children were on average 6.0 years old (interquartile range 3.1–9.8) and race was not reported for 51.7% of the cohort (Table 1). Only 3.5% (n = 66) of children who were enrolled did not receive antibiotics. Children who visited their PPCP in the intervention phase were more likely to have a documented drug allergy to penicillin (11% in the intervention phase vs 9.2% in the preintervention period; P value <.01).

TABLE 1

Demographics and Clinical Characteristics of Patient Population

CharacteristicTotal (n = 1906)Preintervention Period (n = 959)Intervention Period (n = 947)P
Age, y, median (IQR) 6.0 (3.1–9.8) 6.4 (3.3–10.1) 5.5 (2.9–9.6) .003 
Male sex, n (%) 997 (52.4) 500 (52.1) 497 (52.7) .80 
Fever, n (%) 1242 (65.2) 645 (67.3) 597 (63.0) .05 
Hypoxia, n (%)     
 Pulse oxygen not performed 1155 (60.7) 918 (95.7) 237 (25.1) <.001 
 Normal oxygen saturation, ≥92% 729 (38.3) 38 (4.0) 691 (73.2) .08 
 Abnormal oxygen saturation or hypoxic, <92% 19 (1.0) 3 (0.3) 16 (1.7)  
Asthma, n (%) 324 (17.0) 151 (15.7) 173 (18.3) .14 
Wheezing, n (%) 373 (19.6) 173 (18.1) 200 (21.2) .09 
Immunizations up to date, n (%) 1734 (91.0) 869 (90.6) 865 (91.3) .04 
Receipt of influenza vaccination, n (%) 1048 (55.0) 512 (53.4) 536 (56.6) <.001 
Medication allergy, n (%) 222 (13.8) 88 (13.1) 134 (14.2) .52 
 Penicillin 163 (11.6) 59 (9.2) 104 (11.0) <.001 
 Macrolide 10 (0.5) 5 (0.5) 5 (0.5) .98 
 Cephalosporin 39 (2.0) 21 (2.2) 18 (1.9) .66 
 Vancomycin 2 (0.1) 1 (0.1) 1 (0.1) .99 
 Penicillin and cephalosporin 9 (0.5) 6 (0.6) 3 (0.3) .33 
 Penicillin and macrolide 1 (0.1) 0 (0.0) 1 (0.1) .31 
CharacteristicTotal (n = 1906)Preintervention Period (n = 959)Intervention Period (n = 947)P
Age, y, median (IQR) 6.0 (3.1–9.8) 6.4 (3.3–10.1) 5.5 (2.9–9.6) .003 
Male sex, n (%) 997 (52.4) 500 (52.1) 497 (52.7) .80 
Fever, n (%) 1242 (65.2) 645 (67.3) 597 (63.0) .05 
Hypoxia, n (%)     
 Pulse oxygen not performed 1155 (60.7) 918 (95.7) 237 (25.1) <.001 
 Normal oxygen saturation, ≥92% 729 (38.3) 38 (4.0) 691 (73.2) .08 
 Abnormal oxygen saturation or hypoxic, <92% 19 (1.0) 3 (0.3) 16 (1.7)  
Asthma, n (%) 324 (17.0) 151 (15.7) 173 (18.3) .14 
Wheezing, n (%) 373 (19.6) 173 (18.1) 200 (21.2) .09 
Immunizations up to date, n (%) 1734 (91.0) 869 (90.6) 865 (91.3) .04 
Receipt of influenza vaccination, n (%) 1048 (55.0) 512 (53.4) 536 (56.6) <.001 
Medication allergy, n (%) 222 (13.8) 88 (13.1) 134 (14.2) .52 
 Penicillin 163 (11.6) 59 (9.2) 104 (11.0) <.001 
 Macrolide 10 (0.5) 5 (0.5) 5 (0.5) .98 
 Cephalosporin 39 (2.0) 21 (2.2) 18 (1.9) .66 
 Vancomycin 2 (0.1) 1 (0.1) 1 (0.1) .99 
 Penicillin and cephalosporin 9 (0.5) 6 (0.6) 3 (0.3) .33 
 Penicillin and macrolide 1 (0.1) 0 (0.0) 1 (0.1) .31 

IQR, interquartile range.

Across the 5 PPCPs with 10 clinic locations, there was an average of 9.2 providers per practice (range 3–19 providers), and providers were on average 14.5 years out from their residency training (range 1–57 years). The PPCPs were geographically disbursed over urban, suburban, and rural areas. The primary payers were largely commercial insurance (range 64%–95%), followed by Medicaid (range 4%–35%), and lastly self-payers (range 1%–3.3%).

The proportion of recommended antibiotic prescriptions increased from a median of 24.9% to 68.0% (Fig 5). However, there was not a statistical difference in the overall proportion of children who were prescribed antibiotics during this time (97% in the preintervention period vs 98% in the intervention period; P = .26). Nonguideline recommended antibiotics were prescribed most commonly for patients who received an antibiotic within the previous 30 days for a different diagnosis (eg, recurrent otitis media, urinary tract infection; Fig 6).

FIGURE 5

P-chart of percent of guideline recommended antibiotics prescribed over time. On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the period of time the practice was performing peer chart review, before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

FIGURE 5

P-chart of percent of guideline recommended antibiotics prescribed over time. On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the period of time the practice was performing peer chart review, before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

Close modal
FIGURE 6

Pareto chart of reasons for not prescribing guideline antibiotics.

FIGURE 6

Pareto chart of reasons for not prescribing guideline antibiotics.

Close modal

The proportion of children in whom pulse oximetry was performed increased from a mean of 4.3% to 85.0% (Fig 7). Pulse oximetry was performed statistically more often in the intervention period compared with the preintervention period, but the proportion of children with hypoxia was not statistically different (0.3% and 1.7%, respectively; P = .08; Table 1).

FIGURE 7

P-chart of the percentage of pulse oximetry measures that were obtained over time (black dotted line indicates goal, solid black line indicates centerline, solid gray line indicates data points, and light gray lines indicate control limits). On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the time the practice was performing peer chart review, before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

FIGURE 7

P-chart of the percentage of pulse oximetry measures that were obtained over time (black dotted line indicates goal, solid black line indicates centerline, solid gray line indicates data points, and light gray lines indicate control limits). On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the time the practice was performing peer chart review, before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

Close modal

The proportion of CXRs decreased from 9.7% to 5.0% between the preintervention and intervention periods (Fig 8). The proportion of CBCs also decreased from 6.4% to 1.5% (Fig 9).

FIGURE 8

P-chart of the percentage of CXR orders that were obtained over time. On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the time the practice was performing peer chart review, before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

FIGURE 8

P-chart of the percentage of CXR orders that were obtained over time. On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the time the practice was performing peer chart review, before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

Close modal
FIGURE 9

P-chart of the percentage of CBC count orders that were obtained over time. On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the time the practice was performing peer chart review before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

FIGURE 9

P-chart of the percentage of CBC count orders that were obtained over time. On the p-chart, the black dotted line indicates the goal the team was aiming to achieve, the solid black line indicates the centerline, the solid gray line connects each month’s data points, and the light gray lines indicate control limits. Each row of the table below the graph represents 1 of the practices, with the white space indicating the preintervention period, the light gray shading indicating the educational intervention, the subsequent darker gray shading indicating the time the practice was performing peer chart review before implementing the EMR reminder (the next shaded box), and the darkest gray indicates the amount of time the practice was identifying and mitigating barriers.

Close modal

There were 201 unscheduled follow-up visits, including 41 (2.2%) ED visits and 8 (0.4%) hospitalizations. There was no statistical difference between unscheduled follow-up visits across the cohort in the preintervention (10.5%) and intervention (10.7%) periods (P = .52). Children who presented to the PPCP with fever, asthma, or who had a CXR performed were more likely to have an unscheduled follow-up visit (Table 2). Children >5 years old who received guideline antibiotic therapy had greater odds of any unscheduled follow-up compared with patients who did not receive the recommended antibiotic (Table 2). However, children ≤5 years of age who received guideline antibiotic therapy were not statistically different in odds of any unscheduled follow-up visits compared with those who did not receive recommended antibiotics (Table 2).

TABLE 2

Association Between Guideline Recommendations and Unscheduled Follow-up Visits to the Primary Care Physician or ED or Hospitalization

Adjusted Unscheduled Clinic Visita,b OR (95% CI)
CXR performed 1.64 (1.04–2.60) 
CBC obtained 0.76 (0.27–2.18) 
Pulse oximetry performed 0.96 (0.70–1.32) 
Children ≤5 y old receiving guideline-recommended antibiotics 0.75 (0.43–1.30) 
Children >5 y old receiving guideline-recommended antibiotics 2.12 (1.31–3.43) 
Fever 1.66 (1.31–2.09) 
Asthma 1.53 (1.10–2.14) 
Wheezing 0.99 (0.63–1.55) 
Time, mo 1.01 (0.97–1.06) 
Adjusted Unscheduled Clinic Visita,b OR (95% CI)
CXR performed 1.64 (1.04–2.60) 
CBC obtained 0.76 (0.27–2.18) 
Pulse oximetry performed 0.96 (0.70–1.32) 
Children ≤5 y old receiving guideline-recommended antibiotics 0.75 (0.43–1.30) 
Children >5 y old receiving guideline-recommended antibiotics 2.12 (1.31–3.43) 
Fever 1.66 (1.31–2.09) 
Asthma 1.53 (1.10–2.14) 
Wheezing 0.99 (0.63–1.55) 
Time, mo 1.01 (0.97–1.06) 

CI, confidence interval; OR, odds ratio.

a

Adjusted for all listed variables and primary care practice.

b

Reference category for a given characteristic is not having the characteristic.

Of the 201 unscheduled follow-up visits, no change was made to antibiotic coverage in 136 (68%) visits, coverage was broadened (eg, amoxicillin to cephalosporin and macrolide) in 53 (26.4%) visits, and coverage was narrowed (eg, cephalosporin to amoxicillin) in 12 (6%) follow-up visits. However, patients were statistically more likely to have their antibiotic coverage broadened in the intervention period compared with patients in the preintervention period (21.8% preintervention period vs 38.6% intervention period; P = .004). The most common change in antibiotic therapy during the unscheduled follow-up visit was adding macrolide therapy to penicillin or cephalosporin therapy which was prescribed at the index visit.

The interventions were used to increase adherence to the guideline recommendations for the outpatient management of CAP. Unlike in previously published studies, reducing diagnostic testing did not change the overall proportion of antibiotics that were prescribed in this population.9 Receiving recommended antibiotic therapy in children >5 years old was associated with a twofold increase in unscheduled follow-up visits, and performing pulse oximetry did not statistically change the proportion of patients who were found to be hypoxic.

We found that >75% of children who are diagnosed with CAP in the ED, urgent care, or a PPCP receive an antibiotic.19 However, >50% of antibiotic prescriptions for acute respiratory infections are unnecessary and can lead to an increase in antibiotic-resistant organisms in the community.20 Thus, efforts to standardize antibiotic prescribing in the PPCP are important. Although authors of multiple studies have demonstrated an improved adherence to guideline-recommended antibiotic therapy among children who are hospitalized who are diagnosed with CAP,21,23 few researchers have focused on the outpatient setting.24 The authors of 1 study implemented an outpatient antimicrobial stewardship program across a single hospital-affiliated network of primary care practices; narrow spectrum antibiotic prescribing for pneumonia increased by 11.5% over a 1-year period.24 In contrast, the current study, in which we incorporated interventions that were designed for application across different practice environments and EMR systems, was associated with an increase of 43.1% in recommended antibiotics.

Children >5 years of age were more than twice as likely to have an unscheduled follow-up visit that was independent of the time spent on the interventions. Most of the children who were categorized as not receiving guideline-recommended antibiotic therapy received macrolide monotherapy (73% in the preintervention period and 41% in the intervention period); however, a change in antibiotics at the unscheduled follow-up visit occurred in <5% of the children who received macrolide monotherapy. In a previous study, children >5 years of age who received β-lactam and macrolide combination therapy had lower odds of treatment failure than those who received β-lactam monotherapy.25 In another study, there was no statistical difference in treatment failure among children who received macrolide monotherapy versus β-lactam monotherapy, regardless of age.26 In studies of children hospitalized with CAP, there was no clinical benefit found of macrolide and β-lactam combination therapy versus β-lactam monotherapy.27,28 These results may indicate that a subpopulation of children with pneumonia, presumably caused by atypical bacteria, may benefit from macrolide monotherapy. However, most children this age may have pneumonia that will resolve without antibiotic therapy.

The authors of this guideline recommend that all children have a pulse oximetry measurement if pneumonia or hypoxemia is suspected because hypoxemia should guide site-of-care decisions.3 The increased use of pulse oximetry was not associated with increased identification of hypoxia or decreased unscheduled follow-up visits even when adjusting for markers of illness severity in children with CAP. Our findings indicate that there may be little to gain from routinely performing pulse oximetry in the PPCP.

CXRs remain an imperfect tool for confirming pneumonia etiology, although they are routinely obtained to diagnose pneumonia in the ED and hospital.29 Authors of 1 study suggested that diagnosing children with clinical signs and symptoms of pneumonia without a CXR would lead to 55% to 65% of children being unnecessarily treated with antibiotics.9 Authors of another study found that ED providers who had a high suspicion of pneumonia for their patients had a sixfold increased odds of prescribing antibiotics regardless of CXR results, revealing that CXR findings do not typically alter a provider’s decision to prescribe antibiotics.30 No statistical differences in antibiotic prescribing between the preintervention and intervention periods was detected, although there was a 4.7% decrease in the ordering of CXRs during the study period. We support the recommendation that CXRs are not routinely required for the diagnoses of CAP in the PPCP.

This study had several limitations. First, PPCPs had different processes for initial evaluation and follow-up for patients (eg, every patient with pneumonia has scheduled telephone follow-up 2 days after the initial diagnosis versus no scheduled follow-up is made at the time of the initial diagnosis). In addition, the physicians were not blinded to the prescribed antibiotic therapy; therefore, unconscious bias may have existed to scheduling more follow-up visits when prescribing recommended therapy. However, within a practice, the determination of scheduled versus unscheduled follow-ups did not formally change during the study, thus indicating nondifferential misclassification of the balancing measure. Second, the appropriateness of the antibiotic that was prescribed could not be assessed because causative pathogens were not identified. Third, the staff of the 5 PPCPs volunteered to participate in the study and have an established relationship with Cincinnati Children’s Hospital Medical Center. Therefore, the population may not be generalizable to all PPCPs. Lastly, it was difficult to report the sustainability of the interventions because access to the different EMRs was limited. However, the interventions were developed and implemented per the workflow of each PPCP and included higher reliability interventions (eg, EMR reminders) that increase the likelihood of sustainability.

We actively implemented a national guideline in the PPCP and further support the recommendations for decreased CXR and CBC count ordering in the outpatient setting. The benefit of obtaining a pulse oximeter reading in the PPCP remains unclear. Macrolide monotherapy may need to be considered as empirical antibiotic therapy for children >5 years of age.

     
  • CAP

    community-acquired pneumonia

  •  
  • CBC

    complete blood cell

  •  
  • CXR

    chest radiograph

  •  
  • ED

    emergency department

  •  
  • EMR

    electronic medical record

  •  
  • GEE

    generalized estimating equation

  •  
  • PPCP

    pediatric primary care practice

Dr Ambroggio participated in the design of the study, developed the data collection criteria, conducted the statistical analysis, executed interventions, and drafted the initial manuscript; Mrs Mangeot participated in the design of the study, conducted the statistical analysis, and reviewed all drafts of the manuscript; Drs Murtagh Kurowski and Brinkman participated in the design of the study and reviewed all drafts of the manuscript; Drs Graham, Korn, Strasser, Cavallo, and Brady participated in the design of the study, performed chart reviews, designed and executed interventions, and reviewed all drafts of the manuscript; Ms Campanella participated in the design of the study, developed the data collection criteria, participated in data collection, and reviewed all drafts of the manuscript; Ms Clohessy participated in data collection, executed interventions, and reviewed all drafts of the manuscript; Dr Shah participated in the design of the study, designed and executed interventions, and reviewed all drafts of the manuscript; and all authors approved the final manuscript for submission.

FUNDING: Supported by the Cincinnati Children’s Hospital Medical Center Place Outcomes Research Award.

We are grateful for the time and effort that Dr Mark Dine, who died on August 16, 2017, committed to this study, to his practice, and to all his patients over his 60 years of practicing medicine. He will be missed. The authors are also grateful to all the providers, staff, and patients at the 5 participating practices for their involvement in this study.

1
Kronman
MP
,
Hersh
AL
,
Feng
R
,
Huang
YS
,
Lee
GE
,
Shah
SS
.
Ambulatory visit rates and antibiotic prescribing for children with pneumonia, 1994-2007.
Pediatrics
.
2011
;
127
(
3
):
411
418
[PubMed]
2
Neuman
MI
,
Shah
SS
,
Shapiro
DJ
,
Hersh
AL
.
Emergency department management of childhood pneumonia in the United States prior to publication of national guidelines.
Acad Emerg Med
.
2013
;
20
(
3
):
240
246
[PubMed]
3
Bradley
JS
,
Byington
CL
,
Shah
SS
, et al;
Pediatric Infectious Diseases Society
;
Infectious Diseases Society of America
.
The management of community-acquired pneumonia in infants and children older than 3 months of age: clinical practice guidelines by the Pediatric Infectious Diseases Society and the Infectious Diseases Society of America.
Clin Infect Dis
.
2011
;
53
(
7
):
e25
e76
[PubMed]
4
Swingler
GH
,
Hussey
GD
,
Zwarenstein
M
.
Randomised controlled trial of clinical outcome after chest radiograph in ambulatory acute lower-respiratory infection in children.
Lancet
.
1998
;
351
(
9100
):
404
408
[PubMed]
5
Swingler
GH
,
Zwarenstein
M
.
Chest radiograph in acute respiratory infections [published withdrawal appears in Cochrane Database Syst Rev. 2009;(4):CD001268].
Cochrane Database Syst Rev
.
2008
;(
1
):
CD001268
[PubMed]
6
Novack
V
,
Avnon
LS
,
Smolyakov
A
,
Barnea
R
,
Jotkowitz
A
,
Schlaeffer
F
.
Disagreement in the interpretation of chest radiographs among specialists and clinical outcomes of patients hospitalized with suspected pneumonia.
Eur J Intern Med
.
2006
;
17
(
1
):
43
47
[PubMed]
7
Alario
AJ
,
McCarthy
PL
,
Markowitz
R
,
Kornguth
P
,
Rosenfield
N
,
Leventhal
JM
.
Usefulness of chest radiographs in children with acute lower respiratory tract disease.
J Pediatr
.
1987
;
111
(
2
):
187
193
[PubMed]
8
Grossman
LK
,
Caplan
SE
.
Clinical, laboratory, and radiological information in the diagnosis of pneumonia in children.
Ann Emerg Med
.
1988
;
17
(
1
):
43
46
[PubMed]
9
Zimmerman
DR
,
Kovalski
N
,
Fields
S
,
Lumelsky
D
,
Miron
D
.
Diagnosis of childhood pneumonia: clinical assessment without radiological confirmation may lead to overtreatment.
Pediatr Emerg Care
.
2012
;
28
(
7
):
646
649
[PubMed]
10
Balas
EA
,
Boren
SA
.
Managing clinical knowledge for health care improvement.
Yearb Med Inform
.
2000
;(
1
):
65
70
[PubMed]
11
Mdege
ND
,
Man
MS
,
Taylor Nee Brown
CA
,
Torgerson
DJ
.
Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation.
J Clin Epidemiol
.
2011
;
64
(
9
):
936
948
[PubMed]
12
Feudtner
C
,
Feinstein
JA
,
Zhong
W
,
Hall
M
,
Dai
D
.
Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation.
BMC Pediatr
.
2014
;
14
:
199
[PubMed]
13
Cohen
MR
,
Senders
J
,
Davis
NM
.
Failure mode and effects analysis: a novel approach to avoiding dangerous medication errors and accidents.
Hosp Pharm
.
1994
;
29
(
4
):
319
330
[PubMed]
14
DeRosier
J
,
Stalhandske
E
,
Bagian
JP
,
Nudell
T
.
Using health care failure mode and effect analysis: the VA National Center for Patient Safety’s prospective risk analysis system.
Jt Comm J Qual Improv
.
2002
;
28
(
5
):
248
267, 209
15
Langley
G
,
Nolan
K
,
Nolan
T
,
Norman
C
,
Provost
L
.
The Improvement Guide: A Practical Approach to Enhancing Organizational Performance
. 1st ed.
New York, NY
:
John Wiley and Sons
;
1996
16
American Board of Pediatrics
. Improving professional practice and quality improvement (part 4). 2010. Available at: https://www.abp.org/content/improving-professional-practice-part-4. Accessed June 1, 2010
17
Jain
S
,
Williams
DJ
,
Arnold
SR
, et al;
CDC EPIC Study Team
.
Community-acquired pneumonia requiring hospitalization among U.S. children.
N Engl J Med
.
2015
;
372
(
9
):
835
845
[PubMed]
18
Provost
L
,
Murray
S
.
The Health Care Data Guide: Learning From Data for Improvement
, 1st ed.
New York, NY
:
John Wiley and Sons
;
2011
19
Fleming-Dutra
KE
,
Hersh
AL
,
Shapiro
DJ
, et al
.
Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010-2011.
JAMA
.
2016
;
315
(
17
):
1864
1873
[PubMed]
20
Kronman
MP
,
Zhou
C
,
Mangione-Smith
R
.
Bacterial prevalence and antimicrobial prescribing trends for acute respiratory tract infections.
Pediatrics
.
2014
;
134
(
4
). Available at: www.pediatrics.org/cgi/content/full/134/4/e956
[PubMed]
21
Ambroggio
L
,
Thomson
J
,
Murtagh Kurowski
E
, et al
.
Quality improvement methods increase appropriate antibiotic prescribing for childhood pneumonia.
Pediatrics
.
2013
;
131
(
5
). Available at: www.pediatrics.org/cgi/content/full/131/5/e1623
[PubMed]
22
Newman
RE
,
Hedican
EB
,
Herigon
JC
,
Williams
DD
,
Williams
AR
,
Newland
JG
.
Impact of a guideline on management of children hospitalized with community-acquired pneumonia.
Pediatrics
.
2012
;
129
(
3
). Available at: www.pediatrics.org/cgi/content/full/129/3/e597
[PubMed]
23
Neuman
MI
,
Hall
M
,
Hersh
AL
, et al
.
Influence of hospital guidelines on management of children hospitalized with pneumonia.
Pediatrics
.
2012
;
130
(
5
). Available at: www.pediatrics.org/cgi/content/full/130/5/e823
[PubMed]
24
Gerber
JS
,
Prasad
PA
,
Fiks
AG
, et al
.
Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial.
JAMA
.
2013
;
309
(
22
):
2345
2352
[PubMed]
25
Ambroggio
L
,
Test
M
,
Metlay
JP
, et al
.
Beta-lactam versus beta- lactam/macrolide therapy in pediatric outpatient pneumonia.
Pediatr Pulmonol
.
2016
;
51
(
5
):
541
548
[PubMed]
26
Ambroggio
L
,
Test
M
,
Metlay
JP
, et al
.
Comparative effectiveness of beta-lactam versus macrolide monotherapy in children with pneumonia diagnosed in the outpatient setting.
Pediatr Infect Dis J
.
2015
;
34
(
8
):
839
842
[PubMed]
27
Williams
DJ
,
Edwards
KM
,
Self
WH
, et al
.
Effectiveness of β-lactam monotherapy vs macrolide combination therapy for children hospitalized with pneumonia.
JAMA Pediatr
.
2017
;
171
(
12
):
1184
1191
[PubMed]
28
Ambroggio
L
,
Taylor
JA
,
Tabb
LP
,
Newschaffer
CJ
,
Evans
AA
,
Shah
SS
.
Comparative effectiveness of empiric β-lactam monotherapy and β-lactam-macrolide combination therapy in children hospitalized with community-acquired pneumonia.
J Pediatr
.
2012
;
161
(
6
):
1097
1103
[PubMed]
29
Florin
TA
,
French
B
,
Zorc
JJ
,
Alpern
ER
,
Shah
SS
.
Variation in emergency department diagnostic testing and disposition outcomes in pneumonia.
Pediatrics
.
2013
;
132
(
2
):
237
244
[PubMed]
30
Nelson
KA
,
Morrow
C
,
Wingerter
SL
,
Bachur
RG
,
Neuman
MI
.
Impact of chest radiography on antibiotic treatment for children with suspected pneumonia.
Pediatr Emerg Care
.
2016
;
32
(
8
):
514
519
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.