OBJECTIVES

We aimed to examine the impact of a quality improvement (QI) collaborative on adherence to specific recommendations within the American Academy of Pediatrics’ Clinical Practice Guideline (CPG) for well-appearing febrile infants aged 8 to 60 days.

METHODS

Concurrent with CPG release in August 2021, we initiated a QI collaborative involving 103 general and children’s hospitals across the United States and Canada. We developed a multifaceted intervention bundle to improve adherence to CPG recommendations for 4 primary measures and 4 secondary measures, while tracking 5 balancing measures. Primary measures focused on guideline recommendations where deimplementation strategies were indicated. We analyzed data using statistical process control (SPC) with baseline and project enrollment from November 2020 to October 2021 and the intervention from November 2021 to October 2022.

RESULTS

Within the final analysis, there were 17 708 infants included. SPC demonstrated improvement across primary and secondary measures. Specifically, the primary measures of appropriately not obtaining cerebrospinal fluid in qualifying infants and appropriately not administering antibiotics had the highest adherence at the end of the collaborative (92.4% and 90.0% respectively). Secondary measures on parent engagement for emergency department discharge of infants 22 to 28 days and oral antibiotics for infants 29 to 60 days with positive urinalyses demonstrated the greatest changes with collaborative-wide improvements of 16.0% and 20.4% respectively. Balancing measures showed no change in missed invasive bacterial infections.

CONCLUSIONS

A QI collaborative with a multifaceted intervention bundle was associated with improvements in adherence to several recommendations from the AAP CPG for febrile infants.

More than 120 000 febrile infants less than 60 days of age are evaluated annually in emergency departments (EDs) within the United States.1  Among well-appearing, febrile infants ≤60 days of age, there is a 10% risk of bacterial infection, including a 2% to 3% risk of invasive bacterial infections (IBI), ie, bacteremia and/or bacterial meningitis.2 4  In evaluating well-appearing febrile infants, clinicians must balance the risks of IBI with the risks of invasive procedures, including harm of failed procedures,5 ,6  false-positive cultures leading to further unnecessary interventions,7 9  and parental anxiety and stress.10 12  Over the past 3 decades, there has been extensive research to guide risk stratification of febrile infants,13 18  though there remains substantial variation in care.19 ,20 

To reduce variation and standardize evidenced-based care, in 2021 the American Academy of Pediatrics (AAP) released the Clinical Practice Guideline (CPG) for the evaluation and management of well-appearing febrile infants.21  The CPG included 7 key action statements (KAS) and 40 recommendations, including risk stratification into 3 age groups (8–21 days, 22–28 days, and 29–60 days), the incorporation of shared decision-making (SDM) into evaluation and management, and the role of inflammatory markers (IMs) to inform safely doing fewer interventions in low-risk infants.

Performing fewer interventions on patients in the setting where doing more is the historical standard requires the individual clinician and hospital to “disinvest” from previous practices. The structured process of stopping previous practices that are harmful, ineffective, or overused is known as deimplementation.22  The AAP’s CPG included multiple recommendations of substantial practice change requiring deimplementation of previous practices, including but not limited to the use of oral antibiotics alone for urinalysis (UA) positive infants >28 days with normal IMs, potential for not obtaining cerebrospinal fluid (CSF) on infants as young as 22 days, and discharging home UA positive infants >28 days, and CSF negative infants ≥22 days.21  This created an opportunity for a real-world, timely deimplementation intervention for the evaluation and management of febrile infants 8 to 60 days of age. Through a multicenter quality improvement (QI) collaborative, we aimed to evaluate the impact of a multifaceted intervention bundle upon adherence to specific recommendations within the AAP CPG for the evaluation and management of well-appearing febrile infants.

We conducted a multicenter QI intervention, REducing Variation In Sepsis Evaluation II (REVISE II), analyzing predetermined outcomes from 103 hospitals using data from November 2020 to October 2022. The AAP’s Institutional Review Board deemed the REVISE II collaborative as exempt. Data are reported according to the Standards for Quality Improvement Reporting Excellence (SQUIRE) 2.0 guidelines.23 

We used a combination of 2 deimplementation frameworks in designing REVISE II. First was the conceptual model proposed by Helfrich et al that describes the psychologic framing for deimplementation: (1) intentional unlearning to consciously change beliefs about an ineffective practice, and (2) substitution to promote changes that rely on quick thinking.24  The second framework we used was derived specifically from large-scale QI projects, combining both facilitation strategies consistent with implementation science (eg, engaging key stakeholders, establishing buy-in, audit and feedback), as well as mitigation strategies to address the psychological barriers to stopping a previously held practice (eg, giving permission to change, normalizing, and reframing).25 

REVISE II was an international QI collaborative through the AAP’s Value in Inpatient Pediatrics (VIP) QI Network. Concordant with the release of the AAP CPG, interested sites could enroll in REVISE II starting in August 2021. Of the 133 sites originally enrolled in REVISE II, 103 completed all 24 months of data collection. Participating sites were diverse in hospital type, institution size and volume of pediatric patients, and geographic area (Supplemental Table 5). Each site formed project teams consisting of various combinations of pediatric emergency medicine clinicians, pediatric inpatient clinicians, nursing staff, and other specialties (eg, pediatric infectious diseases, general emergency medicine, pharmacy). Supported by the collaborative, sites partnered with key stakeholders at their institutions to develop implementation plans and then launch their local QI intervention(s) (Fig 1).

FIGURE 1

Overview timeline for REVISE II interventions. *Midway PDSA consisted of: (1) development of a one-page matrix delineating the AAP CPG recommended evaluation and management by age and inflammatory marker status, (2) additional site-level educational materials, (3) and 1:1 site meetings for sites below the 50th percentile for project measures.

FIGURE 1

Overview timeline for REVISE II interventions. *Midway PDSA consisted of: (1) development of a one-page matrix delineating the AAP CPG recommended evaluation and management by age and inflammatory marker status, (2) additional site-level educational materials, (3) and 1:1 site meetings for sites below the 50th percentile for project measures.

Close modal

We developed an extensive intervention bundle to support site-level implementation (Table 1, select bundle components available in Supplemental Figs 4–6). The project basics included the key driver diagram (Supplemental Fig 4), webinar series on understanding the AAP CPG, REVISE II specific pathways color coded by project measure (Supplemental Fig 5AC), educational resources for site-level education of clinicians, and a project-wide listserv for real-time collaborative learning and problem-solving. Sites had access to real-time run charts of their data as well as collaborative-wide data. Additionally, when viewing data, sites could choose “flags” that identified hospitals similar to theirs in size, beds, etc, to compare local change to both the collaborative-wide change as well as near-peers. Sites were provided with a data query to identify potentially eligible infants using the electronic health record (EHR), smartphrases for SDM discussions, and templates for discharge instructions regarding 24-hour follow-up in both English and Spanish. Sites also were provided educational resources about SDM and example verbal scripts for facilitating discussions with parents.

TABLE 1

Intervention Bundle for REVISE II

Intervention Bundle
Project basics 
Key driver diagram specific to project aims and interventionsa 
Webinar series on the AAP CPG, project implementation, and quarterly aggregate data review 
Crosswalk between REVISE II measures and AAP CPG recommendations 
REVISE II specific pathways for 8–21 d, 22–28 d, 29–60 d color coded by project measurea 
Literature bundle with summarized findings and access to full text articles 
Adaptive educational PowerPoint for use at individual sites on REVISE II and the AAP CPG 
Project-wide listserv and real-time updated FAQ document 
QI education 
Webinar on visualizing change using data, shifting run charts, and change management 
“Keys to Successful QI” support and resources 
Parent engagement and follow-up 
Template for use of SDM for CSF and disposition for infants 22 to 28 d with normal inflammatory markers 
SDM framework, training, and resources 
Verbal and written discharge instruction templates provided in English and Spanish 
Electronic medical record integration 
EPIC query data pull 
Smart phrase integration for documenting follow-up in 1 calendar day from the ED and use of SDM for CSF and disposition for infants 22–28 d with normal inflammatory markers 
Data visualization 
Standardized electronic data aggregator 
Real-time run charts for individual sites and with comparator capabilities by institutional description (eg, size of hospital, volume of pediatric patients, etc) and overall national collaborative 
Optional opportunities 
Intensive data review with an assigned QI specialist (26 enrolled sites) 
Individual provider feedback (23 enrolled sites) 
“Building QI Expertise” webinars (49 Enrolled Sites) 
Midway PDSA (May 2022) 
One-page matrix showing management by age and inflammatory marker status to simplify recommendationsa 
“Understanding the Literature Behind the Guideline” PowerPoint for educational use at individual sites 
Educational video with resources on oral antibiotic use for positive UAs 
Development of educational cases for use with trainees 
1:1 site meetings with sites that had <50th percentile for performance 
Project participation 
ABP and AAP: 50 points MOC part 4 points; 20 MOC part 2 credit, 20 level 1 CME credits 
Royal College of Physicians and Surgeons of Canada: 50 points MOC part 3; 20 points part 1 
American Board of Emergency Medicine: 20 points of CME 
Group and by-line authorship opportunities 
Intervention Bundle
Project basics 
Key driver diagram specific to project aims and interventionsa 
Webinar series on the AAP CPG, project implementation, and quarterly aggregate data review 
Crosswalk between REVISE II measures and AAP CPG recommendations 
REVISE II specific pathways for 8–21 d, 22–28 d, 29–60 d color coded by project measurea 
Literature bundle with summarized findings and access to full text articles 
Adaptive educational PowerPoint for use at individual sites on REVISE II and the AAP CPG 
Project-wide listserv and real-time updated FAQ document 
QI education 
Webinar on visualizing change using data, shifting run charts, and change management 
“Keys to Successful QI” support and resources 
Parent engagement and follow-up 
Template for use of SDM for CSF and disposition for infants 22 to 28 d with normal inflammatory markers 
SDM framework, training, and resources 
Verbal and written discharge instruction templates provided in English and Spanish 
Electronic medical record integration 
EPIC query data pull 
Smart phrase integration for documenting follow-up in 1 calendar day from the ED and use of SDM for CSF and disposition for infants 22–28 d with normal inflammatory markers 
Data visualization 
Standardized electronic data aggregator 
Real-time run charts for individual sites and with comparator capabilities by institutional description (eg, size of hospital, volume of pediatric patients, etc) and overall national collaborative 
Optional opportunities 
Intensive data review with an assigned QI specialist (26 enrolled sites) 
Individual provider feedback (23 enrolled sites) 
“Building QI Expertise” webinars (49 Enrolled Sites) 
Midway PDSA (May 2022) 
One-page matrix showing management by age and inflammatory marker status to simplify recommendationsa 
“Understanding the Literature Behind the Guideline” PowerPoint for educational use at individual sites 
Educational video with resources on oral antibiotic use for positive UAs 
Development of educational cases for use with trainees 
1:1 site meetings with sites that had <50th percentile for performance 
Project participation 
ABP and AAP: 50 points MOC part 4 points; 20 MOC part 2 credit, 20 level 1 CME credits 
Royal College of Physicians and Surgeons of Canada: 50 points MOC part 3; 20 points part 1 
American Board of Emergency Medicine: 20 points of CME 
Group and by-line authorship opportunities 
a

Available with online Supplements. Additional bundle components publicly available at www.aap.org/REVISE2.

We also provided 3 optional opportunities for further support during the project. These included an intensive data review with an assigned QI specialist (26 sites), individualized provider feedback (23 sites), and an in-depth live webinar series on “Building QI Expertise” (49 sites).

In addition to monthly, site-level Plan-Do-Study-Act (PDSA) cycles, we also conducted a collaborative-wide PDSA 6 months into the 12-month intervention period. As part of this PDSA, we focused upon interventions to streamline decision-making and improve uptake of the secondary measures. This PDSA consisted of the development of a 1-page matrix delineating the AAP CPG recommended evaluation and management by age and IM status (Supplemental Fig 6), additional site-level educational materials, and 1:1 site meetings for sites below the 50th percentile for project measures.

Lastly, participants were also eligible for professional credit, including American Board of Pediatrics Maintenance of Certification (MOC) part 4 and part 2 credit and AAP Level 1 Continuing Education Credits (CME), Royal College of Physicians and Surgeons of Canada MOC points, and American Board of Emergency Medicine CME.

The 24-month timeline for the project is shown in Fig 1.

Infants aged 8 to 60 days were eligible if they presented to a participating institution between November 1, 2020 through October 31, 2022 and had a temperature of ≥38°C. Infants were excluded from data collection if they had a gestational age <37 weeks, complex medical conditions, were ill-appearing,26 28  had a diagnosis of bronchiolitis or clear source of bacterial infection (eg, cellulitis), received immunizations or antibiotics within 48 h of presentation,21  or if they were transferred from a referring hospital if laboratory tests were obtained before the transfer. Infants with nonspecific respiratory findings such as tachypnea or congestion were included unless they had a specific diagnosis of bronchiolitis.

Individual sites entered data into a standardized data aggregator on a monthly basis. Eligible patients were identified via EHR query and then manual chart review was used to collect data unable to be extracted through an automated process. Sites were instructed to enter all eligible infants up to 20 infants per month, although sites with more than 20 qualifying infants per month could choose to enter more than 20 infants for local QI purposes. Supplemental Table 6 shows the number and distribution of charts per cycle per site.

We had 4 primary measures, 4 secondary measures, and 5 balancing measures (Table 2). To guide the development and choosing of the measures, we had several guiding principles. The primary measures were selected to reflect CPG recommendations about safely performing fewer interventions, primarily in infants 29 to 60 days of age, thereby requiring a deimplementation approach. The primary measures aligned with CPG recommendations that used language stating that clinicians “need not” or “should” perform certain interventions. For these measures, we set a project goal of 90% of qualifying infants meeting the stated measure within 12 months to reflect the strength of the CPG recommendation. For example, key action statement (KAS) 18b states, “Clinicians need not obtain CSF for analysis and culture if all IMs obtained are normal” for infants 29 to 60 days. This statement informed the primary measure “Appropriate CSF,” which was defined as 90% of infants 29 to 60 days of age with normal IMs and either a negative or positive UA do not have CSF obtained. The other 3 primary measures were similarly derived, including “Appropriate Disposition from the ED” specifying discharge criteria for qualifying infants (derived from KAS 20c), “Appropriate Receipt of Antibiotics” specifying qualifying infants not receiving antibiotics (KAS 19d), and “Appropriate Hospital Discharge” for infants with negative cultures being discharged within 36 hours (KAS 7a, 14a, and 21a).

TABLE 2

Measures for REVISE II

MeasureDefinitionNumeratorDenominator
Primary measures    
Appropriate CSF 90% of infants 29–60 d with normal inflammatory markers (and either a negative UA or a positive UA) DO NOT have CSF obtained Infants 29–60 d with normal inflammatory markers (and either a negative or a positive UA) who DO NOT have CSF obtained All infants 29–60 d with normal inflammatory markers (and either a negative UA or a positive UA) 
Appropriate disposition from the ED 90% of infants 29–60 d with normal inflammatory markers and negative UA discharged from the ED Infants 29–60 d with normal inflammatory markers and a negative UA who are discharged from the ED All infants 29–60 d with normal inflammatory markers and a negative UA 
Appropriate receipt of antibiotics 90% of infants 29–60 d with normal inflammatory markers and negative UA DO NOT receive antibiotics Infants 29–60 d with normal inflammatory markers and negative UA who DO NOT receive antibiotics All infants 29–60 d with normal inflammatory markers and negative UA 
Appropriate discharge from the hospital 90% of infants 8–60 d with negative cultures have appropriate discharge from the hospital within 36 h from the time blood cultures were received by the laboratory Hospitalized infants 8–60 d s with negative cultures discharged within 36 h All hospitalized infants 8–60 d with negative cultures 
Secondary measures    
Appropriate follow-up 75% of infants 22–60 d discharged from the emergency department have documented education with parents about the importance of follow-up within 1 calendar day Infants 22–60 d discharged from the emergency department who have documented education with parents about the importance of follow-up within 1 calendar day All infants 22–60 d discharged from the emergency department 
Appropriate parent engagement - CSF 75% of infants 22–28 d with normal inflammatory markers and negative UA have documented physician-parent discussion about the harms and benefits of having CSF obtained Infants 22–28 d with normal inflammatory markers and negative UA who have documented physician-parent discussion about the harms and benefits of having CSF obtained Infants 22–28 d with normal inflammatory markers and negative UA 
Appropriate parent engagement - Discharge from the ED 75% of infants 22–28 d with normal inflammatory markers, negative UA, and normal CSF have documented physician-parent discussion about the harms and benefits of hospitalization versus discharge from the ED after 1 dose of parenteral antibiotic therapy Infants 22–28 d with normal inflammatory markers, negative UA, and normal CSF who have documented physician-parent discussion about the harms and benefits of hospitalization versus discharge from the ED after 1 dose of parenteral antibiotic therapy Infants 22–28 d with normal inflammatory markers, negative UA, and normal CSF 
Oral antibiotic use for infants 29–60 d with positive UAs 75% of infants 29–60 d olds with a positive UA, negative inflammatory markers, and normal CSF (if obtained) receive oral antibiotics (with or without 1 dose of parenteral antibiotic therapy) Infants 29–60 d olds with a positive UA, normal inflammatory markers, and normal CSF (if obtained) who receive oral antibiotics (with or without 1 dose of parenteral antibiotic therapy) Infants 29–60 d olds with a positive UA, normal inflammatory markers, and normal CSF (if obtained) 
Balancing measures    
Appropriate evaluation: 8–21 d % of infants 8–21 d who have a urinalysis and/or urine culture, blood culture, and CSF culture obtained, and who are hospitalized on parenteral antibiotic therapy Infants 8–21 d who have a urinalysis and/or urine culture, blood culture, and CSF testing including culture obtained, and who are hospitalized on parenteral antibiotic therapy Infants 8–21 d who present to the emergency department or hospital with fever 
Appropriate evaluation: 22–60 d % of infants 22–60 d who have a urinalysis and/or urine culture, blood culture, and inflammatory markers obtained Infants 22–60 d who have a urinalysis and/or urine culture, blood culture, and inflammatory markers obtained Infants 22–60 d who present to the emergency department or hospital with fever 
ED revisit % of infants 22–60 d who did not have CSF obtained or did not receive antibiotic therapy who return to the emergency department within 7 d of discharge Infants 22–60 d who did not have CSF obtained or did not receive antibiotic therapy who return to the emergency department within 7 d of discharge All infants 22–60 d who are evaluated for a fever and who did not have CSF obtained or receive antibiotic therapy 
Readmission % of infants 22–60 d who did not have CSF obtained or receive antibiotic therapy who are readmitted to the hospital within 7 d of discharge Infants 22–60 d who did not have CSF obtained or receive antibiotic therapy who are readmitted to the hospital within 7 d of discharge All infants 22–60 d who are evaluated for a fever and who did not have CSF obtained or receive antibiotic therapy 
Delayed diagnosis of invasive bacterial infections % of infants age 22–60 d discharged from the emergency department or hospital who did not have CSF obtained or receive antibiotic therapy who have a diagnosis of bacteremia and/or bacterial meningitis within 7 d of discharge Infants 22–60 d discharged from the emergency department or hospital who did not have CSF obtained or receive antibiotic therapy who are diagnosed with bacteremia and/or bacterial meningitis within 7 of discharge All infants 22–60 d discharged from the emergency department or hospital after evaluation for fever who did not have CSF obtained or receive antibiotic therapy 
MeasureDefinitionNumeratorDenominator
Primary measures    
Appropriate CSF 90% of infants 29–60 d with normal inflammatory markers (and either a negative UA or a positive UA) DO NOT have CSF obtained Infants 29–60 d with normal inflammatory markers (and either a negative or a positive UA) who DO NOT have CSF obtained All infants 29–60 d with normal inflammatory markers (and either a negative UA or a positive UA) 
Appropriate disposition from the ED 90% of infants 29–60 d with normal inflammatory markers and negative UA discharged from the ED Infants 29–60 d with normal inflammatory markers and a negative UA who are discharged from the ED All infants 29–60 d with normal inflammatory markers and a negative UA 
Appropriate receipt of antibiotics 90% of infants 29–60 d with normal inflammatory markers and negative UA DO NOT receive antibiotics Infants 29–60 d with normal inflammatory markers and negative UA who DO NOT receive antibiotics All infants 29–60 d with normal inflammatory markers and negative UA 
Appropriate discharge from the hospital 90% of infants 8–60 d with negative cultures have appropriate discharge from the hospital within 36 h from the time blood cultures were received by the laboratory Hospitalized infants 8–60 d s with negative cultures discharged within 36 h All hospitalized infants 8–60 d with negative cultures 
Secondary measures    
Appropriate follow-up 75% of infants 22–60 d discharged from the emergency department have documented education with parents about the importance of follow-up within 1 calendar day Infants 22–60 d discharged from the emergency department who have documented education with parents about the importance of follow-up within 1 calendar day All infants 22–60 d discharged from the emergency department 
Appropriate parent engagement - CSF 75% of infants 22–28 d with normal inflammatory markers and negative UA have documented physician-parent discussion about the harms and benefits of having CSF obtained Infants 22–28 d with normal inflammatory markers and negative UA who have documented physician-parent discussion about the harms and benefits of having CSF obtained Infants 22–28 d with normal inflammatory markers and negative UA 
Appropriate parent engagement - Discharge from the ED 75% of infants 22–28 d with normal inflammatory markers, negative UA, and normal CSF have documented physician-parent discussion about the harms and benefits of hospitalization versus discharge from the ED after 1 dose of parenteral antibiotic therapy Infants 22–28 d with normal inflammatory markers, negative UA, and normal CSF who have documented physician-parent discussion about the harms and benefits of hospitalization versus discharge from the ED after 1 dose of parenteral antibiotic therapy Infants 22–28 d with normal inflammatory markers, negative UA, and normal CSF 
Oral antibiotic use for infants 29–60 d with positive UAs 75% of infants 29–60 d olds with a positive UA, negative inflammatory markers, and normal CSF (if obtained) receive oral antibiotics (with or without 1 dose of parenteral antibiotic therapy) Infants 29–60 d olds with a positive UA, normal inflammatory markers, and normal CSF (if obtained) who receive oral antibiotics (with or without 1 dose of parenteral antibiotic therapy) Infants 29–60 d olds with a positive UA, normal inflammatory markers, and normal CSF (if obtained) 
Balancing measures    
Appropriate evaluation: 8–21 d % of infants 8–21 d who have a urinalysis and/or urine culture, blood culture, and CSF culture obtained, and who are hospitalized on parenteral antibiotic therapy Infants 8–21 d who have a urinalysis and/or urine culture, blood culture, and CSF testing including culture obtained, and who are hospitalized on parenteral antibiotic therapy Infants 8–21 d who present to the emergency department or hospital with fever 
Appropriate evaluation: 22–60 d % of infants 22–60 d who have a urinalysis and/or urine culture, blood culture, and inflammatory markers obtained Infants 22–60 d who have a urinalysis and/or urine culture, blood culture, and inflammatory markers obtained Infants 22–60 d who present to the emergency department or hospital with fever 
ED revisit % of infants 22–60 d who did not have CSF obtained or did not receive antibiotic therapy who return to the emergency department within 7 d of discharge Infants 22–60 d who did not have CSF obtained or did not receive antibiotic therapy who return to the emergency department within 7 d of discharge All infants 22–60 d who are evaluated for a fever and who did not have CSF obtained or receive antibiotic therapy 
Readmission % of infants 22–60 d who did not have CSF obtained or receive antibiotic therapy who are readmitted to the hospital within 7 d of discharge Infants 22–60 d who did not have CSF obtained or receive antibiotic therapy who are readmitted to the hospital within 7 d of discharge All infants 22–60 d who are evaluated for a fever and who did not have CSF obtained or receive antibiotic therapy 
Delayed diagnosis of invasive bacterial infections % of infants age 22–60 d discharged from the emergency department or hospital who did not have CSF obtained or receive antibiotic therapy who have a diagnosis of bacteremia and/or bacterial meningitis within 7 d of discharge Infants 22–60 d discharged from the emergency department or hospital who did not have CSF obtained or receive antibiotic therapy who are diagnosed with bacteremia and/or bacterial meningitis within 7 of discharge All infants 22–60 d discharged from the emergency department or hospital after evaluation for fever who did not have CSF obtained or receive antibiotic therapy 

Our secondary measures had 2 driving principles. Either the measure aligned with CPG recommendations with a moderate to strong recommendation but for which data capture would be challenging (ie, documented follow-up within 24 hours after discharge from the ED [KAS 13a, 20c, and 20d] and parent engagement in SDM for infants 22-28d for obtaining not obtaining CSF [KAS 11a] and ED disposition [KAS 13a]), or the recommendation represented a substantial practice change (eg, the use of oral antibiotics in infants 29–60 days with a positive UA [KAS 19c]). We set project goals at 75% for adherence for secondary measures within 12 months to reflect that capturing data and shifting practice will be more challenging for these measures than for the primary measures.

Lastly, our balancing measure aligned with CPG recommendations for obtaining the appropriate evaluation when indicated for infants 8 to 60 days (KAS 1–6, 8–10, 15–17) and to capture important clinical outcomes, including ED revisit, readmission to the hospital, and delayed diagnosis of an IBI, all assessed within 7 days of the index visit. We restricted the last 3 balancing measures to infants 22 to 60 days, as the primary and secondary measures targeted management for these infants specifically.

Outcome data were evaluated using statistical process control p-charts. All measures were written for ease of real-time interpretability by individual sites with increasing percentages along the y-axis representing improvements in change. Special cause was defined by either 8 points above the mean or 2 of 3 points within the outer third of the upper or lower control limits.29  The primary, secondary, and 2 of the balancing measures were evaluated using statistical process control. The remaining 3 balancing measures occurred rarely and were compared between baseline plus enrollment periods and the intervention period using χ-squared tests. We also conducted a subanalysis by hospital typology (free-standing children’s hospitals, nested children’s hospitals, and general hospitals) on measure performance. Only sites that completed all 24 cycles of data entry were included in the final analysis (Supplemental Table 6). All analyses were conducted using R version 4.0.4 (Vienna, Austria).

Across 103 sites that completed 24 months of data collection, 17 708 infants were included in the final analysis, with 6549 (37%) infants within the baseline and REVISE II enrollment periods and 11 159 (63%) within the intervention period (Table 3). Children included within the final sample were primarily 29 to 60 days of age (69%, N = 12 255) and evaluated at freestanding children’s hospitals (60%, N = 10 642).

TABLE 3

Infant and Hospital Demographics of Participants Within the REVISE II QI Collaborative

Total CohortBaseline and REVISE II EnrollmentaInterventionb
N%N%N%χ2P
Overall 17 708 — 6549 37 11 159 63  
Age (days)       <.001 
 8–21 3203 18 1291 20 1912 17  
 22–28 2250 13 838 13 1412 13  
 29–60 12 255 69 4420 67 7835 70  
Sex, female 7669 43 2779 42 4890 44 .075 
Region       <.001 
 Northeast 3151 18 1074 16 2077 19  
 South 7340 41 2883 44 4457 40  
 Midwest 3678 21 1305 20 2373 21  
 West 2630 15 998 15 1632 15  
 Canada 909 289 620  
Hospital type       <.001 
 Freestanding children’s hospital 10 642 60 4058 62 6584 59  
 Nested children’s hospital 5192 29 1872 29 3320 30  
 General hospital 1874 11 619 9.5 1255 11  
Total CohortBaseline and REVISE II EnrollmentaInterventionb
N%N%N%χ2P
Overall 17 708 — 6549 37 11 159 63  
Age (days)       <.001 
 8–21 3203 18 1291 20 1912 17  
 22–28 2250 13 838 13 1412 13  
 29–60 12 255 69 4420 67 7835 70  
Sex, female 7669 43 2779 42 4890 44 .075 
Region       <.001 
 Northeast 3151 18 1074 16 2077 19  
 South 7340 41 2883 44 4457 40  
 Midwest 3678 21 1305 20 2373 21  
 West 2630 15 998 15 1632 15  
 Canada 909 289 620  
Hospital type       <.001 
 Freestanding children’s hospital 10 642 60 4058 62 6584 59  
 Nested children’s hospital 5192 29 1872 29 3320 30  
 General hospital 1874 11 619 9.5 1255 11  
a

Baseline period, November 2020 to July 2021, and REVISE II Enrollment period, August 2021 to October 2021.

b

Intervention period, November 2021 to October 2022.

Of the 4 primary measures, 2 exceeded the project aim of 90% adherence (Fig 2). Appropriate CSF (ie, appropriately not obtaining CSF for infants 29–60 days of age with normal IMs) achieved a mean rate of 92.4% of qualifying infants meeting the measure with 3 points of special cause variation. The first was after CPG release during the REVISE II enrollment period. The second was at launch of the intervention in November 2021. The last time was in September 2022, corresponding to the final aggregate data review. Similarly, appropriate receipt of antibiotics (ie, infants 29–60 days with normal IMs and a negative UA not receiving antibiotics) achieved a mean rate of 90.0% of qualifying infants with special cause noted again after CPG release during the REVISE II enrollment period and in December 2021, 1 month after project launch. The third primary measure, appropriate discharge from the ED, neared 90% with a mean rate of 86.5% with special cause seen at project launch in November 2021 and again in August 2022 following the third aggregate data review. Our fourth primary measure on appropriate discharge from the hospital within 36 hours for infants with negative cultures demonstrated a significant shift at CPG release during the REVISE II enrollment to a mean rate of 34.0%, but there was no further change during the intervention.

FIGURE 2

Statistical process control charts for primary measures for REVISE II. (A) Appropriate CSF: % of infants 29 to 60 days with normal inflammatory markers (and either a negative UA OR a positive UA) that DO NOT have CSF obtained. (B) Appropriate disposition from the ED: % of infants 29 to 60 days with normal inflammatory markers and negative UA that are discharged from the ED. (C) Appropriate receipt of antibiotics: % of infants 29 to 60 days with normal inflammatory markers and negative UA that do not receive antibiotics. (D) Appropriate discharge from the hospital: % of infants 8 to 60 days with negative cultures that have appropriate discharge from the hospital within 36 hours from the time blood cultures were received by the laboratory.

FIGURE 2

Statistical process control charts for primary measures for REVISE II. (A) Appropriate CSF: % of infants 29 to 60 days with normal inflammatory markers (and either a negative UA OR a positive UA) that DO NOT have CSF obtained. (B) Appropriate disposition from the ED: % of infants 29 to 60 days with normal inflammatory markers and negative UA that are discharged from the ED. (C) Appropriate receipt of antibiotics: % of infants 29 to 60 days with normal inflammatory markers and negative UA that do not receive antibiotics. (D) Appropriate discharge from the hospital: % of infants 8 to 60 days with negative cultures that have appropriate discharge from the hospital within 36 hours from the time blood cultures were received by the laboratory.

Close modal

Of the secondary measures, none reached the project goal of 75%. However, all demonstrated improvement in adherence within the intervention period (Fig 3). In addition, the measures of appropriate parent engagement around ED discharge (ie, infants 22–28 days with normal IMs, negative UA, and normal CSF have a documented physician-parent discussion about ED disposition) improved from a mean of 1.9% in the baseline period to 17.9% in the intervention with special cause variation occurring in December 2021. Similarly, the measure on oral antibiotic use for infants 29 to 60 days with positive UAs improved from a mean of 13.0% in the baseline period to 33.4% in the intervention period, shifting at project launch in November 2021.

FIGURE 3

Statistical process control charts for secondary measures for REVISE II. (A) Appropriate follow-up: % of infants 22 to 60 days discharged from the emergency department that have documented education with parents about the importance of follow-up within 1 calendar day. (B) Appropriate parent engagement – CSF: % of infants 22 to 28 days with normal inflammatory markers and negative UA that have documented physician-parent discussion about the harms and benefits of having CSF obtained. (C) Appropriate parent engagement – discharge from the ED: % of infants 22 to 28 days with normal inflammatory markers, negative UA, and normal CSF that have documented physician-parent discussion about the harms and benefits of hospitalization versus discharge from the ED after 1 dose of parenteral antibiotic therapy. (D) Oral antibiotic use for infants 29 to 60 days with positive UAs: % of infants 29 to 60 days with a positive UA, negative inflammatory markers, and normal CSF (if obtained) that receive oral antibiotics (with or without 1 dose of parenteral antibiotic therapy).

FIGURE 3

Statistical process control charts for secondary measures for REVISE II. (A) Appropriate follow-up: % of infants 22 to 60 days discharged from the emergency department that have documented education with parents about the importance of follow-up within 1 calendar day. (B) Appropriate parent engagement – CSF: % of infants 22 to 28 days with normal inflammatory markers and negative UA that have documented physician-parent discussion about the harms and benefits of having CSF obtained. (C) Appropriate parent engagement – discharge from the ED: % of infants 22 to 28 days with normal inflammatory markers, negative UA, and normal CSF that have documented physician-parent discussion about the harms and benefits of hospitalization versus discharge from the ED after 1 dose of parenteral antibiotic therapy. (D) Oral antibiotic use for infants 29 to 60 days with positive UAs: % of infants 29 to 60 days with a positive UA, negative inflammatory markers, and normal CSF (if obtained) that receive oral antibiotics (with or without 1 dose of parenteral antibiotic therapy).

Close modal

For the balancing measures on appropriate evaluation for infants 8 to 21 days and 22 to 60 days of age, both demonstrated special cause with improvement in adherence within the intervention period (Supplemental Fig 7).

Overall, for infants 22 to 60 days 951 (10.1%) had 7-day ED revisit, 344 (3.6%) had a readmission, and 31 (0.3%) infants had a delayed diagnosis of an IBI (Table 4). There were no differences in the baseline to the intervention periods for delayed diagnosis of IBI (0.4% vs 0.3%, P = .74) or 7-day ED revisits (10.9% vs 9.7%, P = .08). There was a slight decrease in readmissions within the intervention period (4.4% vs 3.3%, P = .007).

TABLE 4

ED Revisits, Readmissions, and Missed Invasive Bacterial Infections During REVISE II for Infants 22 to 60 d

Balancing MeasuresBaseline and REVISE II EnrollmentaN (%)InterventionbN (%)P
Total qualifying infants 22–60 d 2937 6510 — 
ED revisit within 7 dc 320 (10.9) 631 (9.7) .08 
Readmission within 7 dd 130 (4.4) 214 (3.3) .007 
Delayed diagnosis of invasive bacterial infectionse 11 (0.4) 20 (0.3) .74 
Balancing MeasuresBaseline and REVISE II EnrollmentaN (%)InterventionbN (%)P
Total qualifying infants 22–60 d 2937 6510 — 
ED revisit within 7 dc 320 (10.9) 631 (9.7) .08 
Readmission within 7 dd 130 (4.4) 214 (3.3) .007 
Delayed diagnosis of invasive bacterial infectionse 11 (0.4) 20 (0.3) .74 
a

Baseline period, November 2020 to July 2021, and REVISE II Enrollment period, August 2021 to October 2021.

b

Intervention period, November 2021 to October 2022.

c

ED Revisit: % of infants 22 to 60 d who did not have CSF obtained or did not receive antibiotic therapy who return to the emergency department within 7 d of discharge.

d

Readmission: % of infants 22 to 60 d who did not have CSF obtained or receive antibiotic therapy who are readmitted to the hospital within 7 d of discharge.

e

Delayed diagnosis of invasive bacterial infections: % of infants age 22 to 60 d discharged from the emergency department or hospital who did not have CSF obtained or receive antibiotic therapy who have a diagnosis of bacteremia and/or bacterial meningitis within 7 d of discharge.

Lastly, when assessing measure performance by hospital typology, general hospitals performed as well as or better than children’s hospitals in 3 of 4 primary measures and similarly across secondary and baseline measures (Supplemental Table 7).

Through a multifaceted intervention, the REVISE II QI collaborative was able to make significant improvements in adherence to several recommendations from the AAP CPG for well-appearing febrile infants ages 8 to 60 days across 103 participating sites. Uptake of key guideline recommendations around appropriately not obtaining CSF, not administering antibiotics, discharging from the ED, and hospital discharge within 36 hours improved through project participation. Furthermore, guideline recommendations for the administration of oral antibiotics to infants 29 to 60 days of age with positive UAs and normal IMs and engaging parents of infants 22 to 28 days of age with normal IMs around disposition demonstrated substantial improvement with implementation.

Several of the biggest successes within REVISE II were in areas that specifically required a deimplementation approach. Building off our deimplementation frameworks, we used strategies to support and promote safely doing fewer interventions. Within the QI collaborative, the process of normalizing change occurred through group norming and sharing of collaborative-wide data. The peer-pressure of other institutions implementing the guideline along with the availability of real-time site-level, near-peer, and collaborative-level data may have helped to normalize the implementation of the measures. One of the strengths in participating within a national collaborative is the ability to leverage the credibility of the AAP’s CPG to normalize new approaches across more than 100 participating sites. The process of normalizing the change also gives permission for the individual clinician to acknowledge that a previous practice (eg, to obtain CSF on all infants with positive UAs) used to be standard of care, but now the evidence has changed and so can they. We also employed strategies of uncoupling laboratory results from automatic admission criteria for infants ≥22 days of age and from automatically obtaining CSF in infants >28 days. We reframed the stopping of interventions (eg, obtaining CSF on all infants ≥22 days) as action toward a new endpoint (eg, SDM).

Launching REVISE II enrollment in synchrony with the publication of the CPG allowed the project to leverage the momentum of the publication. Although the trickle-down effect of published literature into practice traditionally takes years, participation within a national collaborative has30  demonstrated improved uptake of published guidelines in a more rapid timeline.31  Many of our measures had special cause variation at CPG release, consistent with the initial excitement of both release and enrollment participation. However, we demonstrated both increasing adherence and sustained improvement throughout the intervention period for all primary and secondary measures. This is particularly relevant as a third of our collaborative was comprised of general hospitals, where uptake of pediatric interventions is often challenging in the setting of low pediatric volumes and adult-focused prioritizations.32  Yet, general hospitals participating in our collaborative performed at similar levels or better for our primary and secondary measures as children’s hospitals.

The change in practice of parenteral antibiotics for infants 29 to 60 days of age with positive UAs to oral antibiotics was one of the largest improvements of any of our measures. Historically, over 90% of febrile infants 29 to 60 days of age with urinary tract infections are hospitalized on parenteral antibiotics during their index ED visit.33  Although we did not achieve our goal of 75% of these infants receiving oral antibiotics during the intervention period, the improvement demonstrates clinician willingness within the collaborative to deimplement old practices based upon evolving literature.

For our other secondary measures, challenges related to efficiency and throughput in the ED may have impacted uptake. Although we provided specific educational resources, EHR smart phrases and discharge instructions, and example scripts to facilitate SDM discussions with parents, we had more limited improvement in uptake of these recommendations. These measures required specific documentation in the EHR and may have led to clinicians engaging in SDM but not spending the additional time to document within the EHR, as time constraints are a known factor to limit SDM.34  Future quantitative and qualitative investigation is needed to learn how to best implement SDM with parents of febrile infants in the ED, including how best to integrate decision tools to facilitate this process.35 

Our work has several limitations. First, one of the external challenges of REVISE II was the undertaking of a complex implementation project that occurred entirely during the coronavirus disease 2019 pandemic. Although the pandemic had significant impact upon bed capacity, nursing resources, and clinician burnout,36 39  REVISE II was still effective in advancing uptake of targeted measures. Second, as this project was QI, each individual site decided how and what to focus on within the project. Although the implementation bundle and support were provided, the sites were able to tailor their intervention to their own needs. This led to variability in uptake and improvements in the measures. Third, because of complexity of the CPG and multiple EHRs across sites, we did not develop standardized order sets despite the hard wiring of decision-making being associated with improved implementation. We did, however, develop smartphrases and discharge instructions to provide informational technology support across the collaborative. Lastly, there was no minimum for numbers of patients who had to be entered each month. This likely impacted the data entry during the baseline period in particular. For example, almost double the number of infants were entered in the intervention versus the baseline period. Sites had 6 months from time of enrollment to time to complete baseline period data entry, which may have led to sites entering fewer charts per month simply to complete the baseline data on time.

A multisite international QI intervention was associated with rapid improvement in adherence to several recommendations from the AAP CPG for well-appearing febrile infants. The most notable improvements were in deimplementation measures focused on reducing CSF for infants 29 to 60 days of age with normal IMs and appropriately not administering antibiotics for qualifying infants 29 to 60 days. Future QI work can focus upon improved implementation of SDM.

The collaborators in the AAP REVISE II QI Collaborative are acknowledged as group authors for this study. See Appendix for list of all group authors.

Dr McDaniel conceptualized and designed the study, supervised the collaborative, drafted the initial manuscript, and interpreted the data; Dr Kerns contributed to the design of the study, conducted the data analyses, and interpreted the data; Mr Flores helped to conduct the data analyses and interpreted the data; Ms Jennings and Ms Magee contributed to the design of the study and coordinated data collection for the collaborative; Dr Biondi contributed to the design of the study; Dr Aronson conceptualized and designed the study, supervised the collaborative, contributed to drafting of the initial manuscript, and interpreted the data; and all authors critically reviewed and revised the manuscript, and approved the final manuscript as submitted.

FUNDING: This project was supported by the American Academy of Pediatrics.

POTENTIAL CONFLICTS OF INTEREST DISCLOSURES: The authors have no conflicts of interest relevant to this article to disclose.

AAP

American Academy of Pediatrics

CPG

Clinical Practice Guideline

CSF

cerebral spinal fluid

ED

emergency department

EHR

electronic health record

IBI

invasive bacterial infection

IM

inflammatory markers

KAS

key action statement

REVISE II

Reducing Excessive Variability in Infant Sepsis Evaluation II

SDM

shared decision-making

UA

urinalysis

1
Ramgopal
S
,
Aronson
PL
,
Marin
JR
.
United States’ emergency department visits for fever by young children 2007-2017
.
West J Emerg Med
.
2020
;
21
(
6
):
146
151
2
Greenhow
TL
,
Hung
YY
,
Herz
AM
,
Losada
E
,
Pantell
RH
.
The changing epidemiology of serious bacterial infections in young infants
.
Pediatr Infect Dis J
.
2014
;
33
(
6
):
595
599
3
Biondi
E
,
Evans
R
,
Mischler
M
, et al
.
Epidemiology of bacteremia in febrile infants in the United States
.
Pediatrics
.
2013
;
132
(
6
):
990
996
4
Powell
EC
,
Mahajan
PV
,
Roosevelt
G
, et al
;
Febrile Infant Working Group of the Pediatric Emergency Care Applied Research Network (PECARN)
.
Epidemiology of bacteremia in febrile infants aged 60 days and younger
.
Ann Emerg Med
.
2018
;
71
(
2
):
211
216
5
Neal
JT
,
Kaplan
SL
,
Woodford
AL
,
Desai
K
,
Zorc
JJ
,
Chen
AE
.
The effect of bedside ultrasonographic skin marking on infant lumbar puncture success: a randomized controlled trial
.
Ann Emerg Med
.
2017
;
69
(
5
):
610
619.e1
6
Glatstein
MM
,
Zucker-Toledano
M
,
Arik
A
,
Scolnik
D
,
Oren
A
,
Reif
S
.
Incidence of traumatic lumbar puncture: experience of a large, tertiary care pediatric hospital
.
Clin Pediatr (Phila)
.
2011
;
50
(
11
):
1005
1009
7
Hall
RT
,
Domenico
HJ
,
Self
WH
,
Hain
PD
.
Reducing the blood culture contamination rate in a pediatric emergency department and subsequent cost savings
.
Pediatrics
.
2013
;
131
(
1
):
e292
e297
8
Stalnikowicz
R
,
Block
C
.
The yield of blood cultures in a department of emergency medicine
.
Eur J Emerg Med
.
2001
;
8
(
2
):
93
97
9
Sard
B
,
Bailey
MC
,
Vinci
R
.
An analysis of pediatric blood cultures in the postpneumococcal conjugate vaccine era in a community hospital emergency department
.
Pediatr Emerg Care
.
2006
;
22
(
5
):
295
300
10
De
S
,
Tong
A
,
Isaacs
D
,
Craig
JC
.
Parental perspectives on evaluation and management of fever in young infants: an interview study
.
Arch Dis Child
.
2014
;
99
(
8
):
717
723
11
Aronson
PL
,
Fleischer
E
,
Schaeffer
P
,
Fraenkel
L
,
Politi
MC
,
White
MA
.
Development of a parent-reported outcome measure for febrile infants ≤60 days old
.
Pediatr Emerg Care
.
2022
;
38
(
2
):
e821
e827
12
Paxton
RD
,
Byington
CL
.
An examination of the unintended consequences of the rule-out sepsis evaluation: a parental perspective
.
Clin Pediatr (Phila)
.
2001
;
40
(
2
):
71
77
13
Jaskiewicz
JA
,
McCarthy
CA
,
Richardson
AC
, et al
;
Febrile Infant Collaborative Study Group
.
Febrile infants at low risk for serious bacterial infection--an appraisal of the Rochester criteria and implications for management
.
Pediatrics
.
1994
;
94
(
3
):
390
396
14
Lyons
TW
,
Garro
AC
,
Cruz
AT
, et al
;
Herpes Simplex Virus Study Group Of The Pediatric Emergency Medicine Collaborative Research Committee (PEM CRC)
.
Performance of the modified Boston and Philadelphia criteria for invasive bacterial infections
.
Pediatrics
.
2020
;
145
(
4
):
e20193538
15
Gomez
B
,
Mintegi
S
,
Bressan
S
,
Da Dalt
L
,
Gervaix
A
,
Lacroix
L
;
European Group for Validation of the Step-by-Step Approach
.
Validation of the “step-by-step” approach in the management of young febrile infants
.
Pediatrics
.
2016
;
138
(
2
):
e20154381
16
Aronson
PL
,
Shabanova
V
,
Shapiro
ED
, et al
;
Febrile Young Infant Research Collaborative
.
A prediction model to identify febrile infants ≤60 days at low risk of invasive bacterial infection
.
Pediatrics
.
2019
;
144
(
1
):
e20183604
17
Nigrovic
LE
,
Kuppermann
N
,
Macias
CG
, et al
;
Pediatric Emergency Medicine Collaborative Research Committee of the American Academy of Pediatrics
.
Clinical prediction rule for identifying children with cerebrospinal fluid pleocytosis at very low risk of bacterial meningitis
.
JAMA
.
2007
;
297
(
1
):
52
60
18
Kuppermann
N
,
Dayan
PS
,
Levine
DA
, et al
;
Febrile Infant Working Group of the Pediatric Emergency Care Applied Research Network (PECARN)
.
A clinical prediction rule to identify febrile infants 60 days and younger at low risk for serious bacterial infections
.
JAMA Pediatr
.
2019
;
173
(
4
):
342
351
19
Stephens
JR
,
Hall
M
,
Cotter
JM
, et al
.
Trends and variation in length of stay among hospitalized febrile infants ≤60 days old
.
Hosp Pediatr
.
2021
;
11
(
9
):
915
926
20
Aronson
PL
,
Thurm
C
,
Alpern
ER
, et al
;
Febrile Young Infant Research Collaborative
.
Variation in care of the febrile young infant <90 days in US pediatric emergency departments
.
Pediatrics
.
2014
;
134
(
4
):
667
677
21
Pantell
RH
,
Roberts
KB
,
Adams
WG
, et al
;
Subcommittee on Febrile Infants
.
Evaluation and management of well-appearing febrile infants 8 to 60 days old
.
Pediatrics
.
2021
;
148
(
2
):
e2021052228
22
Wolf
ER
,
Krist
AH
,
Schroeder
AR
.
Deimplementation in pediatrics: past, present, and future
.
JAMA Pediatr
.
2021
;
175
(
3
):
230
232
23
Ogrinc
G
,
Davies
L
,
Goodman
D
,
Batalden
P
,
Davidoff
F
,
Stevens
D
.
SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process
.
BMJ Qual Saf
.
2016
;
25
(
12
):
986
992
24
Helfrich
CD
,
Rose
AJ
,
Hartmann
CW
, et al
.
How the dual process model of human cognition can inform efforts to de-implement ineffective and harmful clinical practices: a preliminary model of unlearning and substitution
.
J Eval Clin Pract
.
2018
;
24
(
1
):
198
205
25
McDaniel
CE
,
House
SA
,
Ralston
SL
.
Behavioral and psychological aspects of the physician experience with deimplementation
.
Pediatr Qual Saf
.
2022
;
7
(
1
):
e524
26
Baskin
MN
,
Fleisher
GR
,
O’Rourke
EJ
.
Identifying febrile infants at risk for a serious bacterial infection
.
J Pediatr
.
1993
;
123
(
3
):
489
490
27
Aronson
PL
,
Wang
ME
,
Nigrovic
LE
, et al
;
Febrile Young Infant Research Collaborative
.
Time to pathogen detection for non-ill versus ill-appearing infants ≤60 days old with bacteremia and meningitis
.
Hosp Pediatr
.
2018
;
8
(
7
):
379
384
28
Aronson
PL
,
McCulloh
RJ
,
Tieder
JS
, et al
;
Febrile Young Infant Research Collaborative
.
Application of the Rochester criteria to identify febrile infants with bacteremia and meningitis
.
Pediatr Emerg Care
.
2019
;
35
(
1
):
22
27
29
Benneyan
JC
,
Lloyd
RC
,
Plsek
PE
.
Statistical process control as a tool for research and healthcare improvement
.
Qual Saf Health Care
.
2003
;
12
(
6
):
458
464
30
Morris
ZS
,
Wooding
S
,
Grant
J
The answer is 17 years, what is the question: understanding time lags in translational research.
J R Soc Med
.
2011
;
104
(
12
):
510
520
31
Rooholamini
SN
,
Jennings
B
,
Zhou
C
, et al
.
Effect of a quality improvement bundle to standardize the use of intravenous fluids for hospitalized pediatric patients: a stepped-wedge, cluster randomized clinical trial
.
JAMA Pediatr
.
2022
;
176
(
1
):
26
33
32
McDaniel
CE
,
Haaland
W
,
Parlaman
J
,
Zhou
C
,
Desai
AD
.
A multisite intervention for pediatric community-acquired pneumonia in community settings
.
Acad Emerg Med
.
2018
;
25
(
8
):
870
879
33
Schnadower
D
,
Kuppermann
N
,
Macias
CG
, et al
;
American Academy of Pediatrics Pediatric Emergency Medicine Collaborative Research Committee UTI Study Group
.
Outpatient management of young febrile infants with urinary tract infections
.
Pediatr Emerg Care
.
2014
;
30
(
9
):
591
597
34
Aronson
PL
,
Schaeffer
P
,
Fraenkel
L
,
Shapiro
ED
,
Niccolai
LM
.
Physicians’ and nurses’ perspectives on the decision to perform lumbar punctures on febrile infants ≤8 weeks old
.
Hosp Pediatr
.
2019
;
9
(
6
):
405
414
35
Aronson
PL
,
Politi
MC
,
Schaeffer
P
, et al
.
Development of an APP to facilitate communication and shared decision-making with parents of febrile infants ≤ 60 days old
.
Acad Emerg Med
.
2021
;
28
(
1
):
46
59
36
Gualano
MR
,
Sinigaglia
T
,
Lo Moro
G
, et al
.
The burden of burnout among healthcare professionals of intensive care units and emergency departments during the COVID-19 pandemic: a systematic review
.
Int J Environ Res Public Health
.
2021
;
18
(
15
):
8172
37
Turale
S
,
Nantsupawat
A
.
Clinician mental health, nursing shortages and the COVID-19 pandemic: Crises within crises
.
Int Nurs Rev
.
2021
;
68
(
1
):
12
14
38
Giannini
A
.
Who gets the last bed? Ethics criteria for scarce resource allocation in the era of COVID-19
.
Minerva Anestesiol
.
2021
;
87
(
3
):
267
271
39
Connor
JA
,
Hurtig
M
,
Ormsby
JA
,
Hickey
PA
.
Expanding hospital capacity during the COVID-19 pandemic: the family voice matters
.
Pediatr Qual Saf
.
2021
;
6
(
3
):
e411

Supplementary data