BACKGROUND:

Clinicians commonly obtain endotracheal aspirate cultures (EACs) in the evaluation of suspected ventilator-associated infections. However, bacterial growth in EACs does not distinguish bacterial colonization from infection and may lead to overtreatment with antibiotics. We describe the development and impact of a clinical decision support algorithm to standardize the use of EACs from ventilated PICU patients.

METHODS:

We monitored EAC use using a statistical process control chart. We compared the rate of EACs using Poisson regression and a quasi-experimental interrupted time series model and assessed clinical outcomes 1 year before and after introduction of the algorithm.

RESULTS:

In the preintervention year, there were 557 EACs over 5092 ventilator days; after introduction of the algorithm, there were 234 EACs over 3654 ventilator days (an incident rate of 10.9 vs 6.5 per 100 ventilator days). There was a 41% decrease in the monthly rate of EACs (incidence rate ratio [IRR]: 0.59; 95% confidence interval [CI] 0.51–0.67; P < .001). The interrupted time series model revealed a preexisting 2% decline in the monthly culture rate (IRR: 0.98; 95% CI 0.97–1.0; P = .01), immediate 44% drop (IRR: 0.56; 95% CI 0.45–0.70; P = .02), and stable rate in the postintervention year (IRR: 1.03; 95% CI 0.99–1.07; P = .09). In-hospital mortality, hospital length of stay, 7-day readmissions, and All Patients Refined Diagnosis Related Group severity and mortality scores were stable. The estimated direct cost savings was $26 000 per year. CONCLUSIONS: A clinical decision support algorithm standardizing EAC obtainment from ventilated PICU patients was associated with a sustained decline in the rate of EACs, without changes in mortality, readmissions, or length of stay. What’s Known on This Subject: Endotracheal aspirate cultures (EACs) are commonly obtained in evaluation of suspected ventilator-associated infection. Bacterial growth in EACs does not distinguish bacterial colonization from infection and may lead to overtreatment with antibiotics. Diagnostic stewardship has been used to improve the use of other microbiology testing. What This Study Adds: Diagnostic stewardship of EACs by using a clinical decision support tool led to a reduction in EAC use and cost savings in a PICU and was not associated with changes in mortality, length of stay, or readmissions. Clinicians commonly obtain endotracheal aspirate cultures (EACs) in the evaluation of suspected ventilator-associated infections (VAIs),1,2 a common hospital-acquired infection.3,4 Although EACs cannot distinguish between bacteria colonizing the respiratory tract from bacteria causing infection,57 positive EAC results prompt clinicians to treat with antibiotics.1,2,8 Therefore, overtesting may contribute to excessive treatment with antibiotics because treatment of VAI accounts for up to one-half of the antibiotic use in the PICU.9,10 There is not a gold standard definition of VAI, and the diagnosis is challenging. VAIs encompass either ventilator-associated pneumonia (VAP) or ventilator-associated tracheobronchitis because these entities are, often, difficult to distinguish and are treated interchangeably.1113 The Centers for Disease Control and Prevention includes respiratory secretion Gram-stain and cultures as a component of the surveillance definition of VAP.14 The Infectious Diseases Society of America and American Thoracic Society guidelines for management of adults with VAP support obtainment of EACs.15 However, there are not national consensus recommendations regarding specific clinical indications for which to obtain EACs in either children or adult patients. A survey of clinicians revealed that fever was the most frequent symptom triggering EACs collection and more than one-half of EACs were obtained for isolated clinical changes and were of little to no utility for patient management.16 In studies of ventilated pediatric patients, researchers suggest that antibiotic treatment of VAI is driven by the presence or absence of bacterial growth in EACs,1,2,8 yet antibiotic treatment was not associated with improvement in clinical outcomes.1 Diagnostic stewardship is the promotion of judicious use of diagnostic tests by either modifying the process of ordering the test or how results are reported to improve the accuracy of diagnosis and treatment.17 Diagnostic stewardship approaches have improved the use of Clostridium difficile testing,18,19 urine cultures,20,21 and blood cultures.22,23 Our objective was to describe the development and implementation of a clinical decision support tool to standardize the use of EACs from mechanically ventilated patients in the PICU and assess the impact of this intervention on EAC use and balancing measures. This study was conducted in the PICU of the Johns Hopkins Children’s Center in Baltimore, Maryland, a quaternary care academic PICU with 36 beds caring for medical, surgical, and cardiac patients from birth to 24 years of age, with 2000 yearly admissions. We performed an analysis of a quality improvement (QI) program to improve EAC use to measure associated outcomes and safety. In this study, we specifically reviewed all patients admitted to the unit who were mechanically ventilated via endotracheal tube or tracheostomy 1 year before implementation of the algorithm (preintervention: April 1, 2017, to March 31, 2018) and 1 year after (postintervention: April 1, 2018, to March 31, 2019). A time line of the QI initiative is summarized in Table 1. In 2016, PICU and infectious diseases physicians expressed concern for the frequently repeated and reflexive use of EACs in the Johns Hopkins Children’s Center PICU. A multidisciplinary QI team reviewed baseline EAC use and conducted clinician surveys, which supported that there was a low threshold to obtain EACs.16 We held focus groups with PICU nurse practitioners, physicians, and respiratory therapists, discussing EAC practices and potential barriers to change. The formative work supported that there were drivers of EAC use including (1) reflexive testing in response to isolated clinical changes (eg, fever) and (2) significance attributed to reported changes in respiratory secretions. With the support of PICU leadership, the QI team used a translating-evidence-into-practice model to improve EAC use.24 The local drivers were combined with existing literature, local data, and workgroup consensus to inform algorithm development and implementation (detailed in Table 2). A workgroup, including critical care attending physicians, fellows, nurse practitioners, and infectious disease physicians, drafted an algorithm, shown in Fig 1, to guide clinician decision-making around EACs. The team solicited input from PICU respiratory therapists and nurses and clinical microbiology laboratory faculty. The primary objective of the algorithm was to reduce reflexive culturing from mechanically ventilated patients without signs or symptoms of a respiratory infection by prompting clinicians to consider specific changes supporting a respiratory infection and timing since last EAC. The algorithm page included guidance for specimen collection to avoid the use of saline lavage because this practice can dilute specimens and is not recommended.25 TABLE 1 Time Line of the Endotracheal Culture QI Initiative Event Winter 2016 Initial concern for overuse of EACs Fall 2017 Review of 1 year of baseline EAC use November 2017 to January 2018 Surveys of PICU staff after EACs obtained January to March 2018 Focus groups with PICU staff and development of clinical decision support algorithm March 26, 2018 Decision support algorithm approved by unit director March 26–28, 2018 E-mail announcement and algorithm sent to all PICU staff; paper copies placed in work rooms; algorithm posted to intranet Web site accessible to staff April 1, 2018 Initiation date to start using algorithm April to May 2018 Paper checklist of algorithm signed by attending physicians and fellows when EACs obtained; Meetings with clinicians to discuss initiative and any safety concerns or observations; QI team walk rounds through the PICU soliciting feedback May 2018 to June 2019 Quarterly discussion at PICU safety meetings February 2020 Electronic monitoring dashboard became available Event Winter 2016 Initial concern for overuse of EACs Fall 2017 Review of 1 year of baseline EAC use November 2017 to January 2018 Surveys of PICU staff after EACs obtained January to March 2018 Focus groups with PICU staff and development of clinical decision support algorithm March 26, 2018 Decision support algorithm approved by unit director March 26–28, 2018 E-mail announcement and algorithm sent to all PICU staff; paper copies placed in work rooms; algorithm posted to intranet Web site accessible to staff April 1, 2018 Initiation date to start using algorithm April to May 2018 Paper checklist of algorithm signed by attending physicians and fellows when EACs obtained; Meetings with clinicians to discuss initiative and any safety concerns or observations; QI team walk rounds through the PICU soliciting feedback May 2018 to June 2019 Quarterly discussion at PICU safety meetings February 2020 Electronic monitoring dashboard became available TABLE 2 Local Drivers, Supporting Literature, and Workgroup Consensus That Informed Development and Implementation of an Algorithm to Reduce EAC Use Primary DriversSecondary DriversLiterature, Data, or Workgroup ConsensusAction Taken Reflexive testing for clinical changes in ventilated patient Absence of guidance when to obtain EAC Common triggers for EACs were fever, ventilator changes, secretions, WBCs, and CRP.16 The workgroup consensus was that infections would be associated with abnormal temperatures and WBC but did not specify cutoff values consistent with recent evidence.35 Addressed common triggers in the algorithm Misconception of EACs’ ability to distinguish colonization from infection Airways are nonsterile and become colonized with bacteria. EAC with bacterial growth is not specific to infection.57 Paired algorithm implementation with education Ordering clinicians are unaware of previous EAC results A review of EAC results from 2017 revealed 75% of EACs repeated within 3 d did not have new bacterial species (unpublished observations). Microbiology laboratory does not repeat antibiotic susceptibilities if repeated within 3 d. Incorporated step necessitating review of existence of previous EACs and avoiding cultures repeated within 3 d Competing demands reducing time to consider utility of EAC Ensured algorithm is brief and easily accessible by print or online Variable experience of members of care team with complex communication pattern of clinical information Involved input from all groups of health care workers in guideline development and ensured the algorithm was accessible to everyone Significance attributed to a reported change in respiratory secretions Variability in interpretation between providers of secretion description Increased secretion quantity supported as a common feature of patients with respiratory infections.36 Sputum color and purulence has not correlated with bacterial respiratory infections.3741 Algorithm included quantity of secretions instead of sputum color or thickness descriptions EACs ordered from patients without sufficient secretions, and RTs used saline lavage to obtain cultures. Saline lavage is not a recommended practice.25 Discouraged saline lavage and empowered RT to inform ordering clinician if insufficient quantity of secretions to send for culture Primary DriversSecondary DriversLiterature, Data, or Workgroup ConsensusAction Taken Reflexive testing for clinical changes in ventilated patient Absence of guidance when to obtain EAC Common triggers for EACs were fever, ventilator changes, secretions, WBCs, and CRP.16 The workgroup consensus was that infections would be associated with abnormal temperatures and WBC but did not specify cutoff values consistent with recent evidence.35 Addressed common triggers in the algorithm Misconception of EACs’ ability to distinguish colonization from infection Airways are nonsterile and become colonized with bacteria. EAC with bacterial growth is not specific to infection.57 Paired algorithm implementation with education Ordering clinicians are unaware of previous EAC results A review of EAC results from 2017 revealed 75% of EACs repeated within 3 d did not have new bacterial species (unpublished observations). Microbiology laboratory does not repeat antibiotic susceptibilities if repeated within 3 d. Incorporated step necessitating review of existence of previous EACs and avoiding cultures repeated within 3 d Competing demands reducing time to consider utility of EAC Ensured algorithm is brief and easily accessible by print or online Variable experience of members of care team with complex communication pattern of clinical information Involved input from all groups of health care workers in guideline development and ensured the algorithm was accessible to everyone Significance attributed to a reported change in respiratory secretions Variability in interpretation between providers of secretion description Increased secretion quantity supported as a common feature of patients with respiratory infections.36 Sputum color and purulence has not correlated with bacterial respiratory infections.3741 Algorithm included quantity of secretions instead of sputum color or thickness descriptions EACs ordered from patients without sufficient secretions, and RTs used saline lavage to obtain cultures. Saline lavage is not a recommended practice.25 Discouraged saline lavage and empowered RT to inform ordering clinician if insufficient quantity of secretions to send for culture CRP, C-reactive protein; ETA, endotracheal aspirate; RT, respiratory therapist; WBC, white blood cell. FIGURE 1 Algorithm for obtaining endotracheal cultures from mechanically ventilated patients. CRP, C-reactive protein; ETA, endotracheal aspirate. FIGURE 1 Algorithm for obtaining endotracheal cultures from mechanically ventilated patients. CRP, C-reactive protein; ETA, endotracheal aspirate. Close modal The algorithm was disseminated to all PICU staff preceding initiation on April 1, 2018. For 2 months, fellows or attending physicians were asked to sign off on a paper checklist of the algorithm when EACs were obtained. The QI team conducted in-person walk rounds to solicit bidirectional feedback from staff. Walk rounds occurred daily for a 1 week, weekly for 1 month, monthly for 3 months, and quarterly thereafter. The QI initiative was reviewed quarterly during quality and safety meetings to monitor trends and discuss any concerns. EAC data were originally presented to the unit as a control chart of the monthly rate of cultures per 100 ventilator days (Fig 2). As part of a sustainability plan in Winter 2020, an electronic dashboard was created tracking monthly EACs per 1000 patient days. No hard stops or electronic medical record changes were implemented. Acknowledging that there can be patients with unique needs and in support of clinician autonomy, clinicians could order an EAC incongruent with the algorithm or treat with antibiotics regardless of culture obtainment. The algorithm has not changed since introduction. FIGURE 2 Control chart of the monthly rate of EACs per 100 ventilator days before and after implementation of a decision support algorithm. This U-chart was followed over the first year after implementation of the QI initiative. FIGURE 2 Control chart of the monthly rate of EACs per 100 ventilator days before and after implementation of a decision support algorithm. This U-chart was followed over the first year after implementation of the QI initiative. Close modal The primary outcome was the monthly rate of EACs defined as the number of EACs obtained per 100 ventilator days. Ventilator days were measured by using National Healthcare Safety Network (NHSN) methodology.14 To facilitate future comparisons and as a sustainability metric, we examined the rate of EACs per 1000 PICU patient days as an alternative denominator that was more readily accessible than ventilator days. Process measures included the number of EACs repeated from the same patient within 3 days and days since a patient’s previous EAC. Objective monitoring of adherence with the full algorithm was not feasible because increased quantity of secretions was not reliably documented. To better understand the impact of the algorithm, secondary outcomes included whether the EACs were obtained reflexively with blood and urine cultures (ie, all cultures were ordered at the same time), whether the patient was ventilated via an endotracheal tube >48 hours or had a tracheostomy at the time of EAC, whether the patient received antibiotic treatment, and if the antibiotic treatment was specifically for a new episode of VAI. Antibiotic treatment included ongoing or newly initiated antibiotics for any condition within 2 days of EAC culture obtainment and excluded prophylactic antibiotics. A new episode of treated VAI was defined as initiation of antibiotics continued >2 days with clinician-documented indication for tracheitis, pneumonia, or treating a positive EAC result. For example, if a patient had an EAC followed by 5 days of cefepime for tracheitis and, then, a new EAC followed by 7 days of meropenem for tracheitis, it was considered 2 VAI episodes. Balancing measures included in-hospital mortality, hospital and PICU length of stay, hospital and PICU readmissions within 7 days, and All Patients Refined Diagnosis Related Groups (APR-DRGs) mortality and severity scores. Lastly, we estimated cost savings. First, we calculated charge savings by applying the microbiology laboratory’s average charge to process 1 EAC (ie, culture, identify bacteria, and perform antibiotic susceptibilities) during the baseline period and taking the difference between the 2 years. Then, we transformed charges into costs by multiplying the charge estimates by the median national cost-to-charge ratio (CCR) from the Kids’ Inpatient Database CCR database, the largest publicly available resource of all-payer inpatient pediatric hospitalizations.26,27 Patient demographics; admission, transfer, and discharge data, mortality; APR-DRG mortality and severity scores; ventilator days, patient days; and EACs were queried from the electronic medical system. Clinical data pertaining to individual EACs (eg, time since last EAC, type of ventilation when the EAC was obtained, and antibiotic treatment after EAC) were completed by manual chart review. The Johns Hopkins University School of Medicine Institutional Review Board acknowledged the QI project and approved the study with a waiver of informed consent. As part of the QI project, we monitored the monthly rate of EACs per ventilator days from April 2017 to March 2019 in a statistical process control chart (U-chart) created in Microsoft Excel. After the initial analysis, to monitor sustainability, we examined the rate of EACs per patient days in a control chart using the electronic dashboard that displayed data from July 2016 (when the current electronic medical record system was introduced) to March 2020. The control limits in these charts were set to 3 SDs above and below the mean monthly rate of EACs. The baseline period included through March 2018, and we adjusted the centerline after special cause variation was demonstrated.28 We analyzed the rate of EAC use per 100 ventilator days and per 1000 patient days among PICU patients in the preintervention period (April 2017 to March 2018) compared with the postintervention period (April 2018 to March 2019) using a Poisson regression model with an indicator variable for the postintervention period; and then, using a quasi-experimental interrupted time series (ITS) model with segmented regression of the log-transformed monthly EAC rates.22,29 The ITS model was used to estimate (1) the monthly rate of change in the preintervention period (April 2017 to March 2018), (2) the immediate effect in the month the intervention was introduced (April 2018), and (3) the rate of change in the postintervention period (April 2018 to March 2019) in comparison with the preintervention period. Patient demographic and other outcomes were evaluated with analysis of variance for normally distributed continuous variables, Wilcoxon rank tests for nonnormally distributed variables, χ2 tests for categorical variable, and a 2-sided Poisson test for comparing incidence rates. Stata version 14 (StataCorp, College Station, TX) was used to perform these statistical analyses. In Figure 2, we show the control chart of EAC use per 100 ventilator days. We shifted the centerline in May 2018, after special cause was demonstrated by 1 data point below the lower control limit (the centerline mean shifted from 10.9 to 6.4 EACs per 100 ventilator days). EAC use per ventilator-days and patient-days data are provided in Table 3. In the preintervention period, there was an average of 46 EACs per month and total of 557 cultures over 5092 ventilator days (an incident rate of 10.9 EACs per 100 ventilator days). After introduction of the algorithm, there was an average of 19 EACs per month and total of 234 EACs over 3654 ventilator days (an incident rate of 6.5 EACs per 100 ventilator days). The absolute reduction in EACs was 58%, and the overall decrease in the monthly rate of EACs per ventilator days was 41% (incidence rate ratio [IRR]: 0.59; 95% confidence interval [CI] 0.51–0.67; P < .001). The ITS model revealed a preexisting 2% decline in the monthly culture rate (IRR: 0.98; 95% CI 0.97–1.0; P = .01), immediate 44% drop (IRR: 0.56; 95% CI 0.45–0.70; P = .02), and stable rate in the postintervention period (IRR: 1.03; 95% CI 0.99–1.07; P = .09). A sensitivity analysis varying the start dates confirmed that the immediate drop in EAC rates corresponded with the intervention start date in April 2018. Over the 2-year study period, there were 402 and 398 mechanically ventilated patients in each year, with similar sex, race, and ethnicity (Supplemental Table 5). TABLE 3 EAC Use, Process Measures, and Secondary Outcomes 1 Year Before and After Initiation of a Diagnostic Stewardship Program in the PICU OutcomesYear PreinterventionYear PostinterventionP Primary outcome, culture rates EACs, n 557 234 — Monthly EACs, mean (SD) 46.4 (10.1) 19.5 (4.3) <.001 Ventilator days, n 5092 3654 — Monthly EAC rate per 100 ventilator days, mean (SD) 10.9 (1.4) 6.5 (1.5) <.001 Patient days, n 11 442 10 763 — Monthly EAC rate per 1000 patient days, mean (SD) 48.9 (11.4) 21.8 (5.0) <.001 Process measures EAC repeated within 3 d, n (%) 108 (19.4) 21 (9.0) <.001 Time since previous EAC, d, median (IQR) 7.0 (3.0–14.0) 13.0 (5.0–37.0) <.001 Secondary outcomes EAC collected reflexive with blood and urine cultures, n (%) 197 (35.4) 90 (38.5) .41 Ventilator support at time of EACa Intubated ≥48 h, n (%) 255 (46.6) 90 (40.9) .06 Tracheostomy, n (%) 207 (37.8) 64 (29.1) .008 Antibiotic treatment of any condition after EAC,bn (%) 465 (83.5) 199 (85.0) .59 Antibiotic treatment of a new episode of VAI after EAC,cn (%) 148 (26.6) 60 (25.6) .79 OutcomesYear PreinterventionYear PostinterventionP Primary outcome, culture rates EACs, n 557 234 — Monthly EACs, mean (SD) 46.4 (10.1) 19.5 (4.3) <.001 Ventilator days, n 5092 3654 — Monthly EAC rate per 100 ventilator days, mean (SD) 10.9 (1.4) 6.5 (1.5) <.001 Patient days, n 11 442 10 763 — Monthly EAC rate per 1000 patient days, mean (SD) 48.9 (11.4) 21.8 (5.0) <.001 Process measures EAC repeated within 3 d, n (%) 108 (19.4) 21 (9.0) <.001 Time since previous EAC, d, median (IQR) 7.0 (3.0–14.0) 13.0 (5.0–37.0) <.001 Secondary outcomes EAC collected reflexive with blood and urine cultures, n (%) 197 (35.4) 90 (38.5) .41 Ventilator support at time of EACa Intubated ≥48 h, n (%) 255 (46.6) 90 (40.9) .06 Tracheostomy, n (%) 207 (37.8) 64 (29.1) .008 Antibiotic treatment of any condition after EAC,bn (%) 465 (83.5) 199 (85.0) .59 Antibiotic treatment of a new episode of VAI after EAC,cn (%) 148 (26.6) 60 (25.6) .79 IQR, interquartile range; —, not applicable. a Proportions do not add up to 100 because if the patient was not intubated ≥48 h or with a tracheostomy, the remaining EACs were obtained from patients intubated <48 h (85 EACs in year 1 and 66 EACs in year 2). b Antibiotic treatment included ongoing or newly initiated antibiotics within 2 d of EAC obtainment. Antibiotics given for prophylaxis were excluded. c Newly treated VAI episodes included initiation of antibiotic treatment continued >2 d with clinician-documented indication for tracheitis, pneumonia, or treating a positive EAC result. When examining EAC rates per patient days, there were 11 442 patient days in the preintervention year and 10 763 patient days in the postintervention year (Table 3). There was an overall 55% decline in the monthly rate of EACs in the year after introduction of the algorithm, from 49 to 22 cultures per 1000 patient days (IRR: 0.45; 95% CI 0.37–0.54; P < .001). The ITS model revealed a stable rate of cultures per 1000 patient days during the preintervention year (IRR: 0.98; 95% CI 0.95–1.02; P = .32), followed by an immediate 45% drop with the introduction of the algorithm in April 2018 (IRR: 0.55; 95% CI 0.40–0.70; P < .001) and stable rate in the postintervention period (IRR: 0.98; 95% CI 0.95–1.02; P = .29). A control chart of EACs per 1000 patient days revealed a sustained reduction in EAC use through April 2020 (Fig 3). FIGURE 3 Control chart of the monthly rate of EACs per 1000 patient days before and after implementation of a decision support algorithm. This U-chart was created by using data from an electronic dashboard established in February 2020. The data begin in July 2016, with the introduction of the electronic medical record system, Epic, and depict the EAC rate ∼2 years before and after the introduction of the clinical decision support algorithm in April 2018. FIGURE 3 Control chart of the monthly rate of EACs per 1000 patient days before and after implementation of a decision support algorithm. This U-chart was created by using data from an electronic dashboard established in February 2020. The data begin in July 2016, with the introduction of the electronic medical record system, Epic, and depict the EAC rate ∼2 years before and after the introduction of the clinical decision support algorithm in April 2018. Close modal The process measures associated with EAC obtainment are presented in Table 3. The number of EACs repeated within 3 days fell from 19% to 9% (P < .001), with an 81% absolute reduction. The median time to repeated EAC from the same patient increased from 7 to 13 days (P < .001). The secondary outcomes are presented in Table 3. There was a 54% absolute reduction of EACs obtained reflexively with blood and urine cultures (from 197 to 90 EACs), although the proportion was similar over time. There was a relative decline (from 46.6% to 40.9%; P = .06) and large absolute reduction of 65% (from 255 to 90 EACs) in EACs obtained from patients intubated >48 hours. Similarly, there was a relative decline (from 37.8% to 29.1%, P = .008) and large absolute reduction of 69% (from 207 to 64 EACs) in EACs obtained from patients with tracheostomies. The majority of EACs were associated with the patient receiving antibiotic treatment across both years (83% vs 85%). Although the proportion of EACs associated with treatment of new episodes of VAI was stable (26.6% vs 25.6%), there was a 59% absolute reduction in treated VAI episodes (from 148 to 60 episodes). In-hospital mortality, hospital length of stay, hospital 7-day readmissions, PICU length of stay, PICU 7-day readmissions, or APR-DRG severity and mortality scores were stable across the 2 years (Table 4). We observed a decrease in average monthly ventilator days (from 424 to 305 days; P < .001) and monthly patient days (from 954 to 897; P = .01) between the first and second years. TABLE 4 Balancing Measures 1 Year Before and After Initiation of a Diagnostic Stewardship Program in the PICU Year PreinterventionYear PostinterventionP Hospital admissions with a PICU visit and mechanical ventilation, n 658 637 — In-hospital mortality, n (%) 38 (5.8) 44 (6.9) .40 APR-DRG severity score 1–4, median (IQR)a 3.0 (2.0–4.0) 3.0 (2.0–4.0) .70 APR-DRG mortality score 1–4, median (IQR)a 4.0 (3.0–4.0) 4.0 (3.0–4.0) .80 Hospital length of stay, d, median (IQR) 9 (4–19) 9 (4–22) .81 7-d hospital readmissions, n (%) 50 (7.6) 45 (7.1) .71 PICU visits with mechanical ventilation, n 697 735 — PICU length of stay, d, median (IQR) 3.8 (1.7–8.7) 3.9 (1.7–9.8) .64 7-d PICU readmissions, n (%) 29 (4.2) 26 (3.5) .54 Year PreinterventionYear PostinterventionP Hospital admissions with a PICU visit and mechanical ventilation, n 658 637 — In-hospital mortality, n (%) 38 (5.8) 44 (6.9) .40 APR-DRG severity score 1–4, median (IQR)a 3.0 (2.0–4.0) 3.0 (2.0–4.0) .70 APR-DRG mortality score 1–4, median (IQR)a 4.0 (3.0–4.0) 4.0 (3.0–4.0) .80 Hospital length of stay, d, median (IQR) 9 (4–19) 9 (4–22) .81 7-d hospital readmissions, n (%) 50 (7.6) 45 (7.1) .71 PICU visits with mechanical ventilation, n 697 735 — PICU length of stay, d, median (IQR) 3.8 (1.7–8.7) 3.9 (1.7–9.8) .64 7-d PICU readmissions, n (%) 29 (4.2) 26 (3.5) .54 IQR, interquartile range; —, not applicable. a Denominator for admissions with available APR-DRG data are 645 of 658 admissions in the first year and 617 of 637 admissions in the second year. The average charge to process 1 EAC during the baseline period was$220. The estimated charges were $122 504 and$51 480 in the preintervention and postintervention years, respectively. Applying the median national CCR of 0.363,26  the estimated direct costs were $44 482 and$18 687, respectively. The estimated direct cost savings was \$25 795 per year.

The introduction of a clinical decision support algorithm was associated with a 41% reduction in the rate of EAC per ventilator days and was not associated with potential safety concerns, such as increased in-hospital mortality, length of stay, or hospital readmissions. The reduction in EAC use was demonstrated by using a statistical process control chart and by using ITS analysis, and the findings were robust when measured on the basis of ventilated patient days and PICU patient days. The final implemented algorithm was a consensus among multidisciplinary experts informed by the local assessment of drivers of EAC use, available literature, and collective clinical experience. The introduction of the algorithm reduced reflexive testing and promoted more judicious EAC ordering practices. The median time to repeated cultures from the same patient increased from 7 to 13 days, and there was a 54% reduction in EACs obtained reflexively with blood and urine cultures. Given the paucity of literature around diagnostic stewardship of EACs, it is possible alternative algorithms or approaches could be successful. We believe our approach conducting formative investigation of local practices and potential barriers, along with engaging multidisciplinary stakeholders, contributed to the success of our initiative and led to a sustained change in culturing practices.

Importantly, this algorithm may not apply to all patients. For example, during routine safety and quality meetings, we discussed that patients on extracorporeal membrane oxygenation may not exhibit sufficient signs and symptoms to meet the algorithm. To accommodate differing patient scenarios, no hard stops in the electronic medical record or microbiology laboratory were put in place, and clinician autonomy and discretion was preserved. We did not evaluate the impact of our intervention on VAI rates because NHSN surveillance definitions include positive EAC results. Therefore, a reduction in EACs would confound the interpretation of VAI rates. Furthermore, alternative surveillance criteria, such as ventilator-associated events, may have low specificity for VAIs.30,31  VAI remains a potential complication of mechanical ventilation, and EACs may have an important role in clinical care, helping to identify causative organisms and tailor antibiotic management when patients do have a VAI.

A risk of overuse of EACs is potentiating unnecessary antibiotic treatment. EACs sample a nonsterile site, yet bacterial growth prompts clinicians to treat with antibiotics.1,2,8  We observed that >80% of patients with EACs still were treated with antibiotics. However, given the reduction in the number of EACs, we saw a 59% reduction in treated episodes of VAI after EAC. It is possible that there were patients treated for VAI who did not have an EAC obtained and were not included. In the postintervention period, the overall rate of EACs was still 1 EAC per 15 ventilator days. Therefore, we postulate a reduction in treated VAI episodes reflects a true reduction in antibiotic use in association with a reduced frequency of EACs. Previous studies have also revealed a reduction in antibiotic treatment after diagnostic stewardship of urine cultures.32,33  We plan to further explore the impact of reduced EAC use on antibiotic prescribing and associated cost savings among mechanically ventilated PICU patients in future studies.

The reduction in EAC use led to substantial direct cost savings, which likely underestimated the true cost savings if indirect measures, such as staff time, or additional resources were considered. Improving EAC practice and reducing unnecessary costs of care aligns with the national Choosing Wisely campaign focused on reducing medical overuse.34  There may be additional opportunities to improve EAC use in other settings, such as in ambulatory, acute care, or long-term care facilities caring for patients with tracheostomies.

We recognize a few limitations of this work. First, this was a single-center QI project with retrospective review. It is possible there could be changes in the patient population over time that influenced EAC use. We observed a decline in ventilator days and patient days in the year after implementing the algorithm. There were 3 notable changes, primarily affecting the second year: (1) an extubation readiness test implemented in February 2018; (2) a temporary reduction in PICU beds, from 36 to 30 PICU beds in September 2018; and (3) reduced cardiac surgeries starting in October 2018. Although we did not collect data around indications for PICU admission, we did not observe a change in basic patient demographics, and, if APR-DRG values were considered as a surrogate for complexity, there was not a change in complex admissions in the postintervention year. The overall decline in ventilator and patient days highlights the importance of measuring EAC rates rather than just absolute reductions. Second, although there was not a statistically significant change in mortality, this study is underpowered to detect small changes. This PICU conducts a review of all in-hospital mortality cases, and, in the many active discussions with PICU clinicians after implementation of the algorithm, there were no reported safety concerns attributed to reduced EAC use leading to delayed treatment of infections or complications. Multicenter studies would be necessary to provide sufficient power to statistically assess mortality. Third, we did not assess ventilator-free days because the NHSN ventilator days did not distinguish the route of ventilation and it would not be an applicable metric for ventilator-dependent patients with tracheostomies. Finally, our center could have unique patient characteristics, workflow, or local culture that influenced both preexisting EAC use and the response to the algorithm. There are no previous data regarding optimal EAC use rates to compare our EAC rates with those of other facilities as a benchmark.

We used local assessments and existing literature to inform the development and implementation of a clinical decision support algorithm to standardize EAC practice among ventilated PICU patients. The introduction of the algorithm was associated with a significant decline in the rate of EACs without changes in mortality, readmissions, or length of stay. Further studies are needed to understand the safety and reproducibility of decision support tools to improve EAC use in other settings and patient populations.

We acknowledge Dr Kristin Voegtline from the Johns Hopkins Biostatistics, Epidemiology and Data Management Core for assistance with data management, Dr Eili Klein for creation of the electronic dashboard, and Avi Gadala and Guanyu Li for clinical data coordination and retrieval from the Core for Clinical Research Data Acquisition, supported in part by the Johns Hopkins Institute for Clinical and Translational Research (UL1TR001079). We thank the Johns Hopkins Microbiology Laboratory and the staff of the Johns Hopkins Children’s Center PICU.

Dr Sick-Samuels conceptualized and designed the study, collected data, analyzed data, drafted the initial manuscript, and reviewed and revised the manuscript; Mr Linz and Dr Bergman designed data collection instruments, collected data, analyzed data, and reviewed and revised the manuscript; Dr Hoops and Mr Dwyer conceptualized and designed the study and critically reviewed the manuscript for important intellectual content; Drs Fackler, Ralston, Berenholtz, and Milstone conceptualized and designed the study and reviewed and revised the manuscript; Dr Colantuoni participated in study design, analyzed data, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: Funded in part by the Johns Hopkins Eudowood Board Bauernschmidt Award to Dr Sick-Samuels; National Institutes of Health grant KL2TR003099 to Dr Sick-Samuels and grant K24AI141580 to Dr Milstone; and Agency for Healthcare Research and Quality grant R18HS025642 to Dr Milstone. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies. Funded by the National Institutes of Health (NIH).

• APR-DRG

All Patients Refined Diagnosis Related Group

•
• CCR

cost-to-charge ratio

•
• CI

confidence interval

•
• EAC

endotracheal aspirate culture

•
• IRR

incidence rate ratio

•
• ITS

interrupted time series

•
• NHSN

National Healthcare Safety Network

•
• QI

quality improvement

•
• VAI

ventilator-associated infection

•
• VAP

ventilator-associated pneumonia

1
Willson
DF
,
Hoot
M
,
Khemani
R
, et al.;
Ventilator-Associated INfection (VAIN) Investigators and the Pediatric Acute Lung Injury and Sepsis Investigator’s (PALISI) Network
.
Pediatric ventilator-associated infections: the Ventilator-Associated INfection Study
.
Pediatr Crit Care Med
.
2017
;
18
(
1
):
e24
e34
2
Willson
DF
,
Kirby
A
,
Kicker
JS
.
Respiratory secretion analyses in the evaluation of ventilator-associated pneumonia: a survey of current practice in pediatric critical care
.
Pediatr Crit Care Med
.
2014
;
15
(
8
):
715
719
3
Raymond
J
,
Aujard
Y
;
European Study Group
.
Nosocomial infections in pediatric patients: a European, multicenter prospective study
.
Infect Control Hosp Epidemiol
.
2000
;
21
(
4
):
260
263
4
Grohskopf
LA
,
Sinkowitz-Cochran
RL
,
Garrett
DO
, et al.;
Pediatric Prevention Network
.
A national point-prevalence survey of pediatric intensive care unit-acquired infections in the United States
.
J Pediatr
.
2002
;
140
(
4
):
432
438
5
Hill
JD
,
Ratliff
JL
,
Parrott
JC
, et al
.
Pulmonary pathology in acute respiratory insufficiency: lung biopsy as a diagnostic tool
.
J Thorac Cardiovasc Surg
.
1976
;
71
(
1
):
64
71
6
Durairaj
L
,
Z
,
Launspach
JL
, et al
.
Patterns and density of early tracheal colonization in intensive care unit patients
.
J Crit Care
.
2009
;
24
(
1
):
114
121
7
Willson
DF
,
Conaway
M
,
Kelly
R
,
Hendley
JO
.
The lack of specificity of tracheal aspirates in the diagnosis of pulmonary infection in intubated children
.
Pediatr Crit Care Med
.
2014
;
15
(
4
):
299
305
8
Venkatachalam
V
,
Hendley
JO
,
Willson
DF
.
The diagnostic dilemma of ventilator-associated pneumonia in critically ill children
.
Pediatr Crit Care Med
.
2011
;
12
(
3
):
286
296
9
Carcillo
JA
,
Dean
JM
,
Holubkov
R
, et al.;
Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) Collaborative Pediatric Critical Care Research Network (CPCCRN)
.
The randomized comparative pediatric critical illness stress-induced immune suppression (CRISIS) prevention trial
.
Pediatr Crit Care Med
.
2012
;
13
(
2
):
165
173
10
Fischer
JE
,
Ramser
M
,
Fanconi
S
.
Use of antibiotics in pediatric intensive care and potential savings
.
Intensive Care Med
.
2000
;
26
(
7
):
959
966
11
Fayon
MJ
,
Tucci
M
,
Lacroix
J
, et al
.
Nosocomial pneumonia and tracheitis in a pediatric intensive care unit: a prospective study
.
Am J Respir Crit Care Med
.
1997
;
155
(
1
):
162
169
12
Gauvin
F
,
Dassa
C
,
Chaïbou
M
,
Proulx
F
,
Farrell
CA
,
Lacroix
J
.
Ventilator-associated pneumonia in intubated children: comparison of different diagnostic methods
.
Pediatr Crit Care Med
.
2003
;
4
(
4
):
437
443
13
Craven
DE
,
Chroneou
A
,
Zias
N
,
Hjalmarson
KI
.
Ventilator-associated tracheobronchitis: the impact of targeted antibiotic therapy on patient outcomes
.
Chest
.
2009
;
135
(
2
):
521
528
14
Centers for Disease Control and Prevention
.
National Healthcare Safety Network (NHSN) Patient Safety Component Manual
.
Atlanta, GA
:
Centers for Disease Control and Prevention
;
2019
15
Kalil
AC
,
Metersky
ML
,
Klompas
M
, et al
.
Management of adults with hospital-acquired and ventilator-associated pneumonia: 2016 Clinical Practice Guidelines by the Infectious Diseases Society of America and the American Thoracic Society. [published corrections appear in Clin Infect Dis. 2017;64(9):1298; Clin Infect Dis. 2017;65(8):1435; and Clin Infect Dis. 2017;65(12):2161]
.
Clin Infect Dis
.
2016
;
63
(
5
):
e61
e111
16
Sick-Samuels
AC
,
Fackler
JC
,
Berenholtz
SM
,
Milstone
AM
.
Understanding reasons clinicians obtained endotracheal aspirate cultures and impact on patient management to inform diagnostic stewardship initiatives
.
Infect Control Hosp Epidemiol
.
2020
;
41
(
2
):
240
242
17
Morgan
DJ
,
Malani
P
,
Diekema
DJ
.
Diagnostic stewardship-leveraging the laboratory to improve antimicrobial use
.
JAMA
.
2017
;
318
(
7
):
607
608
18
Sperling
K
,
Priddy
A
,
Suntharam
N
,
Feuerhake
T
.
Optimizing testing for Clostridium difficile infection: a quality improvement project
.
Am J Infect Control
.
2019
;
47
(
3
):
340
342
19
Truong
CY
,
Gombar
S
,
Wilson
R
, et al
.
Real-time electronic tracking of diarrheal episodes and laxative therapy enables verification of Clostridium difficile clinical testing criteria and reduction of Clostridium difficile infection rates
.
J Clin Microbiol
.
2017
;
55
(
5
):
1276
1284
20
Epstein
L
,
Edwards
JR
,
Halpin
AL
, et al
.
Evaluation of a novel intervention to reduce unnecessary urine cultures in intensive care units at a tertiary care hospital in Maryland, 2011-2014
.
Infect Control Hosp Epidemiol
.
2016
;
37
(
5
):
606
609
21
Mullin
KM
,
Kovacs
CS
,
Fatica
C
, et al
.
A multifaceted approach to reduction of catheter-associated urinary tract infections in the intensive care unit with an emphasis on “stewardship of culturing”
.
Infect Control Hosp Epidemiol
.
2017
;
38
(
2
):
186
188
22
Woods-Hill
CZ
,
Fackler
J
,
Nelson McMillan
K
, et al
.
Association of a clinical practice guideline with blood culture use in critically ill children
.
JAMA Pediatr
.
2017
;
171
(
2
):
157
164
23
Woods-Hill
CZ
,
Lee
L
,
Xie
A
, et al
.
Dissemination of a novel framework to improve blood culture use in pediatric critical care
.
Pediatr Qual Saf
.
2018
;
3
(
5
):
e112
24
Pronovost
PJ
,
Berenholtz
SM
,
Needham
DM
.
Translating evidence into practice: a model for large scale knowledge translation
.
BMJ
.
2008
;
337
:
a1714
25
Branson
RD
.
Secretion management in the mechanically ventilated patient
.
Respir Care
.
2007
;
52
(
10
):
1328
1342, NaN–1347
26
Agency for Healthcare Research and Quality
;
Healthcare Cost and Utilization Project
.
HCUP Cost-to-Charge Ratio for Kids’ Inpatient Database (KID)
.
Rockville, MD
:
Agency for Healthcare Research and Quality
;
2016
27
Healthcare Cost and Utilization Project
.
Cost-to-charge ratio files: user guide for Kids’ Inpatient Database (KID) CCRs. Available at: https://www.hcup-us.ahrq.gov/db/state/CCR_KID_UserGuide_2003-2016.pdf. Accessed January 13, 2020
28
Provost
LP
,
Murray
SK
.
The Health Care Data Guide: Learning from Data for Improvement
.
San Francisco, CA
:
Jossey-Bass
;
2011
29
Schweizer
ML
,
Braun
BI
,
Milstone
AM
.
Research methods in healthcare epidemiology and antimicrobial stewardship-quasi-experimental designs
.
Infect Control Hosp Epidemiol
.
2016
;
37
(
10
):
1135
1140
30
Ziegler
KM
,
Haywood
JD
,
Sontag
MK
,
Mourani
PM
.
Application of the new Centers for Disease Control and Prevention surveillance criteria for ventilator-associated events to a cohort of PICU patients identifies different patients compared with the previous definition and physician diagnosis
.
Crit Care Med
.
2019
;
47
(
7
):
e547
e554
31
Willson
DF
,
Hall
M
,
Beardsley
A
, et al.;
Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network
.
Pediatric ventilator-associated events: analysis of the pediatric ventilator-associated infection data
.
Pediatr Crit Care Med
.
2018
;
19
(
12
):
e631
e636
32
Hartley
SE
,
Kuhn
L
,
Valley
S
, et al
.
Evaluating a hospitalist-based intervention to decrease unnecessary antimicrobial use in patients with asymptomatic bacteriuria
.
Infect Control Hosp Epidemiol
.
2016
;
37
(
9
):
1044
1051
33
Stagg
A
,
Lutz
H
,
Kirpalaney
S
, et al
.
Impact of two-step urine culture ordering in the emergency department: a time series analysis
.
BMJ Qual Saf
.
2018
;
27
(
2
):
140
147
34
Morgan
DJ
,
Croft
LD
,
Deloney
V
, et al
.
Choosing wisely in healthcare epidemiology and antimicrobial stewardship
.
Infect Control Hosp Epidemiol
.
2016
;
37
(
7
):
755
760
35
Cocoros
NM
,
Kleinman
K
,
Priebe
GP
, et al.;
Pediatric Ventilator-Associated Conditions Study Team
.
Ventilator-associated events in neonates and children–a new paradigm
.
Crit Care Med
.
2016
;
44
(
1
):
14
22
36
Pugin
J
,
Auckenthaler
R
,
Mili
N
,
Janssens
JP
,
Lew
PD
,
Suter
PM
.
Diagnosis of ventilator-associated pneumonia by bacteriologic analysis of bronchoscopic and nonbronchoscopic “blind” bronchoalveolar lavage fluid
.
Am Rev Respir Dis
.
1991
;
143
(
5
):
1121
1129
37
Altiner
A
,
Wilm
S
,
Däubener
W
, et al
.
Sputum colour for diagnosis of a bacterial infection in patients with acute cough
.
Scand J Prim Health Care
.
2009
;
27
(
2
):
70
73
38
Brusse-Keizer
MGJ
,
Grotenhuis
AJ
,
Kerstjens
HAM
, et al
.
Relation of sputum colour to bacterial load in acute exacerbations of COPD
.
Respir Med
.
2009
;
103
(
4
):
601
606
39
Daniels
JMA
,
de Graaff
CS
,
Vlaspolder
F
,
Snijders
D
,
Jansen
HM
,
Boersma
WG
.
Sputum colour reported by patients is not a reliable marker of the presence of bacteria in acute exacerbations of chronic obstructive pulmonary disease
.
Clin Microbiol Infect
.
2010
;
16
(
6
):
583
588
40
Reychler
G
,
Andre
E
,
Couturiaux
L
, et al
.
Reproducibility of the sputum color evaluation depends on the category of caregivers
.
Respir Care
.
2016
;
61
(
7
):
936
942
41
Yalamanchi
S
,
Saiman
L
,
Zachariah
P
.
Decision-making around positive tracheal aspirate cultures: the role of neutrophil semiquantification in antibiotic prescribing
.
Pediatr Crit Care Med
.
2019
;
20
(
8
):
e380
e385

## Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.