Diagnostic Stewardship of Endotracheal Aspirate Cultures in a PICU

This study was conducted in the PICU of the Johns Hopkins Children’s Center in Baltimore, Maryland, a quaternary care academic PICU with 36 beds caring for medical, surgical, and cardiac patients from birth to 24 years of age, with 2000 yearly admissions.

We performed an analysis of a quality improvement (QI) program to improve EAC use to measure associated outcomes and safety. In this study, we specifically reviewed all patients admitted to the unit who were mechanically ventilated via endotracheal tube or tracheostomy 1 year before implementation of the algorithm (preintervention: April 1, 2017, to March 31, 2018) and 1 year after (postintervention: April 1, 2018, to March 31, 2019).

A time line of the QI initiative is summarized in Table 1. In 2016, PICU and infectious diseases physicians expressed concern for the frequently repeated and reflexive use of EACs in the Johns Hopkins Children’s Center PICU. A multidisciplinary QI team reviewed baseline EAC use and conducted clinician surveys, which supported that there was a low threshold to obtain EACs.¹⁶  We held focus groups with PICU nurse practitioners, physicians, and respiratory therapists, discussing EAC practices and potential barriers to change. The formative work supported that there were drivers of EAC use including (1) reflexive testing in response to isolated clinical changes (eg, fever) and (2) significance attributed to reported changes in respiratory secretions. With the support of PICU leadership, the QI team used a translating-evidence-into-practice model to improve EAC use.²⁴  The local drivers were combined with existing literature, local data, and workgroup consensus to inform algorithm development and implementation (detailed in Table 2). A workgroup, including critical care attending physicians, fellows, nurse practitioners, and infectious disease physicians, drafted an algorithm, shown in Fig 1, to guide clinician decision-making around EACs. The team solicited input from PICU respiratory therapists and nurses and clinical microbiology laboratory faculty. The primary objective of the algorithm was to reduce reflexive culturing from mechanically ventilated patients without signs or symptoms of a respiratory infection by prompting clinicians to consider specific changes supporting a respiratory infection and timing since last EAC. The algorithm page included guidance for specimen collection to avoid the use of saline lavage because this practice can dilute specimens and is not recommended.²⁵ 

The algorithm was disseminated to all PICU staff preceding initiation on April 1, 2018. For 2 months, fellows or attending physicians were asked to sign off on a paper checklist of the algorithm when EACs were obtained. The QI team conducted in-person walk rounds to solicit bidirectional feedback from staff. Walk rounds occurred daily for a 1 week, weekly for 1 month, monthly for 3 months, and quarterly thereafter. The QI initiative was reviewed quarterly during quality and safety meetings to monitor trends and discuss any concerns. EAC data were originally presented to the unit as a control chart of the monthly rate of cultures per 100 ventilator days (Fig 2). As part of a sustainability plan in Winter 2020, an electronic dashboard was created tracking monthly EACs per 1000 patient days. No hard stops or electronic medical record changes were implemented. Acknowledging that there can be patients with unique needs and in support of clinician autonomy, clinicians could order an EAC incongruent with the algorithm or treat with antibiotics regardless of culture obtainment. The algorithm has not changed since introduction.

The primary outcome was the monthly rate of EACs defined as the number of EACs obtained per 100 ventilator days. Ventilator days were measured by using National Healthcare Safety Network (NHSN) methodology.¹⁴  To facilitate future comparisons and as a sustainability metric, we examined the rate of EACs per 1000 PICU patient days as an alternative denominator that was more readily accessible than ventilator days. Process measures included the number of EACs repeated from the same patient within 3 days and days since a patient’s previous EAC. Objective monitoring of adherence with the full algorithm was not feasible because increased quantity of secretions was not reliably documented. To better understand the impact of the algorithm, secondary outcomes included whether the EACs were obtained reflexively with blood and urine cultures (ie, all cultures were ordered at the same time), whether the patient was ventilated via an endotracheal tube >48 hours or had a tracheostomy at the time of EAC, whether the patient received antibiotic treatment, and if the antibiotic treatment was specifically for a new episode of VAI. Antibiotic treatment included ongoing or newly initiated antibiotics for any condition within 2 days of EAC culture obtainment and excluded prophylactic antibiotics. A new episode of treated VAI was defined as initiation of antibiotics continued >2 days with clinician-documented indication for tracheitis, pneumonia, or treating a positive EAC result. For example, if a patient had an EAC followed by 5 days of cefepime for tracheitis and, then, a new EAC followed by 7 days of meropenem for tracheitis, it was considered 2 VAI episodes.

Balancing measures included in-hospital mortality, hospital and PICU length of stay, hospital and PICU readmissions within 7 days, and All Patients Refined Diagnosis Related Groups (APR-DRGs) mortality and severity scores. Lastly, we estimated cost savings. First, we calculated charge savings by applying the microbiology laboratory’s average charge to process 1 EAC (ie, culture, identify bacteria, and perform antibiotic susceptibilities) during the baseline period and taking the difference between the 2 years. Then, we transformed charges into costs by multiplying the charge estimates by the median national cost-to-charge ratio (CCR) from the Kids’ Inpatient Database CCR database, the largest publicly available resource of all-payer inpatient pediatric hospitalizations.^26,27 

Patient demographics; admission, transfer, and discharge data, mortality; APR-DRG mortality and severity scores; ventilator days, patient days; and EACs were queried from the electronic medical system. Clinical data pertaining to individual EACs (eg, time since last EAC, type of ventilation when the EAC was obtained, and antibiotic treatment after EAC) were completed by manual chart review. The Johns Hopkins University School of Medicine Institutional Review Board acknowledged the QI project and approved the study with a waiver of informed consent.

As part of the QI project, we monitored the monthly rate of EACs per ventilator days from April 2017 to March 2019 in a statistical process control chart (U-chart) created in Microsoft Excel. After the initial analysis, to monitor sustainability, we examined the rate of EACs per patient days in a control chart using the electronic dashboard that displayed data from July 2016 (when the current electronic medical record system was introduced) to March 2020. The control limits in these charts were set to 3 SDs above and below the mean monthly rate of EACs. The baseline period included through March 2018, and we adjusted the centerline after special cause variation was demonstrated.²⁸ 

We analyzed the rate of EAC use per 100 ventilator days and per 1000 patient days among PICU patients in the preintervention period (April 2017 to March 2018) compared with the postintervention period (April 2018 to March 2019) using a Poisson regression model with an indicator variable for the postintervention period; and then, using a quasi-experimental interrupted time series (ITS) model with segmented regression of the log-transformed monthly EAC rates.^22,29  The ITS model was used to estimate (1) the monthly rate of change in the preintervention period (April 2017 to March 2018), (2) the immediate effect in the month the intervention was introduced (April 2018), and (3) the rate of change in the postintervention period (April 2018 to March 2019) in comparison with the preintervention period. Patient demographic and other outcomes were evaluated with analysis of variance for normally distributed continuous variables, Wilcoxon rank tests for nonnormally distributed variables, χ² tests for categorical variable, and a 2-sided Poisson test for comparing incidence rates. Stata version 14 (StataCorp, College Station, TX) was used to perform these statistical analyses.

In Figure 2, we show the control chart of EAC use per 100 ventilator days. We shifted the centerline in May 2018, after special cause was demonstrated by 1 data point below the lower control limit (the centerline mean shifted from 10.9 to 6.4 EACs per 100 ventilator days). EAC use per ventilator-days and patient-days data are provided in Table 3. In the preintervention period, there was an average of 46 EACs per month and total of 557 cultures over 5092 ventilator days (an incident rate of 10.9 EACs per 100 ventilator days). After introduction of the algorithm, there was an average of 19 EACs per month and total of 234 EACs over 3654 ventilator days (an incident rate of 6.5 EACs per 100 ventilator days). The absolute reduction in EACs was 58%, and the overall decrease in the monthly rate of EACs per ventilator days was 41% (incidence rate ratio [IRR]: 0.59; 95% confidence interval [CI] 0.51–0.67; P < .001). The ITS model revealed a preexisting 2% decline in the monthly culture rate (IRR: 0.98; 95% CI 0.97–1.0; P = .01), immediate 44% drop (IRR: 0.56; 95% CI 0.45–0.70; P = .02), and stable rate in the postintervention period (IRR: 1.03; 95% CI 0.99–1.07; P = .09). A sensitivity analysis varying the start dates confirmed that the immediate drop in EAC rates corresponded with the intervention start date in April 2018. Over the 2-year study period, there were 402 and 398 mechanically ventilated patients in each year, with similar sex, race, and ethnicity (Supplemental Table 5).

When examining EAC rates per patient days, there were 11 442 patient days in the preintervention year and 10 763 patient days in the postintervention year (Table 3). There was an overall 55% decline in the monthly rate of EACs in the year after introduction of the algorithm, from 49 to 22 cultures per 1000 patient days (IRR: 0.45; 95% CI 0.37–0.54; P < .001). The ITS model revealed a stable rate of cultures per 1000 patient days during the preintervention year (IRR: 0.98; 95% CI 0.95–1.02; P = .32), followed by an immediate 45% drop with the introduction of the algorithm in April 2018 (IRR: 0.55; 95% CI 0.40–0.70; P < .001) and stable rate in the postintervention period (IRR: 0.98; 95% CI 0.95–1.02; P = .29). A control chart of EACs per 1000 patient days revealed a sustained reduction in EAC use through April 2020 (Fig 3).

The process measures associated with EAC obtainment are presented in Table 3. The number of EACs repeated within 3 days fell from 19% to 9% (P < .001), with an 81% absolute reduction. The median time to repeated EAC from the same patient increased from 7 to 13 days (P < .001).

The secondary outcomes are presented in Table 3. There was a 54% absolute reduction of EACs obtained reflexively with blood and urine cultures (from 197 to 90 EACs), although the proportion was similar over time. There was a relative decline (from 46.6% to 40.9%; P = .06) and large absolute reduction of 65% (from 255 to 90 EACs) in EACs obtained from patients intubated >48 hours. Similarly, there was a relative decline (from 37.8% to 29.1%, P = .008) and large absolute reduction of 69% (from 207 to 64 EACs) in EACs obtained from patients with tracheostomies. The majority of EACs were associated with the patient receiving antibiotic treatment across both years (83% vs 85%). Although the proportion of EACs associated with treatment of new episodes of VAI was stable (26.6% vs 25.6%), there was a 59% absolute reduction in treated VAI episodes (from 148 to 60 episodes).

In-hospital mortality, hospital length of stay, hospital 7-day readmissions, PICU length of stay, PICU 7-day readmissions, or APR-DRG severity and mortality scores were stable across the 2 years (Table 4). We observed a decrease in average monthly ventilator days (from 424 to 305 days; P < .001) and monthly patient days (from 954 to 897; P = .01) between the first and second years.

The average charge to process 1 EAC during the baseline period was $220. The estimated charges were $122 504 and $51 480 in the preintervention and postintervention years, respectively. Applying the median national CCR of 0.363,²⁶  the estimated direct costs were $44 482 and $18 687, respectively. The estimated direct cost savings was $25 795 per year.

The introduction of a clinical decision support algorithm was associated with a 41% reduction in the rate of EAC per ventilator days and was not associated with potential safety concerns, such as increased in-hospital mortality, length of stay, or hospital readmissions. The reduction in EAC use was demonstrated by using a statistical process control chart and by using ITS analysis, and the findings were robust when measured on the basis of ventilated patient days and PICU patient days. The final implemented algorithm was a consensus among multidisciplinary experts informed by the local assessment of drivers of EAC use, available literature, and collective clinical experience. The introduction of the algorithm reduced reflexive testing and promoted more judicious EAC ordering practices. The median time to repeated cultures from the same patient increased from 7 to 13 days, and there was a 54% reduction in EACs obtained reflexively with blood and urine cultures. Given the paucity of literature around diagnostic stewardship of EACs, it is possible alternative algorithms or approaches could be successful. We believe our approach conducting formative investigation of local practices and potential barriers, along with engaging multidisciplinary stakeholders, contributed to the success of our initiative and led to a sustained change in culturing practices.

Importantly, this algorithm may not apply to all patients. For example, during routine safety and quality meetings, we discussed that patients on extracorporeal membrane oxygenation may not exhibit sufficient signs and symptoms to meet the algorithm. To accommodate differing patient scenarios, no hard stops in the electronic medical record or microbiology laboratory were put in place, and clinician autonomy and discretion was preserved. We did not evaluate the impact of our intervention on VAI rates because NHSN surveillance definitions include positive EAC results. Therefore, a reduction in EACs would confound the interpretation of VAI rates. Furthermore, alternative surveillance criteria, such as ventilator-associated events, may have low specificity for VAIs.^30,31  VAI remains a potential complication of mechanical ventilation, and EACs may have an important role in clinical care, helping to identify causative organisms and tailor antibiotic management when patients do have a VAI.

A risk of overuse of EACs is potentiating unnecessary antibiotic treatment. EACs sample a nonsterile site, yet bacterial growth prompts clinicians to treat with antibiotics.^1,2,8  We observed that >80% of patients with EACs still were treated with antibiotics. However, given the reduction in the number of EACs, we saw a 59% reduction in treated episodes of VAI after EAC. It is possible that there were patients treated for VAI who did not have an EAC obtained and were not included. In the postintervention period, the overall rate of EACs was still 1 EAC per 15 ventilator days. Therefore, we postulate a reduction in treated VAI episodes reflects a true reduction in antibiotic use in association with a reduced frequency of EACs. Previous studies have also revealed a reduction in antibiotic treatment after diagnostic stewardship of urine cultures.^32,33  We plan to further explore the impact of reduced EAC use on antibiotic prescribing and associated cost savings among mechanically ventilated PICU patients in future studies.

The reduction in EAC use led to substantial direct cost savings, which likely underestimated the true cost savings if indirect measures, such as staff time, or additional resources were considered. Improving EAC practice and reducing unnecessary costs of care aligns with the national Choosing Wisely campaign focused on reducing medical overuse.³⁴  There may be additional opportunities to improve EAC use in other settings, such as in ambulatory, acute care, or long-term care facilities caring for patients with tracheostomies.

We recognize a few limitations of this work. First, this was a single-center QI project with retrospective review. It is possible there could be changes in the patient population over time that influenced EAC use. We observed a decline in ventilator days and patient days in the year after implementing the algorithm. There were 3 notable changes, primarily affecting the second year: (1) an extubation readiness test implemented in February 2018; (2) a temporary reduction in PICU beds, from 36 to 30 PICU beds in September 2018; and (3) reduced cardiac surgeries starting in October 2018. Although we did not collect data around indications for PICU admission, we did not observe a change in basic patient demographics, and, if APR-DRG values were considered as a surrogate for complexity, there was not a change in complex admissions in the postintervention year. The overall decline in ventilator and patient days highlights the importance of measuring EAC rates rather than just absolute reductions. Second, although there was not a statistically significant change in mortality, this study is underpowered to detect small changes. This PICU conducts a review of all in-hospital mortality cases, and, in the many active discussions with PICU clinicians after implementation of the algorithm, there were no reported safety concerns attributed to reduced EAC use leading to delayed treatment of infections or complications. Multicenter studies would be necessary to provide sufficient power to statistically assess mortality. Third, we did not assess ventilator-free days because the NHSN ventilator days did not distinguish the route of ventilation and it would not be an applicable metric for ventilator-dependent patients with tracheostomies. Finally, our center could have unique patient characteristics, workflow, or local culture that influenced both preexisting EAC use and the response to the algorithm. There are no previous data regarding optimal EAC use rates to compare our EAC rates with those of other facilities as a benchmark.

We used local assessments and existing literature to inform the development and implementation of a clinical decision support algorithm to standardize EAC practice among ventilated PICU patients. The introduction of the algorithm was associated with a significant decline in the rate of EACs without changes in mortality, readmissions, or length of stay. Further studies are needed to understand the safety and reproducibility of decision support tools to improve EAC use in other settings and patient populations.

We acknowledge Dr Kristin Voegtline from the Johns Hopkins Biostatistics, Epidemiology and Data Management Core for assistance with data management, Dr Eili Klein for creation of the electronic dashboard, and Avi Gadala and Guanyu Li for clinical data coordination and retrieval from the Core for Clinical Research Data Acquisition, supported in part by the Johns Hopkins Institute for Clinical and Translational Research (UL1TR001079). We thank the Johns Hopkins Microbiology Laboratory and the staff of the Johns Hopkins Children’s Center PICU.

APR-DRG

All Patients Refined Diagnosis Related Group

CCR

cost-to-charge ratio

CI

confidence interval

EAC

endotracheal aspirate culture

IRR

incidence rate ratio

ITS

interrupted time series

NHSN

National Healthcare Safety Network

QI

quality improvement

VAI

ventilator-associated infection

VAP

ventilator-associated pneumonia