Vignettes Identify Variation in Antibiotic Use for Suspected Early Onset Sepsis

This study was conducted during a statewide antibiotic stewardship QI collaborative in the western United States. The collaborative aimed to reduce antibiotic use by targeting EOS antibiotic use. The collaborative included 28 NICUs. The collaborative began on June 3, 2016, and ended on December 14, 2017. In this study, we used a vignette survey of individual participants that was emailed on January 19, 2017, with responses collected until April 4, 2017.

A previously published set of EOS clinical case vignettes was used as a model for development of the vignette cases.¹⁰  The specific clinical case scenarios were determined from discussions with QI team participants at in-person meetings and monthly webcasts. The vignette survey was piloted with the stewardship collaborative expert panel members. Two separate vignette series, with 3 questions in each series, were created. Vignette series A targeted whether respondents would start antibiotics in the case of a newborn being evaluated for EOS. Vignette series B targeted whether respondents would discontinue antibiotics or continue antibiotics for a treatment course after 48 hours.

Each case vignette included 1 multiple-choice question that allowed for free text answers, if desired (Table 1). The web-based vignette survey was sent to the team leader from each of the 28 participating NICUs. The team leaders were instructed to forward the survey to individuals responsible for ordering antibiotics in their NICU. Surveys were created and distributed by using Qualtrics Survey Platform (Qualtrics, Provo, UT).¹²  Cedars-Sinai Medical Center Office of Research Compliance and QI approved a waiver of consent for this QI study.

The quantitative (multiple choice) and qualitative (free text) survey responses were consolidated into binary results to reflect either “starting antibiotics” or “not starting antibiotics” and “discontinuing antibiotics” or “treating for 5 days or longer” for vignette sets A and B, respectively. Two of the study authors separately (K.S.E.P. and C.P.) coded all the qualitative survey responses that were free texted as the “other option.” The 2 authors obtained similar results.

A coefficient of unalikeability (CU)¹³  was calculated to describe the degree of variability between the final binary results. The CU describes how often answers differed from each other, with 0 representing minimum variability and 0.5 representing maximum variability. Fisher’s exact test and mixed logistics regression were used to determine if responses differed by NICU site or role of the individual respondents. We also estimated the intraclass correlation (ICC), which gives the proportion of the total variation at the site level on the basis of the null model of mixed logistics regression accounting for sites as a random effect. Analysis was computed by using SAS 9.4 (SAS Institute, Inc, Cary, NC).¹⁴ 

We received surveys from at least 1 participant from 16 of the 28 participating NICUs. Respondents included neonatologists (n = 73), hospitalists (n = 15), neonatal nurse practitioners (n = 11), neonatal-perinatal medicine fellows (n = 6), and clinical nurse specialists (n = 2). The range of total number of binary responses for each question analyzed, resulting from the sum of coding of each multiple-choice and free text response, varied from 106 to 116 for each question (Tables 2 and 3).

The most variation (CU = 0.5) was noted in the decision to discontinue antibiotics in the case with “respiratory signs for 72 hours, plus risk factors” (case B3). The other 2 cases with the largest variation included decisions to start antibiotics for the cases describing a newborn who was “well appearing, with elevated CRP, and risk factors” (case A1) and a newborn with “respiratory signs and no risk factors” (case A2), both with a CU of 0.42. The 3 cases with the least variation had CUs of 0.05, 0.19, and 0.32 (Tables 2 and 3).

Analysis of responses by site suggested that 1 NICU had respondents more likely than other sites to choose to stop antibiotics in the case of “respiratory signs for 72 hours plus risk factors” (case B3), but this difference was not statistically significant (P = .09). There were no other notable differences in responses by site or role.

Individual practice variation is a barrier to reducing unnecessary antibiotic use in NICUs. Four helpful targets for antibiotic stewardship efforts emerged from this study related to perceptions of culture negative sepsis, response to elevated C-reactive protein (CRP), response to varying respiratory signs, and response to sepsis risk factors.

First, the largest variation in responses occurred with the decisions to discontinue antibiotics in the case of “respiratory signs for 72 hours plus risk factors” (case B3). Not surprisingly, the responses to this case may reflect a lack of evidence-based consensus regarding assessment of newborns with risk factors and persistent respiratory signs of unclear etiology. Discussions among individual participants during webcasts and face-to-face sessions highlighted that those choosing to continue antibiotic treatment interpreted this scenario as representing “culture negative sepsis.” Alternatively, respondents electing to discontinue antibiotics cited the negative blood culture, lack of critical illness, and resolution of clinical signs to suggest a noninfectious etiology. There continues to be controversy regarding “culture negative sepsis,” with one perspective suggesting that this diagnosis is less common and less relevant than previously thought.¹⁵  The American Association of Pediatrics clinical report on management of infants at ≥35 0/7 weeks’ gestation with suspected or proven culture positive EOS states that “when blood cultures are sterile, antibiotics should be discontinued at 36 to 48 hours, unless there is site specific evidence of infection”.¹⁶  Antibiotic stewardship efforts should address provider perceptions of culture negative sepsis to reduce variation in antibiotic use.

Second, there was notable variation in the antibiotic decisions based on an elevated CRP in a well-appearing newborn (case A1). Current evidence suggests that starting antibiotics for an elevated CRP alone in this scenario is not warranted.^16–18  Additionally, abnormal CBC indices alone and/or serial CRP or other inflammatory markers are not recommended to inform decisions to start or continue antibiotics in well-appearing newborns.¹⁶  Targeting management variation of well-appearing newborns with risk factors and abnormal screening laboratories may be beneficial to stewardship teams.

Third, there was variation in the decision to start antibiotics in the newborn with “respiratory signs and no sepsis risk factors” (case A2). These results suggest that obtaining consensus regarding respiratory sign thresholds prompting antibiotic use could be helpful to minimize variation in starting antibiotics in low risk newborns. Additionally, it highlights that consensus on the duration of time that is acceptable for close monitoring without antibiotics could help reduce variation.

Last, the addition of antenatal maternal risk factors (fever and prolonged rupture of membranes) with all other case variables remaining unchanged prompted a large shift in respondents choosing to start antibiotics (from 29% to 84%, respectively) (case A2 and A3, Table 2). This judgment is consistent with both clinical observations¹⁹  and risk modeling,²⁰  which indicates that risk indicators often compound one another. Thus, standardizing provider responses to EOS risk factors, such as those incorporated in risk-based decision aids such as the neonatal sepsis calculator,^21,22  may reduce variation and unnecessary treatment.

The study had several important methodologic limitations. Some vignette details could have been interpreted differently by respondents. This could have impacted variation in the results. We simplified some of the laboratory elements in the vignettes and this may create ambiguity and variability in responses. We did not determine the total proportion of eligible providers at a given site that received the survey. The results may reflect selection bias in an unknown direction because our results do not represent all participating sites and may include a low proportion of eligible respondents. Finally, the study was not powered to detect differences by role.

Clinical vignettes identified specific opportunities for antibiotic stewardship efforts related to EOS. Targeting clinicians’ antibiotic decisions related to perspectives on culture negative sepsis, abnormal screening laboratory results, threshold of tolerance for respiratory signs, and thresholds of antibiotic response to antenatal maternal risk factors may benefit antibiotic stewardship efforts in NICUs. Stakeholders implementing antibiotic stewardship should prioritize these areas and/or implement customized vignette assessments to understand individual antibiotic use variation within their local context.

The authors thank Jeff Merrill, MD, Linda Lefrak, RN, MS, CNS, and the respondents from participating NICUs for their contributions to this study.