Early Identification of Severe Sepsis in Pediatric Patients Using an Electronic Alert System Free

This quality improvement initiative took place at a tertiary, academic children’s hospital from January 2016 to July 2020, in which we had an average of 160 pediatric sepsis cases and 19 pediatric severe sepsis cases per year. Bedside patient care teams typically include nurses, trainees (students, residents, fellows), attending physicians, and advanced practice providers.

Our interdisciplinary team consisted of information technology, clinical informatics, quality management, pediatric hospitalists, intensivists, and emergency medicine physicians. To build the NT electronic severe sepsis screening tool, we designed a decision tree (Supplemental Fig 1) using systemic inflammatory response syndrome (SIRS) criteria by age (Supplemental Table 1) per the International Pediatric Sepsis consensus statement.³  We included patients younger than 18 years of age and excluded patients in the newborn nursery and neonatal ICU.

The EMR continuously monitors all patients’ charts, including vital signs, laboratory values, and nursing documentation. If a patient met SIRS criteria (per data captured in the EMR), an automatic alert triggers an interruptive alert notification for nurses in real time through our EMR asking if the patient’s medical status is suggestive of a new infection. The nurse then discusses the alert and the patient’s status with the patient’s primary licensed independent practitioner (LIP, which includes attending physicians, resident physicians, fellow physicians, nurse practitioners, and physician assistants) to determine if there is a concern for sepsis. If sepsis is suspected, the nurse answers the alert with an affirmative response, which leads to the pediatric sepsis multiphase powerplan (MPPP) being offered as a suggested powerplan for LIPs to order in the patient’s EMR (Supplemental Fig 1).

At the time of the NT alert’s implementation, the bedside nurse was not required to answer this alert. Although we initially noted a mean treatment bundle compliance improvement from 25% in 2014 to 46% in 2015, after implementing the NT alert, this improvement was not sustained and decreased to 23%, with a mean compliance of 37% from quarter 2 of 2014 through quarter 2 of 2016. To address this, we required the nurse to answer the alert as our first Plan, Do, Check, Act (PDCA) intervention in June 2016 (Fig 1). Although the nurse answered the alert with whether the medical team was concerned about sepsis and which LIP was notified, the nurse only suggested to the LIP that the sepsis bundle be carried out.

The second PDCA intervention was to implement the PT severe sepsis automated, electronic alert (Fig 1). We derived the organ dysfunction criteria per the International Pediatric Sepsis Consensus statement.³  Data documented in the patient’s chart, including nursing documentation and laboratory results, were used to trigger the alert (Supplemental Table 2). Before the PT alert was implemented, we created and implemented the pediatric sepsis MPPP and enabled the first dose of broad-spectrum antibiotics, which could be overridden by our pharmacy dispensing system (BD Pyxis) to facilitate timely initiation of treatment.

Using the established organ dysfunction criteria, a testing plan was created to test all data elements across multiple scenarios in a mock domain of the EMR. A testing plan included each data element as objectives, and the expected outcome was documented. For the initial round of testing, each clinical value established as a criterion was tested to make sure it triggered the rule. Groupings of results were defined to delineate multisystem organ dysfunction. Timing of results were also considered when testing because the rules were designed to look back at clinical events per respective look-back times (Supplemental Table 2). Another round of testing was completed to coincide with the LIP’s response to the alert. If any outcome was unexpected, changes were made to the rule and additional testing was done until all expected outcomes were met. The PT alert went live in July 2018.

When the patient met SIRS plus organ dysfunction criteria, the PT alert was delivered to the provider via a page to the pediatric ED attending for ED patients and the pediatric ICU attending physician for inpatients. These groups of providers were paged because they were the only groups with 24/7 in-house coverage. A simultaneous electronic PT alert appeared on the EMR screen on opening the chart (Supplemental Fig 2), viewable by all LIPs. This interruptive alert displayed the criteria met by the patient and asked the provider if the patient may have severe sepsis. The alert forced the provider to choose 1 of the following options: (1) agree with severe sepsis, (2) disagree with severe sepsis, (3) consider patient with restricted and/or comfort care, or (4) “ask me later.” Another option allowed a consulting provider to be notified of the alert but did not force the provider to respond before entering the patient’s chart.

The attending who received the page notified the patient’s primary team to assess the patient. The team documented its assessment of a patient’s condition using the Situation, Background, Assessment, Recommendation format, including whether it agrees or disagrees that the patient may have severe sepsis, the reason for this assessment, and the next steps in the plan for “assessment” and “recommendation,” respectively (Fig 2). If the PT alert was not answered within 30 minutes, the alert continued to appear, and the page notification was resent.

If a LIP agreed with the PT alert, it ceased to trigger again for 72 hours; if the provider disagreed, the alert ceased for 12 hours. If the provider selected “comfort care/refusal of treatment,” the alert did not fire again. Selecting “ask me later” allowed the provider to investigate the chart to gather more information, and the alert fired again on exiting the chart and continued to refire until another response was made. Once the respective suppression time expired, a new alert fired if new criteria were met.

After the alert was created and tested by information technology, we trained pediatric inpatient LIPs in the PT alert’s function and concepts. There was monthly communication with relevant parties focusing on alert and treatment bundle compliance and performance improvement. Quality management also reviewed all patients coded for severe sepsis and septic shock to evaluate the compliance with collective and individual components of the treatment bundle.

Definitions for severe/shock sepsis were consistent across reporting venues, as defined by the Center for Medicaid/Medicare Services and the NYSDOH for International Classification of Diseases, 10th edition (ICD-10). These ICD-10 codes included severe sepsis without shock (R65.20) and severe sepsis with septic shock (R65.21). Using the Vizient clinical database (a health care analytics database of administrative data and patient outcomes with the purposes of benchmarking and performance improvement), the study population was further defined as patient discharges with either severe sepsis or septic shock and younger than age 18 years (excluding all neonatal service-line patients and patients with an ICD-10 code for do not resuscitate/do not intubate noted as present on admission). Individual severe sepsis alerts were routed to quality management, and a comparative analysis was performed. This analysis confirmed that the physician-targeted alert was triggered on the respective patient when he or she met criteria, with a confirmatory physician progress note stating severe sepsis and/or septic shock was present and what treatment was initiated. Also, this database was used to validate that the real-time reviews had not missed any patients.

In January 2019, the NYSDOH released updated Adult and Pediatric Data Dictionaries for sepsis, in which septic shock was more clearly defined in pediatrics. At this point, we decided against adding another alert specific to septic shock because our current system captured all the septic shock patients. Also, the treatment bundles for pediatric septic shock do not vary from pediatric severe sepsis at our institution.

With advances in clinical decision support systems and their increasing use for sepsis early detection and treatment,^12–16  “alert fatigue” is a challenge faced by many organizations. Excessive alerts may cause clinicians to bypass or override important alerts.¹⁷  To avoid excessive alerts, we adjusted the cease-fire time for accepted positive severe sepsis alerts from 72 hours to 120 hours in September 2019. Also, in reviewing alerts for causation, we discovered that apparent erroneous blood pressure (BP) recordings (eg, from a crying child, movement) on the previous set of vitals triggered many alerts, leading to false-positive alerts. To address this special cause variation, we instituted staff education on the importance of accuracy, with the goal of reducing the number of alerts and therefore further reducing alert fatigue.

One of the issues we noted with this new phase was that patients with insidious declines in BP (where decline was <20% between 2 BP measurements), the EMR did not identify this as a criterion for cardiovascular compromise and did not trigger the severe sepsis alert. This led to a few false negatives; this was a population we needed to find a better way to identify. As a solution, we changed the look-back time from previous BP measurement to the BP measurements over the past 5 hours to better capture these patients.

IT noted that LIPs often avoided answering the EMR notification by selecting “ask me later” every time they entered or exited the electronic chart of a patient who met criteria for potential severe sepsis. This improved after providers were educated on the importance of responding to alerts and were mandated to discuss each case with the attending.

In February 2020, information technology created a dashboard, a centrally located timeline within the patient’s EMR with the data that contributed to meeting criteria for severe sepsis and the treatment bundle components. This visual guide readily displayed for the clinician what treatment bundle elements had been done, when they were completed, and highlighted what was yet to be done (Fig 3). All providers have access to the dashboard; however, the attending who received the severe sepsis alert page and the respective patient’s primary team were responsible for ensuring the treatment bundle components were met. Providers could review the criteria that caused the alert to trigger and respond directly to the interruptive alert via the dashboard for their convenience.

To study the outcomes of mandating nursing alert response to the already implemented NT severe sepsis alert and the implementation of the PT severe sepsis alert, we selected compliance of sepsis treatment bundle elements with severe sepsis patients as mandated by our state’s Department of Health as the primary measure, which had a goal of 60%. PT alert response compliance was the secondary measure, with goal response rate of 90%.

Compliance with treatment bundle elements was defined as obtaining a blood culture before antibiotic administration, the timely administration of broad-spectrum antibiotics within 1 hour of recognition of severe sepsis, and the timely administration of at least 20 mL/kg of intravenous fluids within 1 hour. We defined response compliance as any response to the alert, except for selecting “ask me later.”

Control charts were used to help determine when a new intervention was needed and to assess the effect of the intervention on the metric being measured. Unpaired t tests were used to determine significant differences of pre- and postintervention results. These analyses were conducted using the JMP software for statistical analyses. Confidence intervals were defined at 95% for comparison studies.

Our local institutional review board deemed this project as quality improvement–exempt. This study followed the Standards for Quality Improvement Reporting Excellence 2.0 reporting guidelines.¹⁸ 

Baseline pediatric severe sepsis treatment 1-hour bundle compliance was 37% from quarter 2 of 2014 through quarter 2 of 2016 (n = 41 severe sepsis cases). After requiring nursing response to the NT alert and the implementation of the PT alert, annual 1-hour bundle compliance improved to 50% in 2017 (n = 20), 61% in 2018 (n = 18), 65% in 2019 (n = 17), and 69% in 2020 (n = 13). Because of low monthly patient case volumes, analyses were grouped into 12-month intervals, showing statistically significant improvement between the baseline and postimplementation periods (P = .006).

Our PT alert response compliance was 67% in the 6 months after alert implementation, with a goal of 90%. To further improve PT alert response compliance, the BP rule was amended to a 5-hour look-back and expanded from 20% decline to 23% decline in BP (determined after case analyses noted high false-positive rates). In addition, residents were educated regarding appropriate alert response process, nursing was educated regarding accurate vital sign documentation, and the EMR sepsis dashboard (Fig 3) was implemented. The baseline PT average alert response was 74% between July 2018 and November 2019. These targeted interventions resulted in a centerline shift to 87%. December 2019 through December 2020 achieved the goal of 91% within the most recent 6-month period (Fig 4). There was statistically significant improvement in PT alert response compliance after the implementation of the electronic dashboard in quarter 1 of 2020 (P = .002). Review of data from the Vizient clinical database confirmed that the PT alert did not miss any pediatric patients with severe sepsis or septic shock.

By requiring nursing alert response to the already implemented NT severe sepsis alert and the implementation of the PT severe sepsis alert, our bundle compliance rose to 69% in 2020. This multidisciplinary alert system, coupled with required alert response fields, facilitated sustained improvement in severe sepsis treatment bundle compliance. We also noted continued increase in the response rate to the PT alert with appropriate targeted education, frequent compliance monitoring, and repeated PDCA cycles.

Early detection of severe sepsis and implementation of early evidence-based therapies improve outcomes and decrease sepsis-related mortality.^11,19  With the statewide mandate of bundles aimed at early recognition and timely management of sepsis, along with the implementation of our electronic sepsis screening process, our mortality rates trended at or below state comparisons. Using a bundled approach in the initial treatment of pediatric sepsis aligns with recommendations from the American College of Critical Care Medicine⁶  and Surviving Sepsis Campaign.⁷  Others also noted favorable outcomes when using an electronic, automated pediatric sepsis screening process.^12,20–23  Our alert system uniquely comprises an NT alert and PT alert, promoting multidisciplinary communication and approach to early pediatric severe sepsis assessment and management.

As a single-center study, the generalizability of the results may be limited, although similar findings were noted across the state.¹¹  Also, our study sample excluded patients with sepsis in the newborn nursery and neonatal ICU.

Throughout testing and planning, we discovered several limitations. One parameter described in the Goldstein et al article to determine cardiovascular dysfunction is the “need for vasoactive drug to maintain BP in normal range.”³  We decided not to use the ordering of vasoactive agents as a criterion because not all medications are administered at the time the order is placed in the EMR. We also decided against the inclusion of administering vasoactive drugs as a criterion in the electronic alert because the use of these medications in a critically ill patient is not specific to sepsis alone.

There was a limitation in the determination of neurologic dysfunction, which Goldstein et al defined as an “acute change in mental status with a decrease in Glasgow Coma Score greater than or equal to 3 points from abnormal baseline.”³  The behavior assessment of the pediatric early warning score was used instead because this is a frequent assessment done by nursing.

Severe sepsis continues to be a cause of mortality in the pediatric population. Implementing a biphasic, multidisciplinary electronic severe sepsis screen was associated with improved severe sepsis treatment bundle compliance as mandated by the state. Our interdisciplinary team used the FOCUS (focus, organize, clarify, understand, select) PDCA methodology to impact change, which included ongoing, real-time education with simultaneous compliance monitoring. Our focus has now shifted to developing an electronic tool applicable to the neonatal ICU and the newborn nursery.

The authors thank Margaret Parker, MD, MCCM, for her helpful contributions.