Pediatric Septic Shock Collaborative Improves Emergency Department Sepsis Care in Children

An advisory panel of nurses and physicians with experiential knowledge of pediatric sepsis QI activities and research developed interventions and metrics using best available evidence. They recruited pediatric ED participation from the American Academy of Pediatrics’ Section on Emergency Medicine; participation costs were $400 to $3800, depending on hospital size. Local implementation teams consisted of physician and nursing leadership, front-line clinicians, and other clinical staff, such as pharmacists and QI specialists. Two additional sites did not have the resources to submit data and thus were given access to PSSC materials and participated in webinars and were considered non–data-reporting sites. (Table 1) One-hour virtual meetings occurred monthly. Half-day in-person meetings occurred twice a year (with at least the site lead being required to attend) using the Institute for Healthcare Improvement’s Breakthrough Series Model for collaborative activities²⁰  and the Model for Improvement’s plan-do-study-act construct for local implementation.²¹  Sites had access to the PSSC’s educational materials, references, care bundles and clinical pathways, screening tools, and other resources via a password-protected Web site. Near-time sharing of novel implementation strategies through formal and informal communication as well as Web site interactions allowed the rapid spread of recently developed interventions to accelerate improvement.

Sites determined locally whether and in what order to implement each intervention, leading to site-specific variation based on previous sepsis work, local workflows and available resources; thus, a single time line of interventions is not reflective of the multisite work. In general, recruitment and onboarding began in November 2013, change packages were distributed in April 2014, most sites started implementation by November 2014, and, by April 2015, most interventions had been implemented.

Cases included a patient meeting any of the following criteria: (1) treatment (blood culture obtained and receipt of both parenteral antibiotics and at least 2 intravenous fluid boluses) plus any of the following: positive sepsis screen, ICU admission, lactate assessment, or vasoactive agent use; (2) sepsis order set use; (3) International Classification of Diseases codes for severe sepsis or septic shock. (Fig 1). The same criteria were used to review unplanned floor-to-ICU transfers within 12 hours from ED admission to ensure these patients were included. Patients who were transferred from an outside hospital were excluded if they had already received the majority of sepsis interventions (defined as antibiotics or ≥2 fluid boluses). Although the above strategy was recommended, sites varied in their ability to capture patients, resulting in denominator heterogeneity. Data were abstracted from the electronic health record (EHR) via a combination of automatic data abstraction and manual chart review then entered into a central data portal (with built-in logic checks) via manual entry or data upload. Although variables needed to calculate key metrics were required (eg, therapies and associated time stamps), reporting of secondary, more labor-intensive variables (eg laboratory results, inpatient therapies and severity of illness scores) was optional. (Supplemental Table 4).

Because early recognition is a key component of sepsis care, the PSSC recommended, and all sites implemented, a screening tool to assist in identifying patients with potential sepsis. They disseminated a previously used tool but did not mandate which tool was used (Fig 2). Seventeen sites had a protocol for evaluating patients who screened positive, and all had a standard sepsis order set. Some sites modified the PSSC tool, such as instituting a two-level screen with the initial screen including abnormal vital signs and the secondary screen requiring presence of a high-risk condition, altered mental status, or altered perfusion.²²  Approximately 24% of sites used a paper tool, whereas 76% incorporated screening into their EHR; 39% implemented screening at triage and 61% throughout the ED visit. Responses to a positive screen involved a rapid bedside clinical assessment or huddle (at minimum, the ED physician and bedside nurse to assess the patient’s perfusion, mental status, and pertinent medical history to determine if sepsis care would be initiated). Sites could determine the location of the assessment and subsequent care (eg, regular room versus resuscitation suite), composition of the team (9 sites used a dedicated sepsis team), and associated logistic workflows.

Bundle recommendations included initiation of a rapid first bolus within 20 minutes, third bolus (if needed) within 60 minutes, and antibiotics within 60 minutes of recognition of suspected septic shock. Eleven sites had a standardized antibiotic protocol. Additional recommendations addressed standardization of advanced interventions, such as medications for intubation and vasoactive agents. Sites accomplished rapid fluid delivery by using pressure bags, push-pull methodology, or rapid infusers. Six sites had a protocol for physical resources for the sepsis pathway, such as “resource carts” or protocols with equipment needed to accomplish the first hour of care, and most employed tailored clinical pathways supported by order sets. Nine sites provided care team feedback on performance meeting specific metrics of care.

A library of educational tools was shared with and expanded on by individual sites, including relevant scientific literature, staff education materials, and narrated slides. Several sites then developed and shared more in-depth materials, including a sepsis identification aid for badge cards and a comprehensive electronic education module with pre- and postintervention assessments to help sites determine where to target educational efforts.²³ 

Although no specific EHR tools were recommended, learning sessions for informatics representatives from various sites were conducted to improve on the creation and adoption of clinical decision support. Individual sites created their own EHR-specific tools, which were then shared with others, including screening alerts, indicators on the EHR track board, electronic order sets, and EHR-driven sepsis timers. Nine sites had dedicated, ED-based EHR support to facilitate implementation.

Institutional review board approval and/or waiver was obtained centrally and locally.

Local and aggregate process, outcome, and balancing metrics were studied in real time via statistical process control (SPC) charts. Data were submitted to a central data portal (by using the Clinical Trials Management System at Baylor College of Medicine), and individual and aggregate charts were created then provided to sites.

Several metrics were time-dependent, necessitating determination of a “time zero,” which best represented the onset of sepsis. For patients with suspected sepsis, the true physiologic onset cannot be ascertained; thus, “time zero” represented the earliest opportunity for recognition. Many patients with sepsis in the ED are identifiable at triage.¹⁷  Therefore, as a proxy, triage heart rate time was used to represent time zero. If heart rate time was not documented, triage start time was used; if this was not documented, order set initiation time was used. A small proportion of patients developed physiologic signs of sepsis later during the ED visit. These were individually identified by each site, and this time stamp was used as time zero for time-bound metrics.

Process metrics included completion of an initial assessment within 20 minutes (defined as documentation of all vital signs, including pulse oximetry), initiation of a first fluid bolus within 20 minutes (20 mL/kg of normal saline or Lactated Ringer’s solution with an option to deliver smaller boluses if indicated), initiation of a third fluid bolus (if needed) within 60 minutes, and initiation of parenteral antibiotics within 60 minutes. Completion times were not captured because they were not reliably documented.

The goal for these metrics was 95% compliance and was tracked by using p-charts. However, times to interventions (which often precede p-chart improvement) can serve to motivate teams earlier and, thus, we developed median charts for all process metrics as well. Median charts (rather than x-bar or s-charts) were used for process and outcome metrics to account for extreme outliers often encountered in health care processes.

The primary outcome was 3- and 30-day aggregate mortality, defined as in-hospital death from any cause during the index admission. Secondary outcome metrics included hospital and ICU LOS, hours of ventilation, and hours on vasoactive agents. Ventilator hours included increases in chronic noninvasive ventilation settings and new invasive and noninvasive ventilation. Ventilator hours were calculated from time of intubation until extubation, or from initiation of higher noninvasive ventilation settings until return to baseline. If a patient had >1 modality, the total time on all modalities was aggregated. Vasoactive agent hours included continuous intravenous medications intended to support blood pressure or perfusion including dopamine, epinephrine, norepinephrine, dobutamine, milrinone, and vasopressin. For both ventilator hours and vasoactive agent hours, initiation within 24 hours of leaving the ED was included to account for patients who had care escalated shortly after admission.

Although the false-positive rate of the trigger tool was proposed as a balancing measure, this was not possible because of site-specific differences in the components of the screening tool, the intent to treat approach of the screening tool, and consequent lack of true gold standard for the definition of false-positives.

All SPC charts were analyzed for special cause by using the 8-point rule²⁴  followed by the aggregate point rule (APR).²⁵  The addition of the APR, which uses similar statistical probabilities to identify shifts earlier than the 8-point rule alone, is helpful in several situations common in health care in which applications of the 8-point rule are limited, such as when the time needed to satisfy the 8-point rule is cumbersome (eg, waiting for 8 months of datapoints) or in cases in which the outcome is a rare event that falls close to 0 (eg, a month with no mortality). It additionally accounts for distance from the centerline, which influences the probability that an event occurs by chance. Special cause and centerline determination were assessed after a baseline of 1 year (November 2013 to November 2014); this time frame was chosen to account for seasonal variation and was also the inflection point when most local sites had formed teams and initiated improvement efforts. Because of potential differences in sepsis case identification across sites, a subgroup analysis was planned; this included patients with presumed severe sepsis or septic shock requiring ICU care because this denominator was not subject to such definitional variation. Additionally, because 4 large sites had completed extensive sepsis-related improvement work before involvement in the PSSC, SPC charts were reanalyzed for all metrics after removing subjects from these sites.

From November 2013 to May 2016, 7192 patients were reported from 19 sites (Table 1). Fourteen sites (74%) reported data for at least 30 of 31 months. The ICU subgroup represented 47% of all patients (n = 3382) (Table 2).

Sites reported data process metrics and mortality with >98% capture of associated variables, such as timestamps for triage heart rate and critical interventions, including fluid bolus and antibiotic administration (Supplemental Table 4). Optional secondary variables, such as laboratory data and doses of vasoactive agents, were reported less reliably. Fifteen of 19 sites additionally reported data on LOS (representing 94% of overall encounters), and 4 of 19 sites reported data on ventilation and vasoactive agent use.

Over 31 months, improvements were seen in several key process metrics (Table 3). In the whole cohort, the proportion of patients receiving timely antibiotics (within 60 minutes) improved from 47% to 50% (Fig 3), with median time improving from 68 to 63 minutes. The proportion with timely documentation of vital signs improved from 74% to 78%. In the ICU subgroup, the proportion receiving timely antibiotics improved from 50% to 54% (Fig 3), whereas timely first fluid bolus (within 20 minutes) improved from 35% to 39% (Fig 4).

Thirty-day mortality decreased in the whole cohort (2.3% to 1.4%) (Fig 5) but not the ICU subgroup; 3-day mortality did not change over time. Other outcome measures, including hospital and ICU LOS, hours of vasoactive agents, and hours of ventilation, did not change (Table 3).

Similar improvement in sites newer to sepsis work was seen when removing the patients (n = 3955) from the 4 sites with previous sepsis interventions from the SPC analysis.

Over the course of the 3-year project, 19 sites collaboratively implemented or improved their sepsis screening, care pathways and education, with improvements seen in proportion of patients with timely interventions and an observed decrease in 30-day mortality in the whole cohort. Although the magnitude of the improvements in process metrics was smaller than initially targeted, consistent improvements over time in the recognition of sepsis and implementation of evidence-based care across a large multicenter cohort represents important change.

Several sites had previously developed robust sepsis identification and treatment algorithms (as much as 5 years earlier).^15,17,18  Because of previous local work, incremental improvement at these sites was challenging because sites were attempting to improve on an already well-functioning system with fewer low-investment opportunities for optimization. Baseline times for process measures were already close to the collaborative goals and were better than previously described,^15,17,18  making further time-based improvement difficult to discern. However, experienced sites likely contributed to successes overall because they mentored new teams and shared best practices. Analysis demonstrated that, on removal of these more mature sites, the improvement results remained similar and that even mature sites have yet to perform in a highly reliable manner and thus continue to have room for improvement.

This study intentionally encompassed a broadly defined sepsis population to improve on care across the spectrum of disease, including milder cases with early identification and treatment before deterioration occurred. This may have resulted in dilution of the cohort with patients with milder disease, in whom it can be especially challenging to identify improvements. Additionally, as screening processes improved, increasingly more cases were identified, which likely diluted the denominator with less sick patients who did not demand the same level of intense and rapid care as those severely ill. For example, initially, 53% of patients were in the ICU subgroup, but this decreased to 44% by the end of the study period. This could also explain the slight increase in time to third bolus within the whole cohort over time.

The magnitude of change of 30-day mortality was more than expected when considering the modest improvement in process metrics. There are several possible explanations. Dilution of the denominator, by inclusion of those with milder disease over time as early identification processes improved, could have contributed. This would explain why improvements in mortality were seen in the overall group (which theoretically became more diluted with less sick patients over time) but not in the ICU subgroup. Additionally, sites may have made other sepsis-related process changes that were not measured but influenced improvements in mortality.

These improvements did not require additional clinical resources nor novel therapies but instead relied on targeted attention to systems of care and the use of QI methodologies. This highlights the need to understand and use a global, bundled approach to sepsis care that aligns with newer guidelines.^8,9  In the current cost-constrained health care environment, this approach is welcomed because it allows for improvements in safety and outcomes without addition of costly adjunct clinical resources.

Data collection proved to be onerous. Because sepsis is an entity with an ambiguous definition, even among expert clinicians, and complex research definitions are not feasible for real-time QI initiatives, substantial effort was needed to capture all patients. Some sites defaulted to manual chart abstraction, whereas others took up to a year to develop a data pipeline by which patients could be automatically identified and variables electronically abstracted. Additionally, many sites only had the resources to abstract and report primary process and outcome metrics; thus, data on vasoactive agent and ventilation increases were less robust because these were optional variables. These required a clinical informaticist and a data analyst to create the coding infrastructure, and some sites found that deployment of their resources toward data collection delayed implementation of interventions. Despite challenges with data collection, this was the first study to define sepsis for case identification across a large QI collaborative; both manual and electronic abstraction from the EHR was proven to be feasible. Such a definition was previously a major gap in efforts to improve sepsis care, and these definitions have subsequently been adapted for other large sepsis collaboratives.²⁶ 

Limitations included the use of retrospective EHR data and incomplete reporting, especially from smaller sites that focused their limited QI resources on interventions rather than data gathering; both of these are common limitations in QI at both the local and collaborative level. There was case identification heterogeneity across sites and potentially over time, although attempts were made to mitigate this through subgroup analysis of an ICU cohort. Additionally, the APR is a statistical methodology for determination of special cause that is newer to health care applications and is less well tested in these settings. However, it is a valuable addition to the statistical analysis in this study because it overcomes some limitations in interpretation of our data. Specifically, it is helpful in situations in which there is a rare outcome and some data points are close to 0 (eg a month with 0 mortality) and accounts for distance from the centerline in determining if an event (datapoint) occurred by chance.

This novel QI collaborative, which was focused on improving systems of sepsis care across pediatric EDs, showed sustainable improvement in sepsis recognition and some management processes, demonstrating that a bundled QI-focused approach to sepsis management can be effective in improving care and potentially reducing mortality.

APR

aggregate point rule

ED

emergency department

EHR

electronic health record

LOS

length of stay

PSSC

Pediatric Septic Shock Collaborative

QI

quality improvement

SPC

statistical process control