Interventions to improve care team situation awareness (SA) are associated with reduced rates of unrecognized clinical deterioration in hospitalized children. By addressing themes from recent safety events and emerging corruptors to SA in our system, we aimed to decrease emergency transfers (ETs) to the ICU by 50% over 10 months.
An interdisciplinary team of physicians, nurses, respiratory therapists, and families convened to improve the original SA model for clinical deterioration and address communication inadequacies and evolving technology in our inpatient system. The key drivers included the establishment of a shared mental model, psychologically safe escalation, and efficient and effective SA tools. Novel interventions including the intentional inclusion of families and the interdisciplinary team in huddles, a mental model checklist, door signage, and an electronic health record SA navigator were evaluated via a time series analysis. Sequential inpatient-wide testing of the model allowed for iteration and consensus building across care teams and families. The primary outcome measure was ETs, defined as any ICU transfer in which the patient receives intubation, inotropes, or ≥3 fluid boluses within 1 hour.
The rate of ETs per 10 000 patient-days decreased from 1.34 to 0.41 during the study period. This coincided with special cause improvement in process measures, including risk recognition before medical response team activation and the use of tools to facilitate shared SA.
An innovative, proactive, and reliable process to predict, prevent, and respond to clinical deterioration was associated with a nearly 70% reduction in ETs.
Hospitalized children who require emergent life-sustaining care because of unrecognized clinical deterioration experience adverse outcomes, including increased mortality.1,2 Over the past two decades, substantial efforts focused on the detection and mitigation of clinical deterioration through rapid response systems have reduced this type of potentially preventable harm.3–12 However, the success of these efforts has been variable, and opportunities to reduce unrecognized deterioration remain.12
Our institution began systematic efforts to improve clinical team situation awareness (SA) in 2009. As defined by Endsley,13 SA exists in 3 levels: (1) perception of environmental elements with respect to time or space, (2) comprehension of their meaning, and (3) projection of their future status. Inadequate SA was a frequent etiology of inpatient safety events and unrecognized deterioration in our inpatient system.10 To focus specifically on deterioration, the team prospectively defined emergency transfers (ETs) as the transfer of patients from the acute care floor to the ICU where the patient received intubation, inotropes, or ≥3 fluid boluses in the first hour of ICU care or before transfer.10 Through structured SA interventions, including unit-based and inpatient system-wide huddles and robust and explicit action plans for at-risk patients, this original SA model was associated with a nearly 50% reduction in ETs.10 ETs have since been associated with an increased hospital length of stay, ICU length of stay, and mortality, leading to a heightened focus on prevention of these events.2
After the implementation of the original SA model, our institution experienced an increase in the volume and complexity of our patient population and hospital team structure. Concurrently, the robust expansion of available technology led to frequent changes in electronic health record (EHR) functionality, the introduction of a secure text messaging service (which frequently replaced face-to-face interaction), and concern regarding the potential adverse impact of alarm fatigue on the inpatient units.14–18 This institutional evolution allowed for the emergence of multiple “SA demons,” or corruptors of SA in complex systems in which there is a great deal of rapidly changing information that can be difficult to obtain.19 In 2018, a series of root cause analyses after a cluster of safety events revealed two etiologic themes: (1) inadequate SA for patients demonstrating signs of clinical deterioration, and (2) lack of a shared mental model among care providers due to inadequate psychological safety and communication. Psychological safety is defined as the ability to raise concerns and ask questions without fear of negative consequences for self-image, status, or career.20 Before developing an action plan, these themes were validated in conversations with clinical and operational leaders to ensure that subsequent improvement initiatives would be generalizable. An institutional refocus on safety then began, with a particular emphasis on mitigating the most prominent SA demons and fostering a psychologically safe culture to improve communication. Using innovative transformations in communication and EHR technology to redesign the SA model for clinical deterioration, we aimed to decrease ETs to the ICU by 50% over 10 months.
Setting and Context
Cincinnati Children’s Hospital Medical Center is an academic, quaternary-care, freestanding children’s hospital with >33 000 inpatient admissions and 6000 inpatient surgical procedures per year. Inpatient units are led by a dyad team of a nursing director and medical director. Transfer to the ICU occurs exclusively through the medical response team (MRT), which was introduced in 2006 and eliminated both formal and informal ICU consults.21 The MRT process involves defined activation criteria for high-acuity patients, and the response team includes an ICU physician or advanced practice provider, nurse, respiratory therapist, and institutional nursing leader, known as the manager of patient services, who is focused on system-level coordination of operational resources, patient placement, and safety. In addition to defined activation criteria, care team members and families are able to activate an MRT at any time to escalate concerns. A modified version of the Monaghan Pediatric Early Warning Score was introduced across all non-ICU inpatient units in 2007.22 The original SA model for clinical deterioration was introduced in 2010.10 This involved designating high-acuity patients outside of the ICU as “watchers,” defined as patients determined by the clinical team to be at risk for clinical deterioration. After designation as a watcher, care team members huddle at the bedside and establish an action plan, reassessment timeline, and MRT activation criteria. A watcher flag is entered into an EHR flowsheet, which populates a “W” flag in an “SA concern” column on patient lists. These patients are discussed at inpatient system-wide safety huddles 3 times daily.10
As a result of the findings from the root cause analyses in 2018, an SA steering committee was convened and led by two associate chiefs of staff (R.F. and C.M.W.) and a vice president of nursing (J.H.). The SA model team was then formed and co-led by a hospital medicine physician (T.S.), nursing leadership (M.S. and D.H.), and a manager of patient services (K.B.). . This interdisciplinary team included family members of children who have experienced frequent ICU transfers, physicians from the ICU and the emergency department, pediatric resident physicians, nurses, respiratory therapists, and quality outcomes specialists. This study qualified as exempt systems improvement which did not necessitate institutional review board review or exemption.
Interventions: SA Model Development and Testing Process
SA Model Innovation Development
Following the Model for Improvement, the SA model team developed a current state process map and modified failure modes and effects analysis using learnings from safety events and the clinical experience of the team members.10,23,24 To bring additional expertise and validate findings, team leaders conducted cognitive task analysis interviews with families, bedside nurses, charge nurses, resident physicians, and respiratory therapists.25 These interviews used open-ended, probing questions to elicit the successes, challenges, and barriers to caring for deteriorating patients.25 The team then constructed a key driver diagram (Fig 1) using team expert consensus, interview learnings, a review of pertinent literature, and partnership with the ICU, which had recently implemented an enhanced critical care SA system.26 Key drivers and proposed interventions were used to redesign the SA model for clinical deterioration (Fig 2). This new model was adapted frequently via plan-do-study-act cycles.24
The SA model team developed a plan to test this model across all non-ICU inpatient units in 6-week cycles from November 2019 to June 2020. Each cycle involved 4 phases: prework and planning, education, implementation, and feedback (Supplemental Fig 7). In addition to collecting summative qualitative feedback at the conclusion of each cycle, SA model team leadership collected feedback from patients and families, physicians, and nurses on the units two to three times per week during active testing to facilitate real-time learnings and iteration of interventions.
A critical aspect of the process was the creation of an SA learning network, which included the SA model team leadership, a family advocate, and representatives from each unit, which were cumulatively added to the network as testing progressed through the acute care areas. This allowed for consensus to be reached on adoption, adaptation, or abandonment of tested processes and tools before moving to the next testing cycle.
Introduction to the new model and education on SA began in July 2019, with presentations at several leadership, safety, and staff meetings. An online module for education was created for care team members and was assigned to all staff at the initiation of spread to their unit. The SA model team leadership presented the material at faculty and staff meetings for each division, and a virtual staff meeting video was created to facilitate remote learning. Laminated educational materials on SA, the SA model, and tools were provided for just-in-time review on the units. For staff who travel between units, including resource nurses, respiratory therapists, and pediatric residents, education was provided at the initiation of the first testing cycle via the online module and live presentations at staff meetings. All educational materials were updated throughout to reflect the changes made.
Interventions: Key Innovations to SA Model
The SA model team leveraged learnings from previous pediatric and cardiac ICU SA work at our institution to develop a mental model (M2) checklist (Fig 3). This checklist aimed to address the safety event theme which described lack of a shared mental model among care providers due to a lack of psychological safety and inadequate communication. This checklist is completed by the care team members and patient’s family at the conclusion of the SA huddle. It is an expectation that each individual is able to affirmatively answer each checklist item and, if this is not the case, questions or concerns are addressed. Coinciding with our work, the institution conducted a safety culture training, “Safer Together,” for all employees. This training was grounded in a foundation of psychological safety and conveyed that all staff must be empowered to look out for their own safety and the safety of others, speak up with concerns, listen when concerns are raised, and respond with grace and gratitude. A “head-heart-voice” visual emphasized that all staff should (1) know when to speak up, (2) feel they can safely speak up, and (3) have the skills to speak up and listen effectively. To align and integrate with this important and related initiative, the visual from this training was incorporated into the M2 checklist (Fig 3). All units chose to have the charge nurse facilitate discussion of this checklist because this individual is a consistent presence at all unit SA huddles.
Prior to the initiation of our work, a watcher door sign was in use on the gastroenterology, nephrology, and surgery unit as well as in the ICU. Using key experiences and learnings from those units, the SA model team created a watcher sign for all non-ICU inpatient care areas (Fig 4). This sign contains information on the primary clinical concern that initiated the SA huddle, shared mental model, current action plan, reassessment timeline, and MRT criteria. It also includes an escalation plan, detailing whom to call and how to contact that individual with questions or concerns. This facilitated family engagement, accountability of the care team to the established plan, and shared SA across the unit staff. To ensure privacy, no patient identifiers were included on the sign and detailed feedback was collected from families without significant privacy concerns identified. Several adaptations were made to this sign as the project progressed, including (1) changing the color from white to orange to increase visibility, (2) adding detailed reassessment and escalation information, (3) adding free space for patient-specific information that did not fall within the templated prompts, (4) adding the SA model and M2 checklist to the back of the sign for reference, and (5) developing prompts to describe the concepts of “SA huddle” and “watcher” to families.
An SA navigator was developed for the EHR (Epic, Verona, WI) and was released in February 2020 (Supplemental Fig 8). Within this navigator, an "SA Concern - Watcher - Criteria for MRT" order was developed. This order is entered by the physician or advanced practice provider when a patient is designated a watcher, and contains the MRT criteria and expected time to resolution of watcher status. The original SA model included the development of an SA concern note in the EHR, which is expected to be filed as an acute event note within two hours of initiation of watcher status. As part of our work, this note was updated to highlight the shared mental model (formerly “assessment”), plan, expected outcome, reassessment timeline, and MRT criteria by moving them from the bottom of the note to the top. The "SA Concern - Watcher - Criteria for MRT" order prepopulates the MRT criteria directly into the note. Detailed questions about the SA huddle, individuals present, and completion of the M2 checklist were added. In addition to the order and note, the navigator contains links to relevant SA flowsheets, educational materials, and a link to a voluntary Research Electronic Data Capture survey to provide feedback to the SA model team on any aspect of the SA model and tools.27
Study of the Interventions
ICU transfers are reviewed and entered into an institutional database by safety leaders who determine if the transfer meets the criteria for an ET. Data on watcher patients, including information from the SA concern notes, were abstracted via manual chart review by two research nurses and a resident physician (B.M.). Qualitative feedback was collected from unit staff multiple times per week on units conducting active testing and weekly from families whose children experienced the SA model process.
The primary outcome measure was the rate of ETs per 10 000 patient-days. This is a validated, proximal measure for unrecognized clinical deterioration, defined as any ICU transfer in which the patient receives intubation, inotropes, or ≥3 fluid boluses within 1 hour.2,10 Process measures focused on the early recognition and mitigation of deterioration included the percent of patients identified as a watcher at least one hour before MRT or code activation, percent of watcher patients with an SA note filed within two hours of watcher status initiation (the exemplary practice), and percent of SA concern notes that contained objective, measurable, vital sign-based MRT criteria. Balancing measures to ensure that our project did not unnecessarily overburden our safety processes and critical care resources included the rate of watcher patients and the rate of MRTs per 1000 patient-days.
Metrics were tracked via a prospective time series analysis by using statistical process control charts.28 ETs were tracked on a u-chart. All process measures were tracked on p-charts. Balancing measures were also tracked on u-charts. Established rules for Shewhart control charts were used to determine if observed changes were due to special cause variation.29
During the study period, the average rate of ETs improved from 1.34 to 0.41 events per 10 000 patient-days, a nearly 70% reduction (Fig 5). This improvement coincided with special cause signal in all 3 process measures (Fig 6). The rates of watcher patients and MRTs revealed only common cause variation during the study period.
Qualitative learnings from both families and staff were critical to generating engagement in the process and successful adoption of the model and tools. Families consistently shared that their primary priority was to feel “heard” and be updated on the status of the child frequently, and this was successfully facilitated by inclusion in huddles. Key themes from staff included (1) an emphasis on reducing redundancy in the workflow and streamlining documentation, (2) the inclusion of consideration of a broad differential diagnosis and available unit resources on the M2 checklist, (3) the highlighting of the primary clinical concern and MRT criteria on the watcher sign, and (4) the availability of just-in-time educational resources on the unit.
Unrecognized clinical deterioration poses a significant threat to the safety of hospitalized children.2,10,30 With a focus on enhanced communication, psychological safety, and efficient and effective SA tools, we applied innovative processes to a foundational SA system and observed an associated nearly 70% reduction in ETs. This is consistent with previous studies, in which researchers have associated improved SA with decreased adverse events of this type.10,11 Our institution was poised to capitalize on numerous concomitant factors, including (1) the necessity to adapt to a system and patient population with increasing acuity, complexity, and volume, (2) key learnings from safety events with overlapping themes, including inadequate SA and disruptions to communication, and (3) the need to address critical SA demons. Our initiative is the first to focus on psychological safety as a key driver, which was primarily addressed through a novel M2 checklist. Furthermore, we introduced first-of-their-kind watcher door signs and EHR enhancements to address key SA demons. With the support of institutional leadership, an interdisciplinary team of care team members and patient families transformed the SA model to identify and mitigate clinical deterioration and provide safe care to our high-acuity patients outside of the ICU.
This time series analysis reveals the importance of reliability science and sustainability, which are critical to improvement initiatives.31 As denoted in Figs 5 and 6, the announcement of our project and dissemination of SA education began on a large scale throughout our inpatient system in July 2019. This also coincided with the timing of the aforementioned “Safer Together” training. This education and training were critical to the success of our work but represented low reliability interventions. It is common for interventions of this type to lead to quick improvement, but these interventions are much less likely to lead to the desired special cause variation and exceedingly unlikely to lead to a sustained improvement. We, therefore, designed and tested interventions with a higher level of reliability, including the M2 checklist, watcher sign, and EHR SA navigator. We believe these interventions were critical to our ability to both reach special cause improvement and sustain the impact of our work.24,31
Through ongoing qualitative feedback collection, the primary drivers of improvement in SA, and thus, reduction in ETs, emerged as (1) psychological safety to escalate concerns, facilitated through the M2 checklist, (2) unit-wide SA, accountability, and family engagement, facilitated by the watcher door sign, and (3) streamlined documentation processes to allow for a shift of focus from a task-oriented mindset to direct patient care. It was also noted that the M2 checklist was spontaneously adopted by staff to use in other types of huddles, including at MRTs. With this, the profound importance of psychological safety in our system was further emphasized.
To succeed with an inpatient system-wide initiative, it was critical for the SA model team leadership to use the guiding principles of change management for the entirety of the study period (Supplemental Fig 7).32 In addition to active engagement on the units from the prework phase to the feedback phase, the leadership group maintained an active presence at institutional venues, presenting project updates and collecting feedback from a variety of stakeholders, including families. Updated processes and tools were shared with the groups that provided input to demonstrate that this feedback was acknowledged and valued. In this way, institutional culture played a critical role in project success, as has previously been demonstrated in the field of patient safety.33 The alignment and integration with the “Safer Together” training and emphasis on a culture of safety from our senior executive team served to emphasize the importance of the changes to the SA model and encourage a positive shift in the landscape of our institution.
This work was performed at a single, large, quaternary-care children’s hospital with specialized quality improvement and bioinformatics resources that supported successful implementation. The concurrent institutional refocus on a culture of safety, which ran in parallel to our work, certainly influenced our success as well. These factors may limit generalizability to other centers. However, the use of ETs rather than more site-specific metrics, such as unplanned transfers, may facilitate feasible measurement at alternative sites as well as expansion of this and similar work to multicenter collaboratives. We measured the successful adoption of processes and tools to improve SA as process measures and used early recognition of deterioration as the outcome measure; however, we did not directly measure SA as part of this work. As is common in quality improvement, we performed a number of interventions over time, and we cannot say if any individual interventions or the set of interventions were causal.
The application of innovations in communication and technology to a proactive and reliable process to predict, prevent, and respond to clinical deterioration was associated with a nearly 70% reduction in ETs. Ongoing SA efforts are focused on sustainability and further improvement in the inpatient areas as well as the customization of the SA model to apply to other sites of care, including the perioperative areas, psychiatry, ambulatory, pharmacy, radiology, and the laboratories. Engaging these sites of care will also facilitate alignment of SA processes as patients move throughout our system.
We thank Pavan Chundi, Carol Frese, Deborah Hacker, and Katie Simon for their expertise in data collection and analysis, Ken Tegtmeyer for his assistance with the graphic design of the SA model, Matthew Nelson and the Cincinnati Children’s Media Laboratory for the illustration of the SA model graphic, and Philip Hagedorn for his assistance with the EHR enhancements associated with this work.
Dr Sosa conceptualized and designed the study, designed the data collection instruments, collected data, coordinated and supervised data collection, conducted the initial analyses, and drafted the initial manuscript; Drs Sitterding and Dewan conceptualized and designed the study, supervised the analysis and interpretation of data, and assisted in the drafting of the initial manuscript; Ms Coleman, Ms Seger, and Dr Maddock conceptualized and designed the study, designed the data collection instruments, collected data, and conducted the initial analyses; Ms Bedinghaus and Dr Hawkins conceptualized and designed the study; Ms Hausfeld and Drs Falcone, Brady, Simmons, and White conceptualized and designed the study and supervised the analysis and interpretation of data; and all authors reviewed and revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.
FUNDING: The Center for Clinical and Translational Science and Training and access to the Research Electronic Data Capture Database at the University of Cincinnati is funded by the National Institutes of Health Clinical and Translational Science Award Program, grant 2UL1TR001425-05A. The Clinical and Translational Science Award Program is led by the National Institute of Health’s National Center for Advancing Translational Sciences. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.