OBJECTIVES:

Event debriefing has established benefit, but its adoption is poorly characterized among pediatric ward providers. To improve patient safety, our hospital restructured its debriefing process for ward deterioration events culminating in ICU transfer. The aim of this study was to describe this process’ implementation.

METHODS:

In the restructured process, multidisciplinary ward providers are expected to debrief all ICU transfers. We conducted a multimethod analysis using facilitative guides completed by debriefing participants. Monthly debriefing completion served as an adoption metric.

RESULTS:

Between March 2019 and February 2020, providers across 9 wards performed debriefing for 134 of 312 PICU transfers (43%). Bedside nurses participated most frequently (117 debriefings [87%]). There was no significant difference in debriefing by unit, acuity, season, or nurse staffing. Compared with units fully staffed by rotational frontline clinicians (FLCs; eg, resident physicians), units with dedicated FLCs whose responsibilities are primarily limited to that unit (eg, oncology hospitalists) completed significantly more monthly debriefings (average [SD] 57% [30%] vs 33% [28%] of PICU transfers; P = .004). FLC participation was also higher on these units (50% of debriefings [37%] vs 24% [37%]; P = .014). Through qualitative analysis, we identified distinct debriefing themes, with teaming activities such as communication cited most often.

CONCLUSIONS:

Implementation of a multidisciplinary debriefing process for ward deterioration events culminating in ICU transfer was associated with differential adoption across providers and FLC staffing models but not acuity or nurse staffing. Teaming activities were a debriefing priority. Future study will assess patient safety outcomes.

Facilitated review of deterioration events through event debriefing is an important behavior of effective health care teams that can improve individual and team performance, as well as patient outcomes.13  Despite established benefit, event debriefing adoption is poorly characterized among pediatric ward providers, who play pivotal roles in identifying patients at risk for deterioration, maintaining situation awareness, mitigating deterioration, and escalating care to the ICU appropriately.4  These steps are critical to prevent adverse patient outcomes, including in-hospital mortality and prolonged length of stay.5 

In 2019, our hospital restructured its process for reviewing ward deterioration events culminating in ICU transfer to improve our ability to identify, mitigate, escalate,4  and ultimately prevent future clinical deterioration outside the ICU. In its previous state, this process consisted of a multidisciplinary conference organized by PICU leaders in the weeks after any severe deterioration event, such as cardiopulmonary arrest and/or acute respiratory compromise (defined as absent, agonal, or inadequate respirations requiring emergency assisted ventilation).6  Organizers invited ward providers to attend this conference and participate in an ICU-led event debrief. However, this process was limited by its focus on severe events, the delay between the actual event and the debriefing, and the structure that limited ward ownership. Our hospital observed an opportunity to improve safety learning by targeting these limitations in a restructured debriefing process.

The primary aim of this study was to describe the implementation of the new debriefing process. Specific process changes included (1) an immediate debriefing stage to capture a broader number of events and reduce the delay to debriefing; (2) a transition of process ownership to ward leaders; and (3) an option of delayed debriefing for any event culminating in ICU transfer, regardless of severity. We used the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework79  to guide our multimethod evaluation of the restructured process. To our knowledge, no previous researchers have described a hospital-wide event debriefing process aimed at pediatric ward providers.

This retrospective, observational study was conducted at an urban, tertiary care children’s hospital with ∼560 inpatient beds, including a 70-bed PICU, a 100-bed NICU, and a 32-bed cardiac ICU. Roughly 300 beds compose our general medical and surgical wards. Each ward is organized by a nurse manager and a medical director, along with various nurse leaders. Ward clinicians providing direct patient care (referred to herein as ward providers) vary by unit but include a permanent group of nursing staff. Frontline clinicians (FLCs) include advanced practice providers, resident physicians, and pediatric hospitalists serving in nonsupervisory roles. FLCs provide in-hospital care 24 hours a day. Some units are fully staffed by rotational FLCs (eg, resident physicians) who rotate across units at varying intervals (eg, 1–4 weeks). Other units have a staffing segment of dedicated FLCs whose clinical responsibilities are primarily limited to that unit (eg, oncology nurse practitioners). However, all units use rotational FLC staffing to varying degrees.

When ward clinical deterioration takes place, providers can activate the medical emergency team, a dedicated team of PICU clinicians (fellow physician, nurse, and respiratory therapist) who have 30 minutes to respond. After activation of this team, patients may be transferred to an ICU or may remain on the ward. For urgent events, ward providers may activate an immediate-response code blue team.10  In March 2019, our hospital began stepwise implementation of the new debriefing process across wards served by the medical emergency team.

Our hospital’s institutional review board reviewed this study and determined it does not meet criteria for human subjects research according to 45 Code of Federal Regulations 46.102(d).

The expectation of the restructured debriefing process is that all ward deterioration events culminating in ICU transfer will trigger an immediate event review after ICU transfer (Fig 1). Tier I debriefings are immediate reviews organized on an ad hoc basis after an ICU transfer. The vast majority of ICU transfers are to the PICU (>99% of all transfers). Transfers to the neonatal or cardiac ICU rarely occur but are outside the scope of this study.

FIGURE 1

Overview of debriefing stages. aCharge nurse or other unit nurse leader. bAny ward providers involved in clinical deterioration event. cNurse leader in clinical care, quality, or safety who provides longitudinal oversight of unit’s debriefing process. dLeaders (K.E.P., R.M.S.) of the hospital’s resuscitation committee.

FIGURE 1

Overview of debriefing stages. aCharge nurse or other unit nurse leader. bAny ward providers involved in clinical deterioration event. cNurse leader in clinical care, quality, or safety who provides longitudinal oversight of unit’s debriefing process. dLeaders (K.E.P., R.M.S.) of the hospital’s resuscitation committee.

Close modal

During the Tier I debriefing, participants may request additional debriefing (Fig 1). This triggers a notification to the unit’s “code champion”: a nurse leader in clinical care, quality, or safety who provides longitudinal oversight of the unit’s debriefing process. Once additional debriefing is requested, the code champion is notified to organize a Tier II debrief, a delayed debrief involving comprehensive event review. However, all events of acute respiratory compromise,6  cardiopulmonary arrest, and/or emergency transfer to the PICU5  undergo Tier II debriefing (Fig 1). Notably, the Tier II process is outside the scope of this report.

Ward nursing leaders (eg, charge nurses or code champions) facilitate Tier I debriefings. There is no required composition of participants, but multidisciplinary participation is encouraged. In addition to the facilitator, debriefings must include at least 1 ward provider (eg, bedside nurse, FLC) involved in the deterioration event. The facilitator organizes the ad hoc debriefing group via bedside communication and/or through our hospital’s paging system (Fig 1, Supplemental Information). ICU providers do not participate in Tier I debriefings, which target ward providers. However, if activated, Tier II debriefing involves participants from the ICU.

The goal timing for Tier I debriefing is by the end of the index clinical shift (ie, within 12 hours) to capture immediate feedback from providers involved in the event. If indicated, the unit code champion is notified about a Tier II debrief within 72 hours of the index event after review by leaders of the hospital’s resuscitation committee (K.E.P. and R.M.S.) (Fig 1).

Ward providers perform Tier I debriefing using a standardized REDCap11  survey as a facilitative guide. The guide contains both discrete (ie, “please check all that apply”) and free-response items (ie, “other,” which, if selected, prompts the participants to describe additional details) (Supplemental Information). Participants are asked about signs of deterioration and whether the patient was identified as a “watcher”: a situation awareness activation system our hospital uses to identify ward patients at risk for clinical deterioration.12,13  The guide prompts identification of other patients at risk for deterioration and evaluation of ward team performance surrounding the event. The guide also uses branching logic based on participant responses. For example, if participants report the patient was not identified as a watcher, the guide prompts a free-response question about this decision.

Multiple hospital stakeholder groups contributed to the design and implementation of this process. These included representatives from the section of hospital medicine, resuscitation committee, patient safety team, safety and clinical operations council, general pediatrics residency program, department of respiratory therapy, and Center for Healthcare Quality and Analytics. Senior operational leaders advised stakeholder engagement, facilitating awareness among ward leaders and across provider forums. As an example, stakeholders (M.M.G., K.E.P., and R.M.S.) conducted informational sessions at unit governance meetings and resident conferences to create awareness. Before local implementation, reminders were sent to ward leaders, outlining process expectations and offering support.

To guide our analysis, we used the RE-AIM framework, a tool for planning or evaluating a new process (Table 1).79  We used a multimethod analysis to assess process reach, adoption, and implementation patterns. Qualitative methods allowed us to explore contextual factors of implementation.7 

TABLE 1

Applying the RE-AIM Framework

RE-AIM DimensionOverviewOutcome Assessed in this Study
Reach Who was the new process intended to benefit, and who actually participated? Which ward providers participated in event debriefing? 
Effectiveness What was the intended outcome, and what was the likelihood of negative outcomes? n/a 
Adoption Where was the process applied? Which wards participated in event debriefing? How often did wards complete event debriefing for deterioration events culminating in PICU transfer? What factors (eg, staffing, season, acuity) were associated with event debriefing? 
Implementation How was the process delivered or adapted? How much did it cost, and why did the results come about? How did ward providers use the event debriefing process? What themes and priorities were identified during debriefing? 
Maintenance How long were the process and/or the results sustained? n/a 
RE-AIM DimensionOverviewOutcome Assessed in this Study
Reach Who was the new process intended to benefit, and who actually participated? Which ward providers participated in event debriefing? 
Effectiveness What was the intended outcome, and what was the likelihood of negative outcomes? n/a 
Adoption Where was the process applied? Which wards participated in event debriefing? How often did wards complete event debriefing for deterioration events culminating in PICU transfer? What factors (eg, staffing, season, acuity) were associated with event debriefing? 
Implementation How was the process delivered or adapted? How much did it cost, and why did the results come about? How did ward providers use the event debriefing process? What themes and priorities were identified during debriefing? 
Maintenance How long were the process and/or the results sustained? n/a 

Sources: Holtrop et al, 20187 ; Proctor et al, 20118 ; Gaglio et al, 2013.9  n/a, not applicable.

We analyzed completed Tier I guides associated with PICU transfers. We mapped Tier I guides to a list of all PICU transfers occurring after process implementation to measure debriefing completion as an adoption metric. We used chart review to resolve discrepancies.

For each unit, we defined the implementation date as the first Tier I event date. We normalized all Tier I events and PICU transfers to this date, mapping each event to a unit-specific postimplementation month. For each unit-month, we generated a proportion of Tier I debriefings completed for PICU transfers as compared with the total number of PICU transfers. This served as the primary outcome in our statistical analysis. As a secondary outcome, we generated a proportion of Tier I events with reported FLC participation as compared with the total number of Tier I events per unit-month.

To compare adoption across seasons, we assigned each unit-month a seasonal quartile based on the calendar month. To account for acuity, we assigned each unit a global acuity quartile on the basis of the mean number of monthly PICU transfers observed postimplementation. We also assigned each unit-month an acuity quartile to account for dynamic monthly acuity. To account for staffing differences, we assigned each unit a nurse staffing quartile based on the number of full-time nurses providing direct patient care. We used a binary variable to indicate units with dedicated FLC staffing.

Descriptive Analysis

We used descriptive statistics to examine unit, patient, and team-level data from Tier I guides. We used Kruskal–Wallis testing to compare monthly Tier I completion by unit, season, monthly acuity, overall acuity, and nurse staffing. We used Student’s t tests to compare monthly Tier I completion and FLC participation across units with and without dedicated FLC staffing. We performed all analyses by using Stata version 11 (Stata Corp, College Station, TX).

Qualitative Analysis

Two coders (M.M.G. and H.A.W.) reviewed free-response answers from completed Tier I guides, with a focus on event review items (Supplemental Information). Using inductive reasoning with a grounded theory approach, the coders used constant comparative coding to understand and identify themes and priorities identified by debriefing participants.14  This iterative approach led to the development of a codebook that was applied to the data. To reduce bias, coders were blinded to unit and event characteristics. All data were double-coded. Disagreement among the coders was discussed until consensus was reached.

From March 28, 2019, to February 10, 2020, ward providers across 9 units performed Tier I debriefing for 134 of 312 PICU transfers (43%). A subset of 19 Tier I debriefings triggered Tier II debriefing on the basis of clinical criteria, but in 14 of these cases (74%), Tier I participants also requested additional debriefing.

Nurses participated in Tier I debriefing most often: 117 (87%) of debriefings reported participation by a bedside nurse and 116 (87%) by a charge nurse. In contrast, only 65 (49%) of debriefings reported participation by a FLC and 15 (11%) by an attending physician. All debriefings reported participation by at least 1 nurse. There were no significant differences in FLC participation by day versus night shift (P = .22); however, attending physicians were significantly more likely to attend a debrief on day shift (P = .01).

Overall, units performed Tier I debriefing for 20% to 64% of PICU transfers, debriefing an average of 47% of events (Table 2). In bivariate statistical analyses, monthly debriefing completion did not vary significantly by unit, season, monthly acuity, overall acuity, or nurse staffing. However, as compared with units with fully rotational FLC staffing, units with dedicated FLCs completed significantly more monthly debriefings (average of 57% of PICU transfers debriefed [SD: 30%] versus 33% [SD: 28%]; P = .004). Monthly FLC participation was also higher among units with dedicated FLCs (average of 50% of debriefings attended by a FLC [SD: 37%] versus 24% [SD: 37%]; P = .014).

TABLE 2

Summary of Ward Unit Observations FTE, full-time employee; n/a, not applicable.

Patient Population OverviewNo. Unit-Monthsa ObservedNo. Nurse FTEFLCb StaffingNo. PICU Transfers (Total)No. Tier I Debriefingsc (Total)Overall Tier I completion, %Average Monthly Tier I completion, % (SD)
Unit 1 Oncology, bone marrow transplant 12 100 Dedicated and rotational 94 51 54 54 (22) 
Unit 2 General medical, noninvasive respiratory cohort 10 64 Rotational 93 19 20 23 (20) 
Unit 3 General medical 55 Dedicated and rotational 31 17 55 57 (37) 
Unit 4 General medical 31 Rotational 11 36 44 (52) 
Unit 5 General medical 37 Dedicated and rotational 25 16 64 69 (29) 
Unit 6 General medical Rotational 26 11 42 43 (11) 
Unit 7 General medical 31 Rotational 14 43 31 (34) 
Unit 8 General medical 71 Dedicated and rotational 16 56 50 (50) 
Unit 9 General medical 24 Rotational 50 50 (n/a) 
Total n/a n/a n/a n/a 312 134 n/a n/a 
Mean n/a n/a n/a n/a n/a n/a 47 47 
Patient Population OverviewNo. Unit-Monthsa ObservedNo. Nurse FTEFLCb StaffingNo. PICU Transfers (Total)No. Tier I Debriefingsc (Total)Overall Tier I completion, %Average Monthly Tier I completion, % (SD)
Unit 1 Oncology, bone marrow transplant 12 100 Dedicated and rotational 94 51 54 54 (22) 
Unit 2 General medical, noninvasive respiratory cohort 10 64 Rotational 93 19 20 23 (20) 
Unit 3 General medical 55 Dedicated and rotational 31 17 55 57 (37) 
Unit 4 General medical 31 Rotational 11 36 44 (52) 
Unit 5 General medical 37 Dedicated and rotational 25 16 64 69 (29) 
Unit 6 General medical Rotational 26 11 42 43 (11) 
Unit 7 General medical 31 Rotational 14 43 31 (34) 
Unit 8 General medical 71 Dedicated and rotational 16 56 50 (50) 
Unit 9 General medical 24 Rotational 50 50 (n/a) 
Total n/a n/a n/a n/a 312 134 n/a n/a 
Mean n/a n/a n/a n/a n/a n/a 47 47 
a

An implementation date was defined for each unit as the first Tier I event date; stepwise implementation resulted in variation of unit observations.

b

Dedicated FLCs have clinical responsibilities primarily limited to that unit.

c

Limited to PICU transfers.

In reviewing Tier I guides, a change in vital signs was the most frequently cited sign of clinical deterioration (61% of debriefings). In 13% of debriefings, participants reported no signs of clinical status change (eg, mental status changes) preceding the acute deterioration event. Watcher status was in place for fewer than half of patients (45%) at deterioration. In evaluating team performance, participants frequently identified prompt recognition of deterioration (76% of debriefings) and activation of the medical emergency or code blue team (81% of debriefings) as strengths. Appropriate watcher identification was the most commonly cited area for improvement (19% of debriefings).

We identified several themes through qualitative analysis of completed debriefing guides, comprising 165 distinct responses (Table 3). Participants most frequently referenced teaming activities such as communication, collaboration, and decision-making (Fig 2).

TABLE 3

Major Themes Identified in Qualitative Analysis of Tier I Debriefing Guides

ThemeNotesRepresentative Quotations
Teaming activities Addresses team functioning, including communication, collaboration, and/or decision-making; may refer to ward team only or both ward team and ICU response teams “Patient placed on high flow [nasal canula] by Respiratory Therapist, which patient needed at the time of assessment. However, communication between bedside nurse, respiratory, and frontline clinician (FLC) that patient was on high flow needed to be more direct and [more] clear.” 
Clinical skills Addresses clinical skills of ward staff, including the ability to identify clinical deterioration; identify the need for care escalation; and/or identify a need for additional resources “Patient did not have their [oxygen saturation] checked with increase in respiratory rate at 0400 this morning [preceding the event].” 
Situation awareness & escalation Addresses steps to identify and/or mitigate deterioration and/or escalate patient care; may include situation awareness successes, failures, and/or opportunities for improvement “Will need additional follow up. Watcher status was discussed between FLC, bedside nurse, and charge nurse. Chose not to formally identify patient as a watcher although she was being treated as one.” 
Missed opportunities Addresses ward team–identified opportunities for improved ward staff or ward team performance surrounding deterioration “[We] could have asked for a [glucose check] sooner… If a larger team comes to take the patient to the ICU, maybe wait to crowd around patient before the primary team and main code team have completed necessary interventions.” 
Potential for bias Addresses ward team–identified areas with potential for bias, including normalization of patient acuity “Patient had an increase in airway clearance and appeared uncomfortable, but [it was] unclear to the team if this [was] something the patient has exhibited before or a change from ‘normal’” 
Timeliness of Response Addresses timeliness of identification of deterioration and/or escalation of care; may include reference to ICU response team “It took over 3 h to finalize transfer from [ward] to the PICU. After the [Medical Emergency Team] decided to take the patient to the PICU, we had to wait for a room to be finished cleaning, [bedside nursing] handoff, and [FLC] handoff” 
Care environment Addresses physical and/or affective elements impacting care environment “Code team had calming demeanor which helped lower the stress level in the room.” 
Hospital resources Addresses hospital resource availability and/or provision of hospital resources “[Concern for] code alert malfunction. Need clarification from [Medical Emergency Team]. Respiratory did not get alert at same time.” 
Materials Addresses physical materials necessary for provision of patient care “Could not find supplies for [intraosseous] access in the code cart. ICU nurses could not locate in the code cart either. Unsure if it was stocked or if someone removed from the cart.” 
Rapidity of deterioration Addresses the rate of patient decline surrounding deterioration event “The patient had a sudden decompensation consistent with aspiration.” 
ThemeNotesRepresentative Quotations
Teaming activities Addresses team functioning, including communication, collaboration, and/or decision-making; may refer to ward team only or both ward team and ICU response teams “Patient placed on high flow [nasal canula] by Respiratory Therapist, which patient needed at the time of assessment. However, communication between bedside nurse, respiratory, and frontline clinician (FLC) that patient was on high flow needed to be more direct and [more] clear.” 
Clinical skills Addresses clinical skills of ward staff, including the ability to identify clinical deterioration; identify the need for care escalation; and/or identify a need for additional resources “Patient did not have their [oxygen saturation] checked with increase in respiratory rate at 0400 this morning [preceding the event].” 
Situation awareness & escalation Addresses steps to identify and/or mitigate deterioration and/or escalate patient care; may include situation awareness successes, failures, and/or opportunities for improvement “Will need additional follow up. Watcher status was discussed between FLC, bedside nurse, and charge nurse. Chose not to formally identify patient as a watcher although she was being treated as one.” 
Missed opportunities Addresses ward team–identified opportunities for improved ward staff or ward team performance surrounding deterioration “[We] could have asked for a [glucose check] sooner… If a larger team comes to take the patient to the ICU, maybe wait to crowd around patient before the primary team and main code team have completed necessary interventions.” 
Potential for bias Addresses ward team–identified areas with potential for bias, including normalization of patient acuity “Patient had an increase in airway clearance and appeared uncomfortable, but [it was] unclear to the team if this [was] something the patient has exhibited before or a change from ‘normal’” 
Timeliness of Response Addresses timeliness of identification of deterioration and/or escalation of care; may include reference to ICU response team “It took over 3 h to finalize transfer from [ward] to the PICU. After the [Medical Emergency Team] decided to take the patient to the PICU, we had to wait for a room to be finished cleaning, [bedside nursing] handoff, and [FLC] handoff” 
Care environment Addresses physical and/or affective elements impacting care environment “Code team had calming demeanor which helped lower the stress level in the room.” 
Hospital resources Addresses hospital resource availability and/or provision of hospital resources “[Concern for] code alert malfunction. Need clarification from [Medical Emergency Team]. Respiratory did not get alert at same time.” 
Materials Addresses physical materials necessary for provision of patient care “Could not find supplies for [intraosseous] access in the code cart. ICU nurses could not locate in the code cart either. Unsure if it was stocked or if someone removed from the cart.” 
Rapidity of deterioration Addresses the rate of patient decline surrounding deterioration event “The patient had a sudden decompensation consistent with aspiration.” 
FIGURE 2

Pareto chart of themes identified in Tier I debriefings through qualitative analysis.

FIGURE 2

Pareto chart of themes identified in Tier I debriefings through qualitative analysis.

Close modal

In this study, we describe the implementation of a multidisciplinary debriefing process for ward deterioration events. This process involves immediate debriefing for any event culminating in ICU transfer. Guided by the RE-AIM framework, we sought to characterize process reach, adoption and implementation patterns in a multimethod analysis. We found variable process reach across providers, with nurses participating most often. Process adoption varied across units, but without a significant difference in monthly completion. Adoption did not vary significantly by season, observed acuity, or nurse staffing. However, as compared with units with fully rotational FLC staffing, units with dedicated FLCs performed debriefing for a significantly higher proportion of events and had significantly greater monthly FLC participation in debriefings. In descriptive and qualitative analyses of debriefing guides, we identified implementation themes and priorities of debriefing participants, with teaming activities such as situation awareness and communication cited often.

There are many factors to consider in assessing these outcomes. First, it is important to acknowledge that process implementation involves complex decision-making with multilevel predictors.15  At the organizational level, varying professional responsibilities may impact process reach. Because nursing leaders facilitate debriefings, nurses may have participated more actively because of increased awareness and/or investment. This process also leverages jargon (“Tier I” and “Tier II”) from an institutional harm prevention program16,17  led locally by unit nursing leaders. As such, there may have been a spillover effect for debriefing participation among nurses but not other providers. Ward attending physicians also infrequently provide in-hospital care overnight, which likely contributed to significantly lower participation in debriefings taking place during night shifts.

At the provider level, differences across permanent and rotational unit staff may impact process reach and adoption. Rotational providers may participate in debriefing less often as a consequence of decreased awareness of local safety initiatives. Our finding that FLCs participated more actively on units with dedicated staffing would support this argument. In their study, Aponte-Patel et al18  also demonstrated that increasing awareness of debriefing led to improved completion rates in a resident physician-led debriefing process, suggesting awareness may be a driver of debriefing practice. Alternatively, investment in unit-based initiatives may vary across permanent and rotational staff. Consider a full-time nurse on the oncology ward: they may participate more actively in debriefing because it can improve their work environment in a meaningful and longitudinal way. In contrast, a rotational clinician providing 1 to 4 weeks of service on oncology may perceive less direct or sustained benefit from debriefing. Our results could support this hypothesis, because nurses participated in debriefing more frequently than FLCs, who more often serve in rotational roles.

Process design may have also impacted our findings. First, there is no required provider composition for Tier I debriefing. Second, at the time of this study, there were no feedback mechanisms in place for ward leaders to cultivate or monitor debriefing engagement. Third, at the time of this study, there was a lack of provider education surrounding best practices for debriefing. Collectively, this lack of operational fidelity may have contributed to variable process implementation.

It is also worth considering how organizational dynamics of the ward team impact debriefing. In particular, psychological safety,19  cited as a critical component of debriefing,20,21  may influence practice. Because psychological safety is positively associated with professional status,22  one might infer from our data that nurses feel less comfortable inviting physicians and other providers to participate in debriefing. One might also speculate that some degree of team stability can mitigate this effect, as reflected in significantly higher FLC participation on units with more consistent FLC staffing.

Our multimethod analysis of debriefing guides also identified an important role for team culture. We found that participants frequently referenced organizational dynamics of the ward team, including how providers communicated, maintained situation awareness, identified signs of deterioration, and escalated care. These themes may reflect the unique priorities of the immediate debriefing group, a unit-based group faced with continuing to work together as a team for the remainder of their shift. It is perhaps out of necessity, then, that participants placed value on recognizing and reinforcing effective team behaviors.

In response to these findings, our hospital refined debriefing implementation, with a focus on quality assurance and operational fidelity. We designed data systems enabling ward leaders to track and compare their debriefing metrics across units, including Tier I completion and FLC participation. Operational leaders added Tier I completion as a performance metric for unit leaders to create local accountability. To standardize best practices, the patient safety team instituted a reviewer education series for code champions, including strategies for cultivating multidisciplinary participation. To promote physician engagement, we held informational sessions with various physician forums to spread awareness of the process and ask for critical feedback. Physician engagement is key to this initiative’s success, because patient safety programs that rely solely on nurses to implement and enforce new policies often fail to cultivate the multidisciplinary investment required for improvement.23,24  We have also continued to partner with senior hospital leaders to promote awareness at all levels of the organization. Of note, the implementation patterns we identified have provided important feedback to hospital safety initiatives. As an example, our results pertaining to the watcher system have informed situation awareness programming, including the development of a clinician concern score.25 

This study is not without limitations. In our analysis, we did not account for time-varying characteristics such as postimplementation exposure. Although there were no significant changes to implementation during the study period, varying implementation times may confound our findings, given an overrepresentation of early adopting units and our inability to account for interaction among covariates in statistical analysis. Rotational FLCs may have also biased our results because of variable exposures. We also did not account for the time required to debrief or the clinical nature of deterioration events, which have been shown to influence debriefing.26  Additionally, we did not assess debriefing quality, an important dimension of any team reflection activity.27  Importantly, our findings may not adequately reflect multidisciplinary debriefing, given the overrepresentation of nurse participants. However, because of limited sample size, we were unable to evaluate the impact of provider composition on debriefing. We also used documentation as a proxy for in-person debriefing. Documentation could be confounded by staff workload or other factors not captured in our study. There was also potential for selection bias as a consequence of limited operational fidelity at the time of study (eg, not every free response required an answer). Finally, we acknowledge the risk of endogeneity in our qualitative data, given the debriefing guide was designed to facilitate reflection.

Implementation of a multidisciplinary debriefing process for ward deterioration events culminating in ICU transfer was associated with differential adoption across providers and FLC staffing models but not acuity or nurse staffing. Teaming activities emerged as a debriefing priority. Future study will assess patient safety outcomes.

The authors thank the following safety and clinical operations leaders for their support and commitment to the implementation of the program described in this manuscript: Ana Altmann, DM; Lynn Boyle, MSN; Ann Davis, DNP; Haley Hlela, MSN; Megan Snyder, MSN; Tyshawn Toney, MHA; Ellen Tracy, MSN; and Drs Geoffrey Bird, Evan Fieldston, Ron Keren, Naveen Muthu; Ursula Nawab; and Margaret Priestley. The authors also thank Dr Martha Curley for her support in data acquisition.

Dr Galligan conceptualized and designed the study, participated in design of data collection instruments, collected and analyzed the data, and drafted the initial manuscript; Drs Wolfe and Liu participated in study design and data analysis; Ms Papili, Ms Colfer, Ms Neiswender, Ms Granahan, Ms McGowan, and Ms McGrath participated in study design and design of data collection instruments and coordinated and supervised data collection; Mr Porter and Ms O’Shea participated in design of data collection instruments, and they also coordinated and supervised data collection; Dr Shaw participated in study design; Dr Sutton participated in study design and design of data collection instruments, and he also supervised data collection and analysis; and all authors reviewed and revised the manuscript, and approved the final manuscript as submitted.

FUNDING: No external funding.

1
Blankenship
AC
,
Fernandez
RP
,
Joy
BF
, et al
.
Multidisciplinary review of code events in a heart center
.
Am J Crit Care
.
2016
;
25
(
4
):
e90
e97
2
Wolfe
H
,
Zebuhr
C
,
Topjian
AA
, et al
.
Interdisciplinary ICU cardiac arrest debriefing improves survival outcomes
.
Crit Care Med
.
2014
;
42
(
7
):
1688
1695
3
Tannenbaum
SI
,
Cerasoli
CP
.
Do team and individual debriefs enhance performance? A meta-analysis
.
Hum Factors
.
2013
;
55
(
1
):
231
245
4
Brady
PW
,
Muething
S
,
Kotagal
U
, et al
.
Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events
.
Pediatrics
.
2013
;
131
(
1
). Available at: www.pediatrics.org/cgi/content/full/131/1/e298
5
Hussain
FS
,
Sosa
T
,
Ambroggio
L
,
Gallagher
R
,
Brady
PW
.
Emergency transfers: an important predictor of adverse outcomes in hospitalized children
.
J Hosp Med
.
2019
;
14
(
8
):
482
485
6
Raymond
TT
,
Bonafide
CP
,
Praestgaard
A
, et al
;
American Heart Association Get With the Guidelines-Resuscitation Investigators
.
Pediatric medical emergency team events and outcomes: a report of 3647 events from the American Heart Association’s get with the guidelines-resuscitation registry
.
Hosp Pediatr
.
2016
;
6
(
2
):
57
64
7
Holtrop
JS
,
Rabin
BA
,
Glasgow
RE
.
Qualitative approaches to use of the RE-AIM framework: rationale and methods
.
BMC Health Serv Res
.
2018
;
18
(
1
):
177
8
Proctor
E
,
Silmere
H
,
Raghavan
R
, et al
.
Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda
.
Adm Policy Ment Health
.
2011
;
38
(
2
):
65
76
9
Gaglio
B
,
Shoup
JA
,
Glasgow
RE
.
The RE-AIM framework: a systematic review of use over time
.
Am J Public Health
.
2013
;
103
(
6
):
e38
e46
10
Bonafide
CP
,
Localio
AR
,
Roberts
KE
,
Nadkarni
VM
,
Weirich
CM
,
Keren
R
;
CP
.
Impact of rapid response system implementation on critical deterioration events in children
.
JAMA Pediatr
.
2014
;
168
(
1
):
25
33
11
Harris
PA
,
Taylor
R
,
Minor
BL
, et al
;
REDCap Consortium
.
The REDCap consortium: building an international community of software platform partners
.
J Biomed Inform
.
2019
;
95
:
103208
12
Parker
MW
,
Carroll
M
,
Bolser
B
, et al
.
Implementation of a communication bundle for high-risk patients
.
Hosp Pediatr
.
2017
;
7
(
9
):
523
529
13
McClain Smith
M
,
Chumpia
M
,
Wargo
L
,
Nicol
J
,
Bugnitz
M
.
Watcher initiative associated with decrease in failure to rescue events in pediatric population
.
Hosp Pediatr
.
2017
;
7
(
12
):
710
715
14
Bradley
EH
,
Curry
LA
,
Devers
KJ
.
Qualitative data analysis for health services research: developing taxonomy, themes, and theory
.
Health Serv Res
.
2007
;
42
(
4
):
1758
1772
15
Chor
KHB
,
Wisdom
PJ
,
Olin
S-CS
,
Hoagwood
EK
,
Hortwitz
MS
.
Measures for predictors of innovation adoption
.
Adm Policy Ment Health
.
2015
;
42
(
5
):
545
573
16
Miske
LJ
,
Stetzer
M
,
Garcia
M
,
Stellar
JJ
.
Airways and injuries: protecting our pediatric patients from respiratory device-related pressure injuries
.
Crit Care Nurs Clin North Am
.
2017
;
29
(
2
):
187
204
17
Hei
H
,
Bezpalko
O
,
Smathers
SA
,
Coffin
SE
,
Sammons
JS
.
Development of a novel prevention bundle for pediatric healthcare-associated viral infections
.
Infect Control Hosp Epidemiol
.
2018
;
39
(
9
):
1086
1092
18
Aponte-Patel
L
,
Salavitabar
A
,
Fazzio
P
,
Geneslaw
AS
,
Good
P
,
Sen
AI
.
Implementation of a formal debriefing program after pediatric rapid response team Activations
.
J Grad Med Educ
.
2018
;
10
(
2
):
203
208
19
Edmondson
A
.
Psychological safety and learning behavior in work teams
.
Adm Sci Q
.
1999
;
44
(
2
):
350
383
20
Appleton
KP
,
Nelson
S
,
Wedlund
S
.
Distress debriefings after critical incidents: a pilot project
.
AACN Adv Crit Care
.
2018
;
29
(
2
):
213
220
21
Harder
N
,
Lemoine
J
,
Harwood
R
.
Psychological outcomes of debriefing healthcare providers who experience expected and unexpected patient death in clinical or simulation experiences: a scoping review
.
J Clin Nurs
.
2020
;
29
(
3–4
):
330
346
22
Nembhard
IM
,
Edmondson
AC
.
Making it safe: the effects of leader inclusiveness and professional status on psychological safety and improvement efforts in health care teams
.
J Organ Behav
.
2006
;
27
(
7
):
941
966
23
Pronovost
PJ
,
Rosenstein
BJ
,
Paine
L
, et al
.
Paying the piper: investing in infrastructure for patient safety
.
Jt Comm J Qual Patient Saf
.
2008
;
34
(
6
):
342
348
24
McGonigal
M
,
Bauer
M
,
Post
C
.
Physician engagement: a key concept in the journey for quality improvement
.
Crit Care Nurs Q
.
2019
;
42
(
2
):
215
219
25
Romero-Brufau
S
,
Gaines
K
,
Nicolas
CT
,
Johnson
MG
,
Hickman
J
,
Huddleston
JM
.
The fifth vital sign? Nurse worry predicts inpatient deterioration within 24 hours
.
JAMIA Open
.
2019
;
2
(
4
):
465
470
26
Mullan
PC
,
Wuestner
E
,
Kerr
TD
,
Christopher
DP
,
Patel
B
.
Implementation of an in situ qualitative debriefing tool for resuscitations
.
Resuscitation
.
2013
;
84
(
7
):
946
951
27
Otte
KP
,
Konradt
U
,
Oldeweme
M
.
Effective team reflection: the role of quality and quantity
.
Small Group Res
.
2018
;
49
(
6
):
739
766

Competing Interests

POTENTIAL CONFLICT OF INTEREST: Dr Sutton serves as the Chair of the National American Heart Association’s Get with the Guidelines-Resuscitation Pediatric Research Task Force and has been a main author of the Pediatric Advanced Life Support Guidelines since 2015; the other authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: Dr Sutton is the recipient of multiple National Institutes of Health research grants that are unrelated to this article; the other authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data