BACKGROUND

Despite the growth of patient safety programs across the United States, errors and adverse events remain a source of patient harm. Many hospitals rely on retrospective voluntary reporting systems; however, there are opportunities to improve patient safety using novel tools like trigger programs.

METHODS

Children’s National Hospital developed a unique pediatric triggers program that offers customized, near real-time reports of potential safety events. Our team defined a measure to quantify clinical utility of triggers, termed “trigger signal,” as the percentage of cases that represent true adverse or near-miss events (numerator) per total triggers activated (denominator). Our key driver diagram focused on unifying the program structure, increasing data analytics, promoting organizational awareness, and supporting multidisciplinary end user engagement. Using the model for improvement, we aimed to double overall trigger signal from 8% to 16% and sustain for 12 months.

RESULTS

The trigger signal increased from 8% to 41% and sustained during the coronavirus disease 2019 pandemic. A balancing measure of time to implement a new trigger decreased. Key interventions to increase trigger signal were change in the program structure, increasing stakeholder engagement, and development of self-service reports for end users.

CONCLUSIONS

Children’s National Hospital’s triggers program highlights successful evolution of an iterative, customized approach to increase clinical utility that hospitals can implement to impact real-time patient care. This triggers program requires an iterative, customized approach rather than a “1-size-fits-all,” static paradigm to add a new dimension to current patient safety programs.

Medical errors are a significant cause of patient harm; it has been estimated that patient harm is the third leading cause of death in America.1  More recent studies have shown mortality from preventable adverse events is more prevalent than first described by the Institute of Medicine in 2000.2,3  As medicine increases in complexity with new technologies and treatments, adverse events continue to rise; this is true in pediatric teaching hospitals and those that care for complex patients with many chronic problems.46  Hospitals need to continually reassess opportunities to identify and prevent errors to improve patient safety.

Retrospective, voluntary error reporting is currently the mainstay of harm detection at most hospitals.7  Ideally, errors should be identified before harm occurs8 ; unfortunately, these retrospective reporting systems underestimate the true number of medical errors and are restricted by underreporting, inequitable reporting, and limited ability to improve real-time patient care.914  Additionally, adverse events and retrospective reporting programs have demonstrated inequities among certain patient groups, such as decreased adverse event reporting in lower socioeconomic populations, obese patients, and those of African American race.1015 

Although retrospective error documentation is an important source of safety information, we need new tools to advance patient safety; trigger tools add a new dimension to existing safety programs. Triggers are targeted signs or events (from progress notes, vital signs, orders, etc.) embedded in the electronic health record (EHR) that act as a “clue” that an adverse event may have occurred. For example, the administration of naloxone acts as an indication of opioid overdose. A triggers program can be a sensitive and reliable process to identify precursors to adverse events, rather than solely using voluntary reports.16,17  In 2003, the Institute for Healthcare Improvement (IHI) first developed an adult-focused trigger tool and followed with a white paper that demonstrated triggers could detect errors at a higher rate than passive methods.18  In 2015, the Global Assessment of Pediatrics Patient Safety (GAPPS) tool was developed for pediatric inpatients.19  Combined with retrospective error reporting, trigger data may allow hospitals to optimize detection of adverse events in hospitalized patients.14,20,21  Current trigger tools, however, still require retrospective, manual review of patient data; in the era of ubiquitous healthcare data, there is a critical need for near real-time data processing to improve safety.21  Therefore, it is imperative for trigger tools to evolve to create a customized, timely method to identify errors with the aim to improve real-time patient care.

Several years after implementation of Children’s National Hospital (CNH) triggers program, enthusiasm for the program was limited, perhaps driven by concerns for clinical usefulness. The report logic and definitions were broad, resulting in false-positive triggers and ultimately lower use among clinical end users (eg, physicians, nurse managers, unit directors). We sought to revitalize the CNH triggers program as an innovative safety tool using quality improvement methods to increase the value-add of the program, focusing on automation and customization of triggers to create a stronger, more impactful workflow.

As part of our ongoing efforts to increase adoption of the triggers program across the organization, we launched an improvement effort using a novel, tailored approach that sought to increase the trigger signal. We defined a measure to quantify the clinical utility of our triggers, termed “trigger signal,” as the percent of cases identified automatically by the triggers program that represent a true occurrence of a targeted adverse or precursor adverse event per total triggers activated. The specific aim was to double overall trigger signal from our baseline of 8% to 16% and sustain for 12 months.

CNH is a free-standing, academic children’s hospital located in Washington, DC with 323 acute-care beds with over 8000 employees. The CNH pediatric triggers tool program (created in 2013) first adopted concepts from the GAPPS and IHI trigger tools, which provide a framework for useful quality metrics to detect adverse events and opportunities for process improvement.17,22  Our preintervention data showed a baseline of 8% trigger signal before the changes described in this quality improvement report. The current program, a hybrid model of automatic trigger reports with human oversight, aims to capture data from the EHR and apply it in a clinically meaningful, customized manner to improve patient safety and organizational quality.

This project is a quality improvement initiative determined by the CNH Institutional Review Board to not constitute human subjects’ research. Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines were used to present this work.23 

Table 1 describes the desirable characteristics of triggers: automaticity, usability, and end user reliability. CNH’s original triggers were chosen based off the IHI Trigger Tool.17  Since 2018, the program expanded from 6 triggers to 10 triggers, highlighting the adaptability of the program through an iterative process of retirement and addition of triggers based on feedback from end users and improvement in logic creation (Table 2). See Appendix for the description and logic used to create each trigger.

TABLE 1

Ideal Characteristics of Triggers Used in the Children’s National Hospital Trigger Program

Characteristics for Triggers
Automaticity Automated reports instead of manual chart review gives end user flexibility to access reports at their convenience 
Data sets are templated specifically for the content being pulled; increased detail avoiding broad logic will ideally decrease false positives 
Ability to pull from an Electronic Data Warehouse platform allows the program to add socioeconomic parameters to trigger reports, such as race, language, and insurance 
Usability Sufficient information is displayed for end user to use reports immediately; chart review is optional, not necessary 
Reporting platforms directly allow visual trending to pinpoint specific thresholds in any given time frame 
Reporting platform logic can easily change to match evolution of organization’s goals and initiatives without dependence on outside reporting resources 
Reliability Fewer false positives creates positive feedback and engagement from the end user. Increased trust in program so that reports become part of the workflow 
Characteristics for Triggers
Automaticity Automated reports instead of manual chart review gives end user flexibility to access reports at their convenience 
Data sets are templated specifically for the content being pulled; increased detail avoiding broad logic will ideally decrease false positives 
Ability to pull from an Electronic Data Warehouse platform allows the program to add socioeconomic parameters to trigger reports, such as race, language, and insurance 
Usability Sufficient information is displayed for end user to use reports immediately; chart review is optional, not necessary 
Reporting platforms directly allow visual trending to pinpoint specific thresholds in any given time frame 
Reporting platform logic can easily change to match evolution of organization’s goals and initiatives without dependence on outside reporting resources 
Reliability Fewer false positives creates positive feedback and engagement from the end user. Increased trust in program so that reports become part of the workflow 
TABLE 2

Timeline of Trigger Development and Evolution During 2018 Through 2021 With Description of Retired Triggers

Trigger NameOrigin2018201920202021
Unplanned transfers to ICU IHI Global Tool ✓ ✓ ✓ ✓ 
Serum creatinine doubling (>2×baseline)a IHI Global Tool ✓ ✓ ✓ 
Hypoglycemia <40mg IHI Global Tool ✓ ✓ ✓ ✓ 
Infiltrations: hyaluronidase administration IHI Global Tool ✓ ✓ ✓ ✓ 
Naloxone or flumazenil administration IHI Global Tool ✓ ✓ ✓ ✓ 
Anticoagulation: INR > 6, anti-X-A >1.5mg/dL, PTT >139b IHI Global Tool ✓ 
Suspected severe and nonsevere sepsis Novel trigger: HealtheIntent — 1/2019 * * 
PIVIE (peripheral IV infiltration and extravasation) Novel trigger: HealtheIntent — 1/2019 * * 
Hospital readmissions Novel trigger: HealtheIntent — — 5/2020 * 
NICU pain reassessment time Novel trigger: HealtheIntent — — 11/2020 * 
NAKI (nephrotoxic acute kidney injury) Novel trigger: HealtheIntent — — — 1/2021 
ED and inpatient pain reassessment time Novel trigger: HealtheIntent — — — 4/2021 
Trigger NameOrigin2018201920202021
Unplanned transfers to ICU IHI Global Tool ✓ ✓ ✓ ✓ 
Serum creatinine doubling (>2×baseline)a IHI Global Tool ✓ ✓ ✓ 
Hypoglycemia <40mg IHI Global Tool ✓ ✓ ✓ ✓ 
Infiltrations: hyaluronidase administration IHI Global Tool ✓ ✓ ✓ ✓ 
Naloxone or flumazenil administration IHI Global Tool ✓ ✓ ✓ ✓ 
Anticoagulation: INR > 6, anti-X-A >1.5mg/dL, PTT >139b IHI Global Tool ✓ 
Suspected severe and nonsevere sepsis Novel trigger: HealtheIntent — 1/2019 * * 
PIVIE (peripheral IV infiltration and extravasation) Novel trigger: HealtheIntent — 1/2019 * * 
Hospital readmissions Novel trigger: HealtheIntent — — 5/2020 * 
NICU pain reassessment time Novel trigger: HealtheIntent — — 11/2020 * 
NAKI (nephrotoxic acute kidney injury) Novel trigger: HealtheIntent — — — 1/2021 
ED and inpatient pain reassessment time Novel trigger: HealtheIntent — — — 4/2021 

Anticoagulation triggers: specified criteria led to increased false positives because patients should have been excluded or laboratory abnormalities represented progression of the disease. Serum creatinine doubling trigger (>2×baseline): specified criteria led to false negatives (patients whose creatinine doubled but stayed less than 0.6 mg/dL) and false positives (patients whose baseline creatinine is greater than 0.6 mg/dL). HealthIntent: a cloud-based, programmable platform through Cerner that allows for aggregation in near real-time of health data via the electronic health record. —, trigger not yet developed; ✓, IHI Global Tool: current; x, IHI Global Tool: retired;

*

, novel trigger: HealtheIntent (current); ED, Emergency Department; INR, International Normalized Ratio;  PTT, Partial Thromboplastin Time.

a

Retired because of lack of specificity, replaced to NAKI.

b

Retired because of lack of specificity.

A key driver diagram, developed by a multidisciplinary team, shows opportunities to improve triggers data in 4 areas: program structure, reliable data analytics, organizational awareness, and multidisciplinary end user engagement (Fig 1). We developed the following interventions to accomplish these goals (Table 3):

FIGURE 1

Key driver diagram indicating interventions used during the improvement of our triggers program; bolded interventions are planned for the future.

FIGURE 1

Key driver diagram indicating interventions used during the improvement of our triggers program; bolded interventions are planned for the future.

Close modal
TABLE 3

Timeline of Interventions for the Triggers Program Quality Improvement Project

InterventionDate Initiated
A. Establish joint project lead oversight by both quality and clinical informatics Nov 2018 
B. Create self-service reports to improve usability for clinical teams Dec 2018 
C. Facilitate continuous stakeholder engagement to optimize trigger signal Jan 2019 
D. Shift ownership from centralized trigger committee to distributed model embedded throughout organization Dec 2019 
E. Link triggers to organizational safety event reporting system with specific criteria Jan–Jun 2020 
F. Stimulate new trigger development based on existing trigger platforms May 2021 
InterventionDate Initiated
A. Establish joint project lead oversight by both quality and clinical informatics Nov 2018 
B. Create self-service reports to improve usability for clinical teams Dec 2018 
C. Facilitate continuous stakeholder engagement to optimize trigger signal Jan 2019 
D. Shift ownership from centralized trigger committee to distributed model embedded throughout organization Dec 2019 
E. Link triggers to organizational safety event reporting system with specific criteria Jan–Jun 2020 
F. Stimulate new trigger development based on existing trigger platforms May 2021 

A. Establish Joint Project Lead Oversight by Both Quality Safety and Clinical Informatics

A key aspect of the program is the new creation of a collaboration between the Performance Improvement department, reporting to the Chief Quality and Safety Officer, and the department of Clinical Informatics, reporting to the Chief Medical Informatics Officer. These teams include data analysts and specialists who work on performance improvement measures for the organization, which represents a total of 4 to 6 individuals across the 2 departments, including a Triggers Program Lead. The structure utilizes each department’s strengths, primarily the ability to use data that comes from Clinical Informatics, in combination with the Performance Improvement department’s culture of safety. Although triggers are data-driven, they require quality and process improvement methods for the information to be useful to the organization. This connection improves the efficiency with which a trigger can be developed and the reception by clinical teams.

B. Shift Ownership From Centralized Trigger Committee to Distributed Model Embedded Throughout Organization

Previously, there was a central steering committee that met monthly to review all triggers; however, this committee was not effective at discussing triggers in an actionable manner because of lack of accountability for trigger reports and follow-up. The new structure links clinical committees or divisions with individual triggers to capture clear stakeholders for continuous, iterative improvement of the triggers. The new committees meet monthly to review trigger data in detail when reporting to their chair. Additionally, many of the committees report annually to the clinical effectiveness committee, and they use trigger data for pertinent quality and safety meetings. As an example, the peripheral intravenous (IV) infiltrate group has specific organizational safety goals that are aligned and customized within the peripheral IV infiltrate trigger (Appendix).

C. Facilitate Continuous Stakeholder Engagement to Optimize Trigger Signal

When customizing a trigger, we work directly with end users to align with the team’s clinical requirements. This iterative process requires regular conversations between the end users and the triggers program team. Continuous engagement allows end users to add fields and relevant clinical information to triggers to increase trigger signal. For example, the initial trigger for acute kidney injury (Table 2) focused on increased creatinine but was not specific and produced a significant number of false positives, yielding poor trigger signal. Therefore, after review with end users, this trigger was replaced with more specific criteria.

D. Create Self-service Reports to Improve Usability for Clinical Teams

We migrated our triggers data source from the EHR database (Cerner Millennium, Kansas City) to our Electronic Data Warehouse (EDW) (Cerner HealtheIntent, Kansas City), which is a platform that allows for aggregation of near real-time data from the EHR. Triggers created using data from the EDW can identify more specific cohorts of information (guided by the clinical teams), which reduces false positives. The EDW allows for a more fully automated, secure, self-service model for the clinical teams, without relying on scheduled retrospective reports to be distributed. This automation reduces dependency on technical staff, reduces turnaround time, is readily available for clinical teams, and decreases the time required to create new triggers or modify existing triggers.

E. Link Triggers to the Organizational Safety Event Reporting System With Specific Criteria

Although CNH’s safety event reporting system is linked to the triggers program, not all triggers represent adverse events. This created dissatisfaction with the program because clinical teams felt there were many false positives. The burden was placed on clinical teams to investigate a trigger, creating nonvalue-added work. Each trigger now has clear guidelines, determined by the clinical teams, that prompt a safety report. For example, the peripheral IV infiltrate trigger identifies hyaluronidase administrations, which were sometimes not known to the clinical teams (Appendix). Before the trigger, the detection and reporting of IV infiltrates were low without much surveillance, but the addition of the trigger based on hyaluronidase administration fosters communication with clinical end users.

F. Stimulate New Trigger Development Based on Existing Trigger Platforms

CNH’s triggers team works with end users closely to continuously discuss the development process, what types of triggers may be possible, and what characteristics are likely to produce higher trigger signal. Using previous trigger logic, we can build new or adapt old triggers at an even faster rate. Fields extracted from the EHR that have been successful in previous triggers (ie, antibiotic time ordered and administered, reassessment times, laboratory results, etc.) can be used in other triggers as well. For example, the logic for administration of naloxone (opioid antagonist trigger) can be repurposed to build new triggers, such as administration of hyaluronidase for the peripheral IV infiltration and extravasation trigger (Appendix).

The primary outcome measure to evaluate the utility of CNH’s customized program is trigger signal, a term developed to track each trigger’s positive predictive value. The denominator is all reports that result from each trigger in a day. This information is reviewed using established end users’ guidance and iterative feedback to determine which reports are truly clinically useful (numerator). For example, the acute kidney injury trigger may produce a report in 2 patients who received nephrotoxic antibiotics 3 days in a row with an increase in serum creatinine (Appendix). However, if 1 patient’s creatinine is not above 0.5mg/dL (as previously determined with the acute kidney injury committee), that report would be considered a false positive, resulting in a trigger signal of 50%.

Secondary outcomes include: the number of clinical committees involved in redefining the triggers program (a surrogate marker for involvement of key stakeholders) and the number of event reports filed as a direct result of the program. Here, we use event reports as a proxy of improving patient safety, as these reports are used directly to inform safety changes and decisions in the hospital.7  As a balancing measure, it is important to ensure that with increased customization and user-centered design, the time needed to create and implement a new trigger does not increase. As a high-reliability organization, our institution places priority on not increasing nonvalue-added work or time for our employees; therefore, we want to ensure that any increase in the complexity of the trigger development process is not wasteful.

To assess study progress, we use statistical process control charts. We identified special cause variation and shifting of the centerline as outlined in the Improvement Guide: 8 or more consecutive points above or below the centerline (shift) or 6 consecutive increasing or decreasing points (trend).22 

We collected data monthly from January 2018 through August 2021 (baseline January–December 2018, implementation January–December 2019, and sustainability January 2020–August 2021). The control chart shows the cumulation of every trigger’s trigger signal (termed: “overall trigger signal”) by month (Fig 2). Trigger signal increased, surpassing the goal of doubling our baseline by December 2020. In early 2019, trigger signal shifted significantly to 44%, likely because of the implementation of several interventions (change in program structure, increasing stakeholder engagement, and development of self-service reports for end users) all at the end of 2018 or beginning of 2019 (Table 3). Additionally, 2 triggers were introduced to the program at the beginning of 2019 that use the HealtheIntent platform with more specific criteria (Table 2). There is a decrease during 2020 to 31% (annotated for hospital-level changes during the coronavirus disease 2019 pandemic related to cohorting of patients based on respiratory viral status and overall decline in census), with a recovery to 41% during 2021 as census increased. Additionally, in 2021, as the census returned to prepandemic ranges, a few triggers retired and more specific triggers were introduced to the program (Table 2).

FIGURE 2

Control chart showing increase in overall trigger signal from baseline (Jan 2018–Dec 2018), intervention period (Jan 2019–Dec 2019), and sustainability period (Jan 2021–Aug 2021).

FIGURE 2

Control chart showing increase in overall trigger signal from baseline (Jan 2018–Dec 2018), intervention period (Jan 2019–Dec 2019), and sustainability period (Jan 2021–Aug 2021).

Close modal

We identified the number of dedicated clinical committee owners overseeing each trigger, which has increased from 2 to 8 committees. One example is linking the Suspected Severe Sepsis trigger to the Improving Pediatric Sepsis Outcomes Committee, which led to refinement of that trigger over time. The number of event reports resulting from triggers increased dramatically from 9 in 2017 to 188 in 2021 (Fig 3). The balancing measure (time it takes to create and implement a trigger) decreased over the study period from approximately 8 to 12 months to 3 to 4 months.

FIGURE 3

Bar chart showing breakdown of safety event reports filed from the triggers program at Children’s National Hospital from 2017 to 2021.

FIGURE 3

Bar chart showing breakdown of safety event reports filed from the triggers program at Children’s National Hospital from 2017 to 2021.

Close modal

The growth of CNH’s triggers program is an opportunity to employ novel strategies to remain impactful for patients and providers; this evolution requires a customized, iterative methodology rather than a static, standardized program to increase trigger signal and optimize utility for the clinical teams. The interventions that have the largest impact on improving trigger signal include instituting triggers program oversight by both quality and clinical informatics, creating self-service reports for usability by clinical teams, and facilitating continuous stakeholder engagement. These interventions create sustainability in the triggers program by linking multiple departments and facilitating committees and end users to have direct input in the creation and usability of the trigger data. Shifting to multiple trigger committees and linking the triggers to specific safety event reports coincide with the coronavirus disease 2019 pandemic, where we saw a drop in patient numbers and a change in our hospital’s admission protocols. We do not feel that these interventions alone had a negative impact on trigger signal, as evidenced by the recovery at the end of the year with return to prepandemic protocols and census. Strengths of this project include the reliance on multidisciplinary teamwork and the focus on integrating electronic data with end user feedback to promote a process improvement culture that is the foundation of a high reliability organization. With improved real-time awareness of the safety events, committees can use the data obtained from the program to achieve these goals.

Although focus on retrospective harm identification is important, this tool is limited in improving patient safety; only a minority of adverse events are captured retrospectively.24  Therefore, the future of patient safety includes developing innovative safety tools (such as customized, iterative triggers programs) that incorporate information technology and real-time identification of potential adverse events.25  However, there is not currently robust literature that identifies how to improve the utility of triggers or how to adapt a triggers program.26  Recently published work by the Children’s Hospitals Solutions for Patient Safety (SPS) Network highlighted engagement with key stakeholders (including parents, clinicians, and health system leaders) to identify a research agenda for pediatric patient safety; 1 of the top-priority research topics identified was early detection of patient deterioration.27  Triggers programs serve as an example of this aspect of patient safety. Early pediatric work demonstrated that trigger-based systems performed poorly in terms of predictive performance compared with score-based systems for outcomes (ie, respiratory and/or cardiac arrest, unplanned transfer to the pediatric ICU, and/or unexpected death) and highlighted the gap in literature for how to optimize trigger-based programs.26  As shared by SPS, parent responders highlighted early detection of deterioration as key areas for patient safety research; our work with the CNH triggers program contributes to this aspect of patient safety research.27 

Most pediatric early warning systems have been developed by expert opinion alone, and only a minority have undergone any assessment of predictive validity.26,28  One challenge is defining and assessing meaningful outcomes as mortality is a rare event in hospitalized children. Measures that are associated with mortality, such as critical deterioration (defined as ICU transfer and ventilation or pressor support within 12 hours), may have potential for complementing existing patient safety measures.25  However, although validation and outcomes data are important, implementation of systems is also a crucial step. A system should have structure for communication and build consideration of risk deterioration into daily practice.26  Efforts to understand the usability, validity, and calibration of tools (including triggers) is essential because a tool that provides false alerts while missing critical deteriorations carries harm potential by triaging resources incorrectly and increasing response times through “alarm fatigue.”29 

One important novelty of this work lies in the continuous modifications of the program and the use of trigger signal to monitor clinical utility. Although a higher trigger signal may be considered better, the optimal trigger signal is not necessarily 100%; a high trigger signal may indicate an overly narrow definition, producing data that are not sensitive enough, missing precursor cases of interest, and eroding trust in the program. We have seen tremendous increase in end user engagement with an average trigger signal of 40%, however, more investigation, not only at CNH, but within pediatrics and healthcare at large, needs to occur to determine the optimal trigger signal for a triggers program. Our goal initially was to increase trigger signal by doubling our baseline rate (16% from baseline of 8%), but through iterations of our triggers program over the last 3 years, it has become clear that a trigger signal of under 20% is still not sufficient. Although there are no known benchmarks, we consider individual trigger signal to be optimized once a trigger is deemed clinically meaningful and actionable by the clinical end users. This may be shown by the real-time patient care changes and increased number of event reports resulting from the program that in turn allows for process improvement and patient safety changes. Our program’s trigger data fuels many of the improvement initiatives throughout the organization, with data continuously being used for CNH’s preventable harm index.21 

CNH’s trigger tool is a continually evolving system, and the future lies in expanding the program in clinically meaningful directions. First, the data that arises from trigger reports has created several new quality improvement projects to improve patient safety, including surveillance of sepsis and monitoring nephrotoxic medications in acute kidney injury. Second, we are looking to expand to areas outside of inpatient care, such as ambulatory and surgical centers, as this has been identified as a pediatric safety research priority.27  In addition, novel triggers can be implemented using a framework to detect nonbinary information such as social determinants of health and socioeconomic risk factors, perhaps identifying patients at who are at-risk for poorer health outcomes. From this program, we hope to improve outcomes in safety events, including reduction in peripheral IV infiltrations, nephrotoxic acute kidney injuries, and unplanned transfers, along with improving sepsis-related outcomes and processes.

Limitations of this work and its generalizability include that the resources dedicated to triggers may not exist at other programs. For institutions with limited resources, we recommend focusing on 1 or 2 key triggers or trends identified from event reports and/or institutional priorities. Additionally, this work takes place at a single institution, but future work can involve multiple hospitals. Finally, there is no singular outcome that directly links an increase in trigger utility to an improved patient safety experience. Despite this, our institution believes that the triggers program is useful to identify improvement areas, therefore we have committed time and effort to improve the utility of this program and are evaluating more direct patient safety outcomes to track.

The triggers program at CNH shows that there can be successful improvement of clinical support by optimizing trigger signal to provide actionable, near real- time, surveillance tools while encouraging a culture of process improvement. Trigger signal may be a measure that hospitals can use to track the success of their programs; however, more studies are required to develop the optimal trigger signal percentage. Linking the oversight of a triggers program with quality improvement and clinical informatics, improving stakeholder engagement, and developing trigger reports that are easily available for end users improved our trigger signal. Hospitals of comparable complexity and size can implement a triggers program and see similar success by demonstrating the clinical significance of their trigger tools.

We thank the current leads of each trigger tool committee for their work to improve patient safety at Children’s National Hospital: Dr. Nada Mallick and Rosemary Szeles of Late Rescue and Codes Outside the ICU Reduction Committee; Jacqueline Harnarine, Dr. Aadil Kakajiwala, and Dr. Marva Moxey-Mims of the Nephrotoxic Acute Kidney Injury Prevention Committee; Sofia Perazzo and Smitha Israel of the NICU; Daniela Herrera of the Pain Committee; Alia Fink of the Improving Pediatric Sepsis Collaborative at Children’s National Hospital; Dr. Andrew Matisoff of the Sedation Committee; and Eliana Maldonado of the peripheral IV infiltration and extravasation Prevention Committee. We would also like to acknowledge the Quality and Safety Department for their guidance and involvement in the triggers program, Dr. Eva Rubio and Katherine Moore.

Drs Reinhart and Parikh contributed to the creation of quality improvement measures, development of charts, and writing of the manuscript; Mrs Safari-Ferra is the Program Lead for the triggers program, and outside of her critical work in customizing the program, she contributed to the creation of outcome measures, development of charts, and writing of the manuscript; Mr Badh contributed to the automaticity of the triggers program via data sets and logic creation and moving to the HealtheIntent platform; Mr Bhattarai contributed to the creation of new triggers aligned with external data submission goals and process improvement measures for logic creation; Mr Abera is a leader in the clinical logic creation of triggers and alignment with clinical end users, a main focus of the customization of the Children’s National program; Mrs Saha is the Director of Performance Improvement, thus she contributed to the change in the process improvement workflow of the triggers program and standardization with the safety event reporting system; Dr Herstek is the Chief Medical Informatics Officer at Children’s National, therefore she led the conversion to the Electronic Data Warehouse and the increased automaticity of the triggers program, and also contributed directly to the writing of this manuscript; Dr Shah served as the Chief Quality Officer at Children’s National, oversaw the triggers program with a primary leadership role in the customization process, and contributed to and edited this manuscript at various stages; all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

CONFLICT OF INTEREST DISCLOSURES: Mr Bhattarai indicates owning stock of several healthcare companies without any significant contribution or involvement as a shareholder; Dr Parikh owns stock in RLDatix; and the other authors have no conflicts of interest relevant to this article to disclose.

CNH

Children’s National Hospital

EDW

electronic data warehouse

EHR

electronic health record

GAPPS

Global Assessment of Pediatrics Patient Safety

IHI

Institute for Healthcare Improvement

1
Makary
MA
,
Daniel
M
.
Medical error-the third leading cause of death in the US
.
BMJ
.
2016
;
353
:
i2139
2
James
JT
.
A new, evidence-based estimate of patient harms associated with hospital care
.
J Patient Saf
.
2013
;
9
(
3
):
122
128
3
Institute of Medicine Committee on Quality of Health Care in A
. In;
Kohn
LT
,
Corrigan
JM
,
Donaldson
MS
, eds.
To Err is Human: Building a Safer Health System
.
Washington, DC
:
National Academies Press
;
2000
4
Brady
PW
,
Varadarajan
K
,
Peterson
LE
,
Lannon
C
,
Gross
T
.
Prevalence and nature of adverse medical device events in hospitalized children
.
J Hosp Med
.
2013
;
8
(
7
):
390
393
5
Stockwell
DC
,
Landrigan
CP
,
Toomey
SL
, et al;
GAPPS Study Group
.
Adverse events in hospitalized pediatric patients
.
Pediatrics
.
2018
;
142
(
2
):
e20173360
6
Russell
CJ
,
Simon
TD
.
Care of children with medical complexity in the hospital setting
.
Pediatr Ann
.
2014
;
43
(
7
):
e157
e162
7
Crandall
KM
,
Almuhanna
A
,
Cady
R
, et al
.
10,000 good catches: increasing safety event reporting in a pediatric health care system
.
Pediatr Qual Saf
.
2018
;
3
(
2
):
e072
8
Davis
J
,
Harrington
N
,
Bittner Fagan
H
,
Henry
B
,
Savoy
M
.
The accuracy of trigger tools to detect preventable adverse events in primary care: a systematic review
.
J Am Board Fam Med
.
2018
;
31
(
1
):
113
125
9
Epstein
EG
,
Arechiga
J
,
Dancy
M
,
Simon
J
,
Wilson
D
,
Alhusen
JL
.
Integrative review of technology to support communication with parents of infants in the NICU
.
J Obstet Gynecol Neonatal Nurs
.
2017
;
46
(
3
):
357
366
10
Thurtle
DP
,
Daffron
SB
,
Halvorson
EE
.
Patient characteristics associated with voluntary safety event reporting in the acute care setting
.
Hosp Pediatr
.
2019
;
9
(
2
):
134
138
11
Stockwell
DC
,
Landrigan
CP
,
Toomey
SL
, et al;
GAPPS Study Group
.
Racial, ethnic, and socioeconomic disparities in patient safety events for hospitalized children
.
Hosp Pediatr
.
2019
;
9
(
1
):
1
5
12
Spencer
CS
,
Roberts
ET
,
Gaskin
DJ
.
Differences in the rates of patient safety events by payer: implications for providers and policymakers
.
Med Care
.
2015
;
53
(
6
):
524
529
13
Khan
A
,
Yin
HS
,
Brach
C
, et al;
Patient and Family Centered I-PASS Health Literacy Subcommittee
.
Association between parent comfort with English and adverse dvents among hospitalized children
.
JAMA Pediatr
.
2020
;
174
(
12
):
e203215
14
Silva
MDDG
,
Martins
MAP
,
Viana
LG
, et al
.
Evaluation of accuracy of IHI trigger tool in identifying adverse drug events: a prospective observational study
.
Br J Clin Pharmacol
.
2018
;
84
(
10
):
2252
2259
15
Halvorson
EE
,
Irby
MB
,
Skelton
JA
.
Pediatric obesity and safety in inpatient settings: a systematic literature review
.
Clin Pediatr (Phila)
.
2014
;
53
(
10
):
975
987
16
Landrigan
CP
,
Stockwell
D
,
Toomey
SL
, et al
.
Performance of the Global Assessment of Pediatric Patient Safety (GAPPS) tool
.
Pediatrics
.
2016
;
137
(
6
):
e20154076
17
Murphy
DR
,
Meyer
AN
,
Sittig
DF
,
Meeks
DW
,
Thomas
EJ
,
Singh
H
.
Application of electronic trigger tools to identify targets for improving diagnostic safety
.
BMJ Qual Saf
.
2019
;
28
(
2
):
151
159
18
Griffin
FA
,
Resar
RK
.
IHI Global Trigger Tool for Measuring Adverse Events
.
Cambridge, MA
:
Institute for Healthcare Improvement
;
2009
:
1
43
19
Quinonez
RA
,
Schroeder
AR
.
“GAPPS” in patient safety
.
Pediatrics
.
2018
;
142
(
2
):
e20180954
20
O’Leary
KJ
,
Devisetty
VK
,
Patel
AR
, et al
.
Comparison of traditional trigger tool to data warehouse based screening for identifying hospital adverse events
.
BMJ Qual Saf
.
2013
;
22
(
2
):
130
138
21
Bhattarai
S
,
Shah
RK
.
Leveraging real-time data to drive quality improvement in a pediatric health system
.
Pediatr Investig
.
2018
;
2
(
3
):
184
187
22
Langley
GJ
,
Moen
RD
,
Nolan
KM
,
Nolan
TW
,
Norman
CL
,
Provost
LP
.
The Improvement Guide: A Practical Approach to Enhancing Organizational Performance
.
San Francisco, CA
:
Jossey-Bass
;
2009
23
Ogrinc
G
,
Davies
L
,
Goodman
D
,
Batalden
P
,
Davidoff
F
,
Stevens
D
.
SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process
.
BMJ Qual Saf
.
2016
;
25
(
12
):
986
992
24
Classen
DC
,
Lloyd
RC
,
Provost
L
,
Griffin
FA
,
Resar
R
.
Development and evaluation of the institute for healthcare improvement global trigger tool
.
Journal of Patient Safety
.
2008
;
4
(
3
):
169
177
25
Bonafide
CP
,
Roberts
KE
,
Priestley
MA
, et al
.
Development of a pragmatic measure for evaluating and optimizing rapid response systems
.
Pediatrics
.
2012
;
129
(
4
):
e874
e881
26
Chapman
SM
,
Wray
J
,
Oulton
K
,
Pagel
C
,
Ray
S
,
Peters
MJ
.
‘The Score Matters’: wide variations in predictive performance of 18 paediatric track and trigger systems
.
Arch Dis Child
.
2017
;
102
(
6
):
487
495
27
Hoffman
JM
,
Keeling
NJ
,
Forrest
CB
, et al
.
Priorities for pediatric patient safety research
.
Pediatrics
.
2019
;
143
(
2
):
e20180496
28
Chapman
SM
,
Grocott
MP
,
Franck
LS
.
Systematic review of paediatric alert criteria for identifying hospitalised children at risk of critical deterioration
.
Intensive Care Med
.
2010
;
36
(
4
):
600
611
29
Bonafide
CP
,
Lin
R
,
Zander
M
, et al
.
Association between exposure to nonactionable physiologic monitor alarms and response time in a children’s hospital
.
J Hosp Med
.
2015
;
10
(
6
):
345
351

Supplementary data