OBJECTIVES:

To determine if an improvement collaborative of 33 children’s hospitals focused on reliable best practice implementation and culture of safety improvements can reduce hospital-acquired conditions (HACs) and serious safety events (SSEs).

METHODS:

A 3-year prospective cohort study design with a 12-month historical control population was completed by the Children’s Hospitals’ Solutions for Patient Safety collaborative. Identification and dissemination of best practices related to 9 HACs and SSE reduction focused on key process and culture of safety improvements. Individual hospital improvement teams leveraged the resources of a large, structured children’s hospital collaborative using electronic, virtual, and in-person interactions.

RESULTS:

Thirty-three children’s hospitals from across the United States volunteered to be part of the Children’s Hospitals’ Solutions for Patient Safety collaborative. Thirty-two met all the data submission eligibility requirements for the HAC improvement objective of this study, and 21 participated in the high-reliability culture work aimed at reducing SSEs. Significant harm reduction occurred in 8 of 9 common HACs (range 9%–71%; P < .005 for all). The mean monthly SSE rate decreased 32% (from 0.77 to 0.52; P < .001). The 12-month rolling average SSE rate decreased 50% (from 0.82 to 0.41; P < .001).

CONCLUSIONS:

Participation in a structured collaborative dedicated to implementing HAC-related best-practice prevention bundles and culture of safety interventions designed to increase the use of high-reliability organization practices resulted in significant HAC and SSE reductions. Structured collaboration and rapid sharing of evidence-based practices and tools are effective approaches to decreasing hospital-acquired harm.

Hospitalized patients are harmed at concerning rates. In 1999, the National Academy of Medicine (then called the Institute of Medicine) reported that medical errors cause up to 98 000 patient deaths annually,1 with more recent studies suggesting that number may be closer to 400 000.2 Nonfatal harm is estimated to occur in up to one-third of hospitalized patients.3,4 Hospitalized children are equally vulnerable; adverse events in children’s hospitals occur at a rate of 40 per 100 discharges, with almost 50% of these harms deemed preventable.5 

Isolated examples demonstrate that harm reduction is achievable. Several pediatric quality improvement (QI) collaboratives have demonstrated significant harm reduction by implementing high-reliability organization (HRO) principles6 in combination with QI methodologies.7,9 These principles come from an analysis of characteristics of industries that exist in extremely high-risk and complex environments but have much lower than expected rates of harm. Despite skepticism about study design and methodology used in QI collaboratives to assess effectiveness,10,11 the collaborative approach has frequently yielded valid harm reduction in hospitalized patients.6,12,14 

One collaborative that demonstrated sustained harm reduction was the Ohio Children’s Hospitals’ Solutions for Patient Safety (SPS) network. Guiding principles for this collaborative of 8 children’s hospitals included application of HRO principles, data transparency, and sharing best practices to maximize the synergy between improving processes and institutional safety culture. This collaborative reduced surgical site infections by 60%12 and by 2011 had reduced serious safety events (SSEs) and serious harm events by 55% and 40%, respectively.6 With support from the Centers for Medicare and Medicaid Services’ Partnership for Patients initiative, the Ohio network added 25 children’s hospitals from around the United States in 2012 to form the Children’s Hospitals’ SPS hospital engagement network. This study describes the collaborative interventions and impact on hospital-acquired conditions (HACs) and SSE rates (SSERs).

To evaluate the effect of networkwide implementation of HAC- and SSE-reduction practices, we conducted a prospective cohort study of 33 SPS children’s hospitals (Supplemental Table 4) that ranged in size, clinical scope, and geographical region. Each had independent experience with patient safety improvement work and HAC-reduction efforts. This project was reviewed, and consent was waived by the Cincinnati Children’s Hospital Institutional Review Board. Each hospital paid $15 000 as an annual participation fee, which helped offset a ∼$2 500 000 annual SPS network budget that covered in-person and virtual learning, data management and analysis, QI expertise, and staff.

The SPS leadership oversaw identification and dissemination of best practices related to HAC and SSE reduction. Each participating hospital was asked to incorporate these best practices by using local QI methods, such as the Model for Improvement15 or Lean Six Sigma,16 and leverage the structure and resources provided by the collaborative to accelerate and sustain improvement. At the collaborative level, improvement strategies were designed by using the Model for Improvement, including aims, metrics, and key drivers.

The 9 HACs targeted for improvement are listed in Table 1. A leadership team for each HAC was responsible for adopting a specific aim and identifying evidence-based best practices, which were bundled together to produce a cadre of practices (a “bundle”) to disseminate to participating hospitals.17 Hospitals were encouraged to establish local improvement teams for each HAC and instructed to target 90% compliance to the bundles. As such, hospitals identified team leaders for each HAC, assembled multidisciplinary teams, developed HAC-reduction goals that aligned with the collaborative’s goals, and then planned, implemented, and measured tests of change designed to reduce hospital-acquired harm. These tests primarily focused on the institution and reliable use of the evidence-based bundles identified at the network level.

TABLE 1

HAC and SSE Operational Definitions

Harm TypeMeasurementNotes
Adverse drug events  No. of adverse drug events per 1000 patient days Preventable F-I events on NCC MERP scale18  
CAUTIs No. of urinary tract infections per 1000 urinary catheter days Operational definition per NHSN19  
Central line-associated blood stream infections  No. of blood stream infections per 1000 central line days Operational definition per NHSN20  
Falls, moderate or greater harm No. of falls of moderate or greater severity per 1000 patient days Fall severity defined by NDNQI 
Obstetric adverse events No. of obstetric adverse events per 100 births Per CMS 
Surgical site infections No. of surgical site infections per 100 surgical procedures Cardiothoracic, spinal fusion, ventriculoperitoneal shunt procedures; operational definition per NHSN21  
Pressure ulcers No. of pressure ulcers per 1000 patient days Includes stages 3, 4, and unstageable; operational definition per NDNQI22  
Ventilator-associated pneumonia No. of ventilator-associated pneumonia infections per 1000 ventilator days Operational definition per NHSN23  
SSER 12-mo rolling average No. of SSEs per 10 000 adjusted patient days in previous 12 mo SSEs are harms that result in death, severe permanent harm, moderate permanent harm, or severe or moderate temporary harm resulting from deviations in standards of care 
SSE monthly rate No. of SSEs divided by 10 000 adjusted patient days in a given month — 
VTE events No. of VTE events per 1000 patient days — 
Harm TypeMeasurementNotes
Adverse drug events  No. of adverse drug events per 1000 patient days Preventable F-I events on NCC MERP scale18  
CAUTIs No. of urinary tract infections per 1000 urinary catheter days Operational definition per NHSN19  
Central line-associated blood stream infections  No. of blood stream infections per 1000 central line days Operational definition per NHSN20  
Falls, moderate or greater harm No. of falls of moderate or greater severity per 1000 patient days Fall severity defined by NDNQI 
Obstetric adverse events No. of obstetric adverse events per 100 births Per CMS 
Surgical site infections No. of surgical site infections per 100 surgical procedures Cardiothoracic, spinal fusion, ventriculoperitoneal shunt procedures; operational definition per NHSN21  
Pressure ulcers No. of pressure ulcers per 1000 patient days Includes stages 3, 4, and unstageable; operational definition per NDNQI22  
Ventilator-associated pneumonia No. of ventilator-associated pneumonia infections per 1000 ventilator days Operational definition per NHSN23  
SSER 12-mo rolling average No. of SSEs per 10 000 adjusted patient days in previous 12 mo SSEs are harms that result in death, severe permanent harm, moderate permanent harm, or severe or moderate temporary harm resulting from deviations in standards of care 
SSE monthly rate No. of SSEs divided by 10 000 adjusted patient days in a given month — 
VTE events No. of VTE events per 1000 patient days — 

CMS, Center for Medicare and Medicaid Services; NCC MERP, National Coordinating Council for Medication Error Reporting and Prevention; NDNQI, National Database of Nursing Quality Indicators; NHSN, National Healthcare Safety Network; —, not applicable.

Culture of safety best practices recommended by SPS leadership were similar to those successfully used by the Ohio collaborative to reduce Ohio’s statewide SSERs.6 These collective tools and techniques, known as the “culture work,” aimed to change attitudes and behaviors of hospital personnel within the following 4 domains: error-prevention behaviors, leadership methods, cause analysis, and patient–family engagement. Several of these practices were modeled after commercially available tools,24 and others were developed by SPS leaders.

Error-prevention behaviors were taught to all hospital personnel in classes facilitated by hospital educators trained by SPS. In addition, SPS hospitals developed a peer-to-peer safety coach program and a hospital leader training program. Leaders were taught to engage in structured unit-based safety rounds, implement routine safety huddles, and establish daily organization-wide safety briefings. SPS experts trained a team from each hospital in a standardized root cause analysis process. The collaborative supported recruitment and inclusion of parents on hospital board committees, hospital-level quality and safety committees, and local HAC teams. The Child Health Patient Safety Organization afforded participating hospitals the ability to transparently discuss serious harm event details, subsequent analysis, and follow-up actions with colleagues at other hospitals.

Adhering to the philosophical tenet of “all teach, all learn,” multiple opportunities for collaboration across the network were offered. Opportunities included semiannual, in-person learning sessions; monthly webinars on topics including safety culture, QI science, and HRO theory; discussions on a password-protected Web site; outcome and process data reports for chief executive officers and quality leaders; and published reports that highlighted which hospitals were achieving the best results in each category. The SPS support staff consisted of quality consultants, project managers, and data specialists who provided guidance and analytics support to the network. Informational webinars for each HAC occurred monthly, and webinars focusing on the implementation of HRO principles occurred weekly.

The primary outcome measures were HACs and SSERs over time. HACs were identified by the Centers for Medicaid and Medicare Services and defined when possible by using established standards from national organizations (Table 1). When standard definitions were not available, operational definitions were established by expert pediatric consensus. All operational definitions were consistent throughout the study period. Each hospital submitted monthly HAC outcome data for the following 3 time periods: a 12-month “pre-SPS” period (April 1, 2011–March 31, 2012); a 3-month “ramp-up” period (April 1, 2012–June 30, 2012); and a 33-month “post-SPS” period (July 1, 2012–March 31, 2015). The pre-SPS period included retrospective baseline data for the 12 months before the beginning of SPS. Only hospitals that submitted baseline data were included in the analysis. During the ramp-up period, hospitals organized prospective data collection processes and began bundle implementation.

SSEs were defined as serious patient harm events resulting from standard of care deviation.16 Examples include a surgical procedure performed on the wrong patient or 10-fold opiate overdose resulting in respiratory arrest. Hospitals were trained to detect SSEs by all means, including incident reports, legal activity, patient complaints, and quality-assurance processes at the department and unit levels. SSER reduction has been associated with organizational safety culture improvement.25,29 The monthly SSER was defined as the number of SSEs divided by the number of adjusted patient days in a given month. In addition, a 12-month SSER rolling average was calculated with the following: the number of SSEs in the last 12 months divided by the total number of adjusted patient days. Hospitals were included in the SSE analysis if they submitted SSE data beginning in 2011 and completed culture training by December 31, 2013. In alignment with previous work by researchers using this metric, improvement was measured from the maximum study period rate and thereby accounting for rate changes that occurred as a result of better detection processes.17,24,27 

Participating hospitals were expected to submit monthly bundle compliance data for each HAC. Because several SPS-proposed HAC bundles did not have a comprehensive evidence basis in 2011, hospitals were permitted initially to either use SPS bundles or develop their own. However, after June 2014, SPS endorsed specific bundles for central line-associated blood stream infections, surgical site infections, catheter-associated urinary tract infections (CAUTIs), pressure ulcers, and ventilator-associated pneumonia. Hospitals were expected to adopt them and instructed to submit at least 20 bundle compliance audits per month for each HAC. Bundle compliance was defined as adherence to all elements of the bundle. Central line-associated blood stream infections and CAUTIs had 2 compliance metrics that correlated with dual prevention bundles, one for device insertion and one for device maintenance.

Collaborative participation was estimated by using the frequency of attendance on webinars, frequency of presentations on webinars, and attendance at both face-to-face learning sessions and 1 or more in-person board training sessions.

HAC Outcome and Process Analysis

Monthly HAC outcome data were collected from each hospital using a Web-based form and were defined as the number of HAC events per month per 1000 relevant exposure units (eg, patient days and catheter days). Monthly process data reflecting compliance with the specific HAC bundles were defined as the number of observations of perfect bundle execution divided by the total number of observations. Hospitals were additionally categorized based on whether they were reporting pre-SPS compliance data to determine the impact of this level of organizational attention on their initial outcome rates or rate change over time.

Monthly data were analyzed from April 2011 to March 2015. To examine the HAC outcome data and account for temporal correlation within individual hospitals, generalized Poisson mixed effects modeling with a log link was used. Random hospital effects were included to address any temporal correlation of observations within hospitals. The analytic model included indicator variables for SPS periods and allowed the estimation of HAC rates pre- and post-SPS centerlines as well as the estimation of relative rate changes from pre- to post-SPS through exponentiated regression coefficients interpreted as relative rates. Computed P values tested the null hypothesis of no change in HAC outcome rates from pre- to post-SPS. To illustrate monthly change in HAC rates during the 3 study periods, monthly mean profile plots of HACs over the study period and estimated lowess (locally weighted scatterplot smoother) curves for nonparametric data were produced.30 

To illustrate hospital-level variation in HAC rate changes from pre- to post-SPS and to identify the percentage of hospitals that improved after SPS initiation, we computed pre- and post-period HAC rates for each hospital and graphically display the change. We did this for each HAC overall and stratified by reporting of bundle compliance data.

SSE Analysis

The SSER was calculated in the following 2 ways: by using a 12-month retrospective rolling average and by using a monthly rate. To estimate improvement, 12 months of data from before the peak rate (December 2013) were used to establish the baseline, which was then compared with the most recent 12 months (postintervention period) by using a Wilcoxon rank test of the change. For analysis of monthly rates, an overall baseline and postintervention period of SSEs were estimated, and the change between periods through a Wilcoxon rank test was compared. A u-chart based on count data with a Poisson distribution to visually describe the trend for monthly SSEs was created. All statistical tests were 2-sided, and significance was defined as a P < .05. Statistical analyses were performed by using SAS v.9.3 and R v.3.1.1.

Thirty-three hospitals participated in the SPS collaborative. One site was excluded from the analyses because of failure to provide baseline data. Of the remaining 32, 21 (63.6%) were freestanding children’s hospitals, and 29 (90.6%) had academic affiliations. The mean number of hospital beds was 330 (range = 75–628), and the mean number of admissions varied from 13 016 (range = 4083–31 262) in 2011 to 15 713 (range = 1952–35 054) in 2014. The hospitals represented 19 states and the District of Columbia, 15 of which were in the Midwest, 5 in the Northeast, 7 in the South, and 5 in the West.

Significant reductions occurred in 8 of the 9 HACs after initiation of the SPS collaborative (Fig 1, Supplemental Fig 4–7). The relative rate reduction from pre- to post-SPS for the 8 improved HACs ranged from 9% to 71% (Table 2). Depending on the HAC, between 50% and 69% of hospitals had significant HAC reductions. Monthly individual HAC rates decreased after the ramp-up period for all HACs except venous thromboembolism (VTE).

FIGURE 1

Adverse drug event and CAUTI rates during pre- and post-SPS periods. A, Adverse drug events. B, CAUTIs. See Supplemental Fig 4 for other HAC run charts.

FIGURE 1

Adverse drug event and CAUTI rates during pre- and post-SPS periods. A, Adverse drug events. B, CAUTIs. See Supplemental Fig 4 for other HAC run charts.

TABLE 2

HAC and SSE Outcomes Pre- and Post-SPS Periods

HAC (Unit of Rate)Pre-SPS Rate Per 1000 UPost-SPS Rate Per 1000 URelative Rate Change From Pre- to Post-SPS, %aPHospitals With Improvement in Rate, %b
ADEs (patient days) 0.044 0.029 −34c <.001c 50 
CAUTIs (catheter days) 2.601 1.738 −33c <.001c 69 
CLABSIs (central line days) 1.331 1.206 −9c .005c 59 
Falls (patient days) 0.036 0.011 −71c <.001c 66 
OBAEs (births)d 16.98 7.48 −56c <.001c 67 
PUs (patient days) 0.107 0.082 −24c <.001c 55 
SSIs (procedures) 24.35 18.39 −24c <.001c 61 
VAP (ventilator days) 0.552 0.384 −30c <.001c 65 
VTE events (patient days) 0.237 0.230 −3 .515 44 
SSEs (adjusted patient days) 0.082 0.066 −19c <.001e 72 
HAC (Unit of Rate)Pre-SPS Rate Per 1000 UPost-SPS Rate Per 1000 URelative Rate Change From Pre- to Post-SPS, %aPHospitals With Improvement in Rate, %b
ADEs (patient days) 0.044 0.029 −34c <.001c 50 
CAUTIs (catheter days) 2.601 1.738 −33c <.001c 69 
CLABSIs (central line days) 1.331 1.206 −9c .005c 59 
Falls (patient days) 0.036 0.011 −71c <.001c 66 
OBAEs (births)d 16.98 7.48 −56c <.001c 67 
PUs (patient days) 0.107 0.082 −24c <.001c 55 
SSIs (procedures) 24.35 18.39 −24c <.001c 61 
VAP (ventilator days) 0.552 0.384 −30c <.001c 65 
VTE events (patient days) 0.237 0.230 −3 .515 44 
SSEs (adjusted patient days) 0.082 0.066 −19c <.001e 72 

ADE, adverse drug event; CLABSI, central-line associated blood stream infection; OBAE, obstetric adverse event; PU, pressure ulcer; SSI, surgical site infection; VAP, ventilator-associated pneumonia.

a

A generalized Poisson mixed model was used to estimate relative rate change from pre- to post-SPS and to test for significant change.

b

Improvement is defined as any rate decrease from baseline.

c

Denotes a statistically significant reduction in rate from pre- to post-SPS.

d

Only 6 SPS hospitals have obstetric services and report data on this measure.

e

A Wilcoxon rank test was used for statistical significance.

Twenty-one of the 32 eligible hospitals participated in the culture work to reduce SSEs. Of the original 32, 10 hospitals had not completed the required culture training by December 31, 2013, and thus were excluded from the SSE outcome analysis. Additionally, 1 hospital did not submit SSE data in 2014 and was excluded. The 12-month rolling average SSER of the remaining 21 hospitals decreased by 50% from a peak of 0.82 per 10 000 adjusted patient days in March 2012 to 0.41 in March 2015 (P < .001) (Fig 2A). Of the 21 participating hospitals, 15 had an SSER reduction (percent reduction range = 28–100; mean 62), and 6 had an increase (percent increase range = 3–211; mean 78). The monthly number of SSEs per 10 000 adjusted patient days also decreased from a mean of 0.77 to 0.52 (P < .001), a 32% reduction (Fig 2B).

FIGURE 2

SPS SSERs. A, Twelve-month rolling average of the number of SSEs per 10 000 adjusted patient days. B, Monthly number of SSEs per 10 000 adjusted patient days. a All hospitals completed culture training.

FIGURE 2

SPS SSERs. A, Twelve-month rolling average of the number of SSEs per 10 000 adjusted patient days. B, Monthly number of SSEs per 10 000 adjusted patient days. a All hospitals completed culture training.

Among hospitals that reported both pre- and post-SPS compliance data, compliance to HAC bundles increased (Table 3; range = 0.4%–26.6%). No hospitals had complete pre-SPS compliance data for every HAC, and no hospitals were initially measuring obstetric adverse events and VTE compliance. Excluding obstetric adverse events, because this HAC is only relevant to 6 of the participating children’s hospitals, the number of hospitals reporting bundle compliance to any HAC increased from a pre-SPS mean of 7.3 (range = 0–17) to a post-SPS mean of 27.8 (range = 19–32).

TABLE 3

SPS Collaborative Compliance With HAC Bundles

HAC BundleNo. of Hospitals Reporting Compliance BundlesHospitals Reporting Both Pre- and Post-SPS Bundle ComplianceHospitals Reporting Only Post-SPS Bundle Compliance
Pre-SPSPost-SPSN HospitalsPre-SPS: Mean Compliance, %Post-SPS: Mean Compliance, %Absolute Mean Change from Pre- to Post-SPS, %No. of HospitalsPost-SPS: Mean Compliance, %
ADEs 27 92.4 93.0 +0.6 20 91.2 
CAUTIsa 27 73.0 92.2 +19.2 23 74.5 
CAUTI-IB 25 — — — — 25 84.4 
CAUTI-MB 32 — — — — 32 71.2 
CLABSI-lB 14 31 14 78.7 85.4 +6.7 17 95.9 
CLABSI-MBa 17 32 17 57.1 83.7 +26.6 15 83.2 
Falls 27 79.3 91.2 +11.9 24 65.8 
OBAEsb — — — — 76.4 
PUs 28 95.5 96.0 +0.5 26 68.4 
SSIs 11 30 11 89.7 90.1 +0.4 19 95.6 
VAP 29 84.5 87.4 +2.9 21 80.9 
VTE events 19 — — — — 19 89.2 
HAC BundleNo. of Hospitals Reporting Compliance BundlesHospitals Reporting Both Pre- and Post-SPS Bundle ComplianceHospitals Reporting Only Post-SPS Bundle Compliance
Pre-SPSPost-SPSN HospitalsPre-SPS: Mean Compliance, %Post-SPS: Mean Compliance, %Absolute Mean Change from Pre- to Post-SPS, %No. of HospitalsPost-SPS: Mean Compliance, %
ADEs 27 92.4 93.0 +0.6 20 91.2 
CAUTIsa 27 73.0 92.2 +19.2 23 74.5 
CAUTI-IB 25 — — — — 25 84.4 
CAUTI-MB 32 — — — — 32 71.2 
CLABSI-lB 14 31 14 78.7 85.4 +6.7 17 95.9 
CLABSI-MBa 17 32 17 57.1 83.7 +26.6 15 83.2 
Falls 27 79.3 91.2 +11.9 24 65.8 
OBAEsb — — — — 76.4 
PUs 28 95.5 96.0 +0.5 26 68.4 
SSIs 11 30 11 89.7 90.1 +0.4 19 95.6 
VAP 29 84.5 87.4 +2.9 21 80.9 
VTE events 19 — — — — 19 89.2 

Compliance is defined as adherence to all elements of the bundle. ADE, adverse drug event; CLABSI, central line-associated blood stream infection; IB, insertion bundle; MB, maintenance bundle; OBAE, obstetric adverse event; PU, pressure ulcer; SSI, surgical site infection; VAP, ventilator-associated pneumonia; —, not applicable.

a

2011–2013 CAUTI compliance was counted as single measure; 2014–2015 measure was subdivided between compliance with the insertion bundle and maintenance bundle.

b

Only 6 SPS hospitals have obstetric services and report data on this measure.

To examine the relationship between bundle compliance and outcomes, pre- and post-SPS HAC outcome data were divided into categories based on bundle compliance rates (Supplemental Table 5). Hospitals that were not reporting bundle compliance data in the pre-SPS period but did in the post-SPS period had the most significant reductions in HAC rates. For example, improvement in adverse drug event rates for the 20 hospitals without baseline compliance data but with post-SPS compliance data was 41% (P < .001), whereas improvement in adverse drug event rates across all hospitals was 34%. A similar trend was observed for 6 other HACs. For hospitals that did not report bundle compliance in either the pre- or post-periods, no change was observed from pre- to post-SPS HAC outcomes.

To examine the level of variation in HAC rates between hospitals before SPS initiation and the variation in improvement after SPS, we computed pre- and post-period HAC rates for each hospital and graphically display the change (Fig 3, Supplemental Fig 8–11). Hospitals varied in their pre-SPS performance across all HACs. The percent of hospitals that improved after SPS initiation varied across HACs from 44% (VTE) to 69% (CAUTIs). Overall, the largest improvements in HAC rates were seen for hospitals that did not have bundle compliance reporting before SPS and began reporting after starting SPS. For example, the overall percentage of hospitals that improved for CAUTIs was 69%, with hospitals that started reporting bundle compliance after SPS initiation reporting 83% improvement. Lastly, the denominator for each HAC was averaged across the study period, and we report the median and interquartile range of the average denominators across hospitals in Fig 3 and Supplemental Fig 8–11. Denominators varied slightly across hospitals but didn’t vary drastically across compliance bundle groups.

FIGURE 3

Hospital-specific adverse drug events and CAUTIs mean rates from pre- and post-SPS periods overall and stratified by reporting of bundle compliance data. A, Adverse drug events. B, CAUTIs. See Supplemental Fig 5 for other HAC hospital figures. Each line represents a unique children’s hospital. A blue line suggests improvement in HAC rates from pre- to post-SPS, and a red line suggests worsening in rates from pre- to post-SPS. The black line is the overall rate change from pre- to post-SPS. For each hospital, the denominator for each HAC rate was averaged across the study period. “Denominator median” reports the median of the average denominators across hospitals, and “denominator IQR” denotes the interquartile range. Hospitals with no reporting of bundle compliance before SPS that started reporting bundle compliance after SPS are denoted as “Pre/+Post.” Hospitals that were reporting bundle compliance pre- and post-SPS are denoted as “+Pre/+Post.” Hospitals with no pre- or post-SPS reporting of bundle compliance are denoted as “−Pre/−Post.”

FIGURE 3

Hospital-specific adverse drug events and CAUTIs mean rates from pre- and post-SPS periods overall and stratified by reporting of bundle compliance data. A, Adverse drug events. B, CAUTIs. See Supplemental Fig 5 for other HAC hospital figures. Each line represents a unique children’s hospital. A blue line suggests improvement in HAC rates from pre- to post-SPS, and a red line suggests worsening in rates from pre- to post-SPS. The black line is the overall rate change from pre- to post-SPS. For each hospital, the denominator for each HAC rate was averaged across the study period. “Denominator median” reports the median of the average denominators across hospitals, and “denominator IQR” denotes the interquartile range. Hospitals with no reporting of bundle compliance before SPS that started reporting bundle compliance after SPS are denoted as “Pre/+Post.” Hospitals that were reporting bundle compliance pre- and post-SPS are denoted as “+Pre/+Post.” Hospitals with no pre- or post-SPS reporting of bundle compliance are denoted as “−Pre/−Post.”

An average of 20 (62.5%) hospitals attended each of the 205 available webinars, and 100% of hospitals served as teachers of 1 or more webinars or learning sessions. From 2012 to 2014, 97% of hospitals had at least 1 representative at each of the 4 national learning sessions. Sixty-six percent of hospitals sent at least 1 board member to an in-person board training session. During the intervention period, >600 discussions occurred on the SPS password-protected online discussion board.

To our knowledge, this is the first report in which researchers describe a significant reduction in patient harm (HACs and SSEs) in a cohort of children’s hospitals participating in a structured collaborative network. Our results suggest that implementing and measuring compliance to best-practice bundles, implementing HRO principles, and robustly participating in a multihospital learning collaborative can significantly reduce harm in hospitalized children on a large scale. These data support the hypothesis that participation in a hospital engagement network can result in significant harm reduction. Comparing baseline to study-end rates for each HAC and by using previously published data on average costs of each individual HAC, we estimate that this work resulted in >5000 children being spared harm with an estimated savings to the health care system of $93 000 000.26,31,35 

One important finding in this study is that significant HAC reduction can occur at the hospital and collaborative levels when compliance to best-practice bundles is emphasized, measured, and reported. Collaborative leadership aggressively communicated the importance of reliable adherence to all bundle elements. By using this approach, 8 of 9 targeted HACs in the SPS network showed collaborative-wide improvement. Although the vast majority of hospitals experienced improvement during the study period, the most dramatic HAC reduction came from the subset of hospitals that did not report bundle compliance data at the onset but did during the post-SPS phase of the study. We believe the acts of measuring, reporting, and comparing local best-practice compliance to defined 90% targets contributes substantially to improvement. Notably, this rate reduction occurred despite the fact that increased focus on harm is frequently associated with increased event detection.

The activities of the participating hospitals that are associated with successful harm reduction are as follows: engagement of executive leadership, identification of HAC leadership, development of harm-reduction goals and teams focused on implementing collaborative bundles aimed at maximizing compliance, and incorporating tactics designed to improve the organization’s culture of safety through basic error-prevention training, leadership methods, and cause analysis. Although this intervention highlights the value of the collaborative model in supporting hospital participation, institutions that cannot participate in the collaborative model may benefit from implementing similar strategies on an individual basis.

A second interesting study finding is that SSEs, which are typically generated by complex system failures unique to local hospital environments, can be reduced collectively by implementing tools based on HRO principles. Both the 12-month rolling average and the monthly SSER significantly decreased in this study. We believe that this robust SSER reduction strongly suggests that targeting improvements in organizational culture can be a highly effective and necessary adjunct to focused process improvement on particular HACs.

Our study addresses several of the concerns identified in previous studies of hospital engagement networks.11 First, rather than using a simple “pre-post” study design, we used the more rigorous autoregressive integrated moving average (ARIMA) time series analysis to determine more confidently the impact of our intervention. Second, a well-demarcated baseline period was established a priori. The timing of the intervention was easily identified based on this defined baseline period. Finally, all HAC and SSE metrics in this study were based on standard, previously published definitions and remained consistent throughout the study period.

Our study had several limitations. First, because several hospitals were not measuring processes or outcomes related to some HACs before the SPS intervention, baseline data collection for these hospitals was retrospective. This could bias the results in an unknown direction. Second, although the available SPS data demonstrated the impact of multielement HAC-prevention bundles and culture of safety interventions, more detailed data would be necessary to appreciate the effect of individual bundle elements. Third, we did not require hospitals to formally communicate their compliance to the culture work, resulting in an inability to determine the extent of implementation of best practices across the collaborative. This limitation would likely bias the results toward the null, so the impact of these practices would most likely be underestimated in our study. Finally, because this was a cohort study with historical controls, we were unable to determine the effect of secular trends on our outcomes. However, it is unlikely that concurrent interventions occurred at the exact time we launched our interventions at both the hospital or collaborative levels. Additionally, the statistical approaches we chose minimize the risk of secular trends impacting the outcomes.

Participation in a structured collaborative dedicated to implementing HAC-related best practice prevention bundles and culture of safety interventions designed to increase the use of HRO practices resulted in significant HAC and SSE reductions. The potential implications of these findings on national patient harm reduction are substantial. Future research should focus on determining which bundle and cultural elements have the most impact on outcomes, finding novel and effective ways to drive toward zero harm, and formally evaluating the cost effectiveness of implementing these interventions.

     
  • CAUTI

    catheter-associated urinary tract infection

  •  
  • HAC

    hospital-acquired condition

  •  
  • HRO

    high-reliability organization

  •  
  • QI

    quality improvement

  •  
  • SPS

    Solutions for Patient Safety

  •  
  • SSE

    serious safety event

  •  
  • SSER

    serious safety event rate

  •  
  • VTE

    venous thromboembolism

Dr Lyren conceptualized and designed the intervention, drafted the initial manuscript, and performed revisions; Drs Brilli and Sharek participated in the design of the intervention and performed revisions; Ms Zieker coordinated and supervised data collection and reviewed and revised the manuscript; Dr Marino provided statistical analysis of data and reviewed and revised manuscript; Dr Muething conceptualized and designed the intervention and reviewed and revised drafts of the manuscript; and all authors approved the final manuscript as written.

FUNDING: The Centers for Medicare and Medicaid Services Partnership for Patients program provided financial support for the operation of the collaborative; the Cardinal Health Foundation provided financial support for the in-person boards training program that was part of the collaborative intervention; the Children’s Hospital Association provided financial support for the operation of the collaborative and participating children’s hospitals.

1
Institute of Medicine
, ed.
Crossing the Quality Chasm: A New Health System for the Twenty-First Century
.
Washington, DC
:
National Academy Press
;
2001
2
James
JT
.
A new, evidence-based estimate of patient harms associated with hospital care.
J Patient Saf
.
2013
;
9
(
3
):
122
128
[PubMed]
3
Classen
DC
,
Resar
R
,
Griffin
F
, et al
.
‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured.
Health Aff (Millwood)
.
2011
;
30
(
4
):
581
589
[PubMed]
4
Landrigan
CP
,
Parry
GJ
,
Bones
CB
,
Hackbarth
AD
,
Goldmann
DA
,
Sharek
PJ
.
Temporal trends in rates of patient harm resulting from medical care.
N Engl J Med
.
2010
;
363
(
22
):
2124
2134
[PubMed]
5
Stockwell
DC
,
Bisarya
H
,
Classen
DC
, et al
.
A trigger tool to detect harm in pediatric inpatient settings.
Pediatrics
.
2015
;
135
(
6
):
1036
1042
[PubMed]
6
Weick
K
,
Sutcliffe
K
.
Managing the Unexpected
. 2nd ed.
San Francisco, CA
:
John Wiley & Sons, Inc
;
2007
7
Lyren
A
,
Brilli
R
,
Bird
M
,
Lashutka
N
,
Muething
S.
Ohio Children's Hospitals' Solutions for Patient Safety: a framework for pediatric patient safety improvement.
J Healthc Qual
.
2016
;
38
(
4
):
213
222
8
Miller
MR
,
Niedner
MF
,
Huskins
WC
, et al;
National Association of Children’s Hospitals and Related Institutions Pediatric Intensive Care Unit Central Line-Associated Bloodstream Infection Quality Transformation Teams
.
Reducing PICU central line-associated bloodstream infections: 3-year results.
Pediatrics
.
2011
;
128
(
5
). Available at: www.pediatrics.org/cgi/content/full/128/5/e1077
[PubMed]
9
Crandall
WV
,
Margolis
PA
,
Kappelman
MD
, et al;
ImproveCareNow Collaborative
.
Improved outcomes in a quality improvement collaborative for pediatric inflammatory bowel disease.
Pediatrics
.
2012
;
129
(
4
). Available at: www.pediatrics.org/cgi/content/full/129/4/e1030
[PubMed]
10
Pronovost
PJ
,
Miller
MR
,
Wachter
RM
.
Tracking progress in patient safety: an elusive target.
JAMA
.
2006
;
296
(
6
):
696
699
[PubMed]
11
Pronovost
P
,
Jha
AK
.
Did hospital engagement networks actually improve care?
N Engl J Med
.
2014
;
371
(
8
):
691
693
[PubMed]
12
Billett
AL
,
Colletti
RB
,
Mandel
KE
, et al
.
Exemplar pediatric collaborative improvement networks: achieving results.
Pediatrics
.
2013
;
131
(
suppl 4
):
S196
-
S203
[PubMed]
13
Toltzis
P
,
O’Riordan
M
,
Cunningham
DJ
, et al
.
A statewide collaborative to reduce pediatric surgical site infections.
Pediatrics
.
2014
;
134
(
4
). Available at: www.pediatrics.org/cgi/content/full/134/4/e1174
14
Sharek
PJ
,
McClead
RE
 Jr
,
Taketomo
C
, et al
.
An intervention to decrease narcotic-related adverse drug events in children’s hospitals.
Pediatrics
.
2008
;
122
(
4
). Available at: www.pediatrics.org/cgi/content/full/122/4/e861
[PubMed]
15
Langley
GJ
,
Moen
RD
,
Nolan
KM
,
Nolan
TW
,
Norman
CL
,
Provost
LP
.
The Improvement Guide: A Practical Approach to Enhancing Organizational Performance
.
San Francisco, CA
:
Jossey-Bass
;
2009
16
Ponde
PS
,
Neuman
RP
,
Cavanagh
RR
.
The Six Sigma Way: How GE, Motorola, and Other Top Companies are Honing Their Performance
.
New York, NY
:
McGraw-Hill Education
;
2000
17
Children’s Hospitals Solutions for Patient Safety
. Solutions for patient safety prevention bundles. Available at www.solutionsforpatientsafety.org/for-hospitals/hospital-resources/. Accessed May 16, 2016
18
National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP)
. NCC MERP Index for categorizing medical errors. Available at: www.nccmerp.org/sites/default/files/indexBW2001-06-12.pdf. Accessed July 4, 2017
19
Centers for Disease Control and Prevention
. Urinary tract infection. Available at: www.cdc.gov/nhsn/pdfs/pscManual/7pscCAUTIcurrent.pdf. Accessed July 4, 2017
20
Centers for Disease Control and Prevention
. Bloodstream infection event. Available at: www.cdc.gov/nhsn/PDFs/pscManual/4PSC_CLABScurrent.pdf. Accessed July 4, 2017
21
Centers for Disease Control and Prevention
. Surgical site infection. Available at: www.cdc.gov/nhsn/PDFs/pscManual/9pscSSIcurrent.pdf. Accessed July 4, 2017
22
National Database of Nursing Quality Indicators (NDNQI)
. Pressure injuries and staging. Available at: https://members.nursingquality.org/NDNQIPressureUlcerTraining/Module1/Default.aspx. Accessed July 4, 2017
23
Centers for Disease Control and Prevention
. Pneumonia event. Available at: www.cdc.gov/nhsn/PDFs/pscManual/6pscVAPcurrent.pdf. Accessed July 4, 2017
24
Healthcare Performance Improvement LLC
. HPI: safety and high reliability: strive to be a zero event organization. Available at: www.pressganey.com/consulting/safety-high-reliability. Accessed September 28, 2016
25
Berry
JC
,
Davis
JT
,
Bartman
T
, et al
.
Improved safety culture and teamwork climate are associated with decreases in patient harm and hospital mortality across a hospital system [published online ahead of print January 7, 2016].
J Patient Saf
. doi:
26
Brilli
RJ
,
McClead
RE
 Jr
,
Crandall
WV
, et al
.
A comprehensive patient safety program can significantly reduce preventable harm, associated costs, and hospital mortality.
J Pediatr
.
2013
;
163
(
6
):
1638
1645
[PubMed]
27
Muething
SE
,
Goudie
A
,
Schoettker
PJ
, et al
.
Quality improvement initiative to reduce serious safety events and improve patient safety culture.
Pediatrics
.
2012
;
130
(
2
). Available at: www.pediatrics.org/cgi/content/full/130/2/e423
[PubMed]
28
Hilliard
MA
,
Sczudlo
R
,
Scafidi
L
,
Cady
R
,
Villard
A
,
Shah
R
.
Our journey to zero: reducing serious safety events by over 70% through high-reliability techniques and workforce engagement.
J Healthc Risk Manag
.
2012
;
32
(
2
):
4
18
[PubMed]
29
Peterson
TH
,
Teman
SF
,
Connors
RH
.
A safety culture transformation: its effects at a children’s hospital.
J Patient Saf
.
2012
;
8
(
3
):
125
130
[PubMed]
30
Cleveland
WS
.
The Elements of Graphing Data
.
Monterey, CA
:
Wadsworth Advanced Books and Software
;
1985
:
170
178
31
Brilli
RJ
,
Sparling
KW
,
Lake
MR
, et al
.
The business case for preventing ventilator-associated pneumonia in pediatric intensive care unit patients.
Jt Comm J Qual Patient Saf
.
2008
;
34
(
11
):
629
638
[PubMed]
32
Sparling
KW
,
Ryckman
FC
,
Schoettker
PJ
, et al
.
Financial impact of failing to prevent surgical site infections.
Qual Manag Health Care
.
2007
;
16
(
3
):
219
225
[PubMed]
33
Goudie
A
,
Dynan
L
,
Brady
PW
,
Rettiganti
M
.
Attributable cost and length of stay for central line-associated bloodstream infections.
Pediatrics
.
2014
;
133
(
6
). Available at: www.pediatrics.org/cgi/content/full/133/6/e1525
[PubMed]
34
Goudie
A
,
Dynan
L
,
Brady
PW
,
Fieldston
E
,
Brilli
RJ
,
Walsh
KE
.
Costs of venous thromboembolism, catheter-associated urinary tract infection, and pressure ulcer.
Pediatrics
.
2015
;
136
(
3
):
432
439
[PubMed]
35
Nowak
JE
,
Brilli
RJ
,
Lake
MR
, et al
.
Reducing catheter-associated bloodstream infections in the pediatric intensive care unit: business case for quality improvement.
Pediatr Crit Care Med
.
2010
;
11
(
5
):
579
587
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data