A Quality Improvement Initiative to Reduce Unnecessary Rapid Responses Using Early Warning Scores

Our military treatment facility is an academic medical center with a 16-bed pediatric inpatient ward and 6-bed PICU. The clinical team at our institution includes pediatric and intensive care nurses, medical technicians, residents, fellows, pediatricians, pediatric subspecialists, and pediatric intensivists. Family members have an active role in the RRT system and participate in family-centered rounds.

The first PDSA cycle included implementing the PEWS system in July 2016 to replace the vital sign–based system in our pediatric inpatient ward. Preimplementation vital sign–based system data were collected from October 2015 to June 2016, and postimplementation PEWS system data were collected from July 2016 to June 2017. The PEWS system was devised by the PICU nursing staff, with the assistance of other clinical providers, and was modified for our institution from the study by Akre et al.⁷  In the PEWS criteria, 3 patient domains were evaluated: behavior, cardiovascular, and respiratory (Fig 2). Depending on patient characteristics, point values (ranging in value from 0 to 3) were assigned to each domain.

Clinical team members received comprehensive training on the PEWS system before implementation and were informed of changes in the RRT system through monthly department meetings, emails, and scheduled training updates. The PEWS was obtained every 4 hours at each patient assessment and charted in the electronic health record (EHR). The algorithm for the PEWS is outlined in a detailed flowchart (Fig 3). Subsequent evaluation of the patient during the RRT activation was completed by the PICU resident, PICU charge nurse, and respiratory therapist in collaboration with the patient’s primary team.

A second PDSA cycle was implemented in July 2017 after we discovered an increased sensitivity to heart rate scoring due to the parameters. The upper limit for the heart rate parameter was 110 beats per minute for a 13-month-old toddler to a 6-year-old child. We proposed a modified Pediatric Early Warning Score (m-PEWS) that altered the toddler (13 months to 3 years) heart rate range to 70 to 130 beats per minute and the preschool-aged (4–6 years) heart rate range to 70 to 120 beats per minute on the basis of deviations from age-appropriate heart rate parameters.²¹  A sample of the most recent 20 RRT events in the PEWS group was analyzed to determine the utility of the proposed heart rate changes and to ensure that deteriorating patients in that age group would not be missed with the proposed change. The m-PEWS system data were collected from July 2017 to June 2018.

Data for each pediatric RRT activation were prospectively collected through chart review from all pediatric admissions to assess the impact of implementing the new systems. Data included the date of birth, date of admission, admission diagnosis, RRT activation criteria, interventions performed per RRT, PICU transfers, missed RRT activations, and hospital-wide data, including patient acuity, patient care days, and monthly discharge rates. Specific interventions performed during each RRT activation were collected and organized into categories: respiratory support, intravenous fluids, imaging (plain radiographs, ultrasound, etc), medications (acetaminophen, ibuprofen, respiratory treatments, pain control, etc), other (electrocardiograms, transfusions, temperature warming systems, etc), no intervention performed, and transfer to the PICU. Acuity was estimated for the inpatient pediatric unit for each intervention period by using nursing workload data from the Workload Management System for Nursing–Internet (WMSNi), a database that quantifies nursing contributions to clinical care.²²  Patient acuity scores range from 2 to 6, and higher acuity scores translate into more nursing care requirements. The WMSNi analyzes data in the hospital units. Details regarding each patient’s acuity level during an RRT activation were not available.

Details of each RRT event were transcribed into a secure RRT database, deidentified, and reviewed by the PEWS implementation team. A collaborative chart review of each RRT event was performed to confirm the accuracy of the data collected. There were no deaths reported, and mortality was not assessed.

The primary outcome measure was to evaluate the percentage of unnecessary RRT activations and to decrease this by 50% over 2 years. Unnecessary RRT activations were defined as RRT activations that required no intervention. The increased number of unnecessary RRT activations for the vital sign parameter RRT system started the quality improvement initiative and was chosen for the primary outcome measure. Balancing measures to evaluate for delayed or missing RRT activations included the RRT activation rate, PICU transfer rate, missed RRT activation rate (all per 1000 patient care days), and total interventions performed per RRT activation. The aims were to see a decreasing RRT rate, a stable number of PICU transfers, a decreasing number of missed RRT activations, and a stable number of interventions performed per RRT activation. Missed RRT activations are defined as scenarios that met RRT activation criteria without an RRT activation. Missed RRT activations were identified through an EHR vital sign and PEWS monitoring system and investigated through a manual chart review and voluntary reporting from the staff. The RRT database was the source of data for the outcome and balancing measures.

Secondary outcome measures involved staff satisfaction and knowledge and were evaluated with satisfaction surveys. Mean Likert scale responses with scales from 1 to 5 were used to assess pre- and postintervention survey questions regarding knowledge, accuracy, perception, confidence in the RRT system, clinical autonomy, and team communication. Survey questions were partially derived from the study by Kaul et al¹⁸  and modified for our institution.

The data for this quality improvement initiative were in 2 formats for statistical analysis. The first set included monthly summaries of patient data identified as RRT aggregate data. These data were pooled together by month and intervention periods (vital sign, PEWS, and m-PEWS) to look at trends over time. A time series analysis was completed by using run charts to analyze the primary outcome and balancing measures. Baseline median values for the center lines were obtained from vital sign intervention period data. An analysis to identify special cause variation was completed by using the Institute for Healthcare Improvement’s quality improvement essentials tool kit to identify shifts, trends, runs, and astronomical data points.²³  Continuous data, such as RRT data and interventions performed, were summarized by using the mean ± SD for parametric data or median with interquartile range (IQR) for nonparametric data and analyzed by using the Kruskal-Wallis test (with a Tukey test for post hoc corrections). The second set included categorical data from the surveys and was analyzed by using the χ² test or Fisher’s exact test, as appropriate. P values <.05 were considered statistically significant.

Our institutional review board screening process determined that this quality improvement initiative was exempt from formal review because it was not considered human subjects research. There were no potential conflicts of interest.

Data for the pediatric RRT activations by using the vital sign–based, PEWS, and m-PEWS systems collected over the quality improvement initiative period are presented in Table 1. There were a total of 38, 51, and 34 RRT activations for the vital sign, PEWS, and m-PEWS periods, respectively. The age of the patients was not statistically different between the vital sign, PEWS, and m-PEWS intervention periods (median [IQR] 4.4 [1.8–6.9] vs 2 [1.5–6.8] [P = .17] vs 3.1 [1.2–6.5] [P = .37]). The PEWS values for the PEWS (5 [3–6]) and m-PEWS (4 [3–5]; P = .05) periods were similar. There was no statistically significant difference in admission diagnosis between the 3 groups. Patient acuity in the vital sign period (median [IQR] 3.5 [3.3–4.1]) was higher compared with the PEWS (3.4 [3.3–3.6]; P < .001) and m-PEWS periods (3.3 [3.2–3.4]; P = .03). Mean (± SEM) monthly patient care days increased from 193.1 ± 9.7 in the vital sign period to 213.3 ± 9.6 in the PEWS period (P = .31) and then to 235.3 ± 15.1 in the m-PEWS period (P = .08). Mean (± SEM) monthly hospital discharges decreased from 102.3 ± 2.4 in the vital sign period to 88.6 ± 2.3 in the PEWS period (P < .001) and then to 83.6 ± 2.9 in the m-PEWS (P = .34) period.

Data for interventions and measures for each period are shown in Table 2. For the percentage of unnecessary RRT activations, the run chart analysis revealed a decreasing shift in the percentage of unnecessary RRT activations after implementation of the PEWS and m-PEWS systems (Fig 4). The data point for quarter 7 may have been an astronomical data point. When examined by period, the percentage of unnecessary RRT activations decreased from a mean (± SEM) of 33% ± 9.9% to 15% ± 5% (P = .11) from the vital sign period to the PEWS period and then to 3.5% ± 2% (P = .003) in the m-PEWS period. Additionally, the number of RRT activations without intervention per 1000 patient care days decreased from 7.7 ± 2.2 to 0.7 ± 0.5 from the vital sign period to the m-PEWS period (P = .004).

The time series analysis revealed that the RRT activation rate remained stable, without any identifiable trends or shifts, after the implementation of the PEWS and m-PEWS systems (Fig 5). When examined by period, the RRT activation rate decreased from 22.6 ± 3.1 RRT activations per 1000 patient care days to 18.9 ± 2.8 (P = .69) from the vital sign period to the PEWS period and then further decreased to 13.3 ± 2.7 in the m-PEWS period (P = .04). A run chart analysis revealed a shift to increasing PICU transfer rates (Fig 6). When the PICU transfer rate was analyzed by period, it increased from 5.5 ± 2.2 in the vital sign period to 6.9 ± 1.6 transfers per 1000 patient care days (P = .5) in the PEWS period and then to 7.7 ± 1.6 transfers per 1000 patient care days (P = .32) in the m-PEWS period. A run chart analysis for missed RRT activations revealed a significant shift and trend for decreasing rates after the implementation of the PEWS and m-PEWS systems. There were too few runs in the chart, with the majority of the data points below the median center line (Fig 7). When analyzed per period, the missed RRT activation rate decreased from 16.5 ± 5.7 RRT activations per 1000 patient care days in the vital sign period to 2.2 ± 1.7 in the PEWS period (P = .02), then further decreased to 0.3 ± 0.2 in the m-PEWS period (P = .005). A run chart analysis revealed that the total number of interventions performed per RRT activation remained stable over the intervention periods (Fig 8). When analyzed per period, the number of interventions per RRT activation increased from 1.4 ± 0.3 interventions per RRT activation to 1.6 ± 0.3 from the vital sign period to the PEWS period and then to 2.3 ± 0.2 in the m-PEWS period (P = .02).

A total of 67 vital sign presurveys (58%), 73 PEWS postsurveys (63%), and 51 m-PEWS postsurveys (44%) were collected. There was a total of 115 potential survey participants, including 40 pediatric residents, 26 staff pediatricians, 16 PICU nurses, and 33 pediatric ward staff. A total of 7 vital sign surveys, 14 PEWS postsurveys, and 9 m-PEWS postsurveys from the PICU nursing staff were incorrectly completed and, thus, were not included in the analysis.

Demographic data for survey results for physicians and nurses are outlined in Tables 3 and 4, respectively. Physician confidence in the nursing staff to correctly identify deteriorating children increased when comparing the vital sign parameters with the PEWS system, with mean (± SEM) scores of 3.0 ± 0.2 and 3.7 ± 0.2, respectively (P = .04). Physicians felt that the PEWS system infrequently missed important signs and symptoms of clinical deteriorating compared with the vital sign parameters, with mean (± SEM) scores of 2.7 ± 0.2 and 3.2 ± 0.1, respectively (P = .02). Nurses reported improved management and prioritization of ill patients when comparing the vital sign parameters with the PEWS system, with mean (± SEM) scores of 3.1 ± 0.3 vs 4.0 ± 0.1. (P = .003). Comparing the vital sign parameters with the PEWS system, nurses correctly responded to a question regarding an RRT activation scenario, with mean (± SEM) scores of 3.8 ± 0.3 vs 1.6 ± 0.2, respectively, (P < .001). Clinical autonomy increased when comparing the vital sign parameters with the PEWS system, with mean (± SEM) scores of 2.8 ± 0.3 vs 3.8 ± 0.2, respectively, (P = .009).

This quality improvement initiative is one of the first in which a pediatric early warning scoring system is compared with a single-parameter vital sign trigger system. Early warning scoring systems have been proven to identify trends ranging from 11 to 24 hours preceding a cardiopulmonary event with a sensitivity of up to 95%.^{4,12,16,24,25}  The vital sign parameter system led to a high incidence of false alarms, creating a sense of frustration and wasted resources. These results build on such existing literature, demonstrating the clinical utility of identifying deteriorating patients and reducing unwarranted RRT activations.

Implementing the PEWS and m-PEWS systems improved recognition of deteriorating patients by decreasing the percentage of unnecessary RRT activations from 33% to 3.5% over 2 years, allowing for more resources to be available for urgent intervention. The considerable reduction of unnecessary RRT activations resulted in an overall decreased RRT activation rate without significantly changing the PICU transfer rate. The missed RRT activation rate significantly decreased because of an evidence-based scoring system and appropriate RRT activations. The number of interventions performed per RRT activation increased for the m-PEWS period, but no shifts or trends were revealed on the run chart analysis. The increase in interventions is possibly due to decreasing unnecessary RRT activations (ie, those that did not require interventions), therefore correctly identifying patients requiring clinical intervention. We can conclude that the changes with the RRT system led to accurate identification of deteriorating patients without compromising care for patients requiring a higher level of care.

Keys to success for early warning scoring systems include appropriate and consistent education, cooperation from nursing and physician staff, and validated scoring tools.^10,19,25  Integration of the early warning scoring tools into the EHR facilitates accurate documentation and adherence to the protocol. Clear and appropriate activation criteria and a simple algorithm facilitate immediate action.⁷  Additionally, early warning scoring systems have been shown to improve confidence and autonomy.^16,25  In one study, 80% of providers reported improved confidence in identifying deteriorating children.⁶  This statistic aligned with our measures and resulted in physicians and nurses reporting increased confidence in managing deteriorating patients as well as nursing autonomy. Overall, the PEWS activation system has enhanced communication and collaboration between team members.

There were limitations to this quality improvement initiative. There was a small sample size of patients over a limited time in a single military treatment facility. The focus was on matching for the same period. However, the intervention groups were not matched to assess the same months of the year; this could have affected the acuity level, the rate of RRT activations, and the types of admission diagnoses. Additionally, interventions were organized into categories, and we did not account for multiple interventions per category. A retrospective power analysis revealed that our study was underpowered; therefore, some interventions did not reach statistical significance. The data available allowed us to analyze the RRT activations as a system. However, we lacked physiologic data or risk-based scoring systems to compare groups or assess acuity after transfer to the PICU. The data available on the WMSNi as an accurate measure of acuity are limited. The acuity level was statistically lower in the m-PEWS period than in the vital sign and PEWS periods; however, that difference between acuity scores of 3.5 and 3.3 may not be clinically relevant. We can conclude that the inpatient unit acuity was similar between intervention periods. Satisfaction surveys are accompanied by limitations, including the high turnover of residents, nurses, and technicians and recall bias associated with survey questions.

This quality improvement initiative has proven to be sustainable through the early identification and inclusion of key stakeholders in the system change process. To further advance patient care and safety, we will continue to collect data, with plans to improve on the basis of additional feedback and PDSA cycles. Future directions include implementing the PEWS system in the emergency department to guide patient acuity and disposition before admission. Furthermore, we anticipate an extension of the PEWS system for pediatric RRT activation to other military treatment facilities with pediatric care teams.

We acknowledge the patients and their families, as well as the pediatric care teams at Brooke Army Medical Center, who were involved and impacted by the quality improvement initiative. We also acknowledge James Aden, the statistician, for his contributions to the statistical analysis throughout the initiative.

EHR

electronic health record

IQR

interquartile range

m-PEWS

modified Pediatric Early Warning Score

PDSA

plan-do-study-act

PEWS

Pediatric Early Warning Score

RRT

rapid response team

WMSNi

Workload Management Score for Nursing–Internet