OBJECTIVES:

The Pediatric Early Warning Score (PEWS) is an evidence-based tool that allows early collaborative assessment and intervention for a rapid response team (RRT) activation. The goal of our quality improvement initiative was to reduce the percentage of unnecessary RRT activations by 50% over 2 years without increasing PICU transfers or compromising patient safety and timely evaluation.

METHODS:

A PEWS system replaced preexisting vital signs–based pediatric RRT criteria and was modified through plan-do-study-act cycles. Unnecessary RRT activations, total RRT activation rate, transfers to the PICU, total clinical interventions performed per RRT, and missed RRT activation rate were compared between intervention periods. Likert scale surveys were administered to measure satisfaction with each modification.

RESULTS:

There was a significant decrease in the percentage of unnecessary RRT activations from 33% to 3.5% after the implementation of the PEWS and modified-PEWS systems (P < .05). The RRT activation rate decreased from 22.6 to 13.3 RRT activations per 1000 patient care days after implementation of the PEWS and modified-PEWS systems (P < .05), without changes in PICU transfer rates. Physicians reported that the PEWS system improved nursing communication and accuracy of RRT criteria (P < .05). Nursing reported that the PEWS system improved patient management and clinical autonomy (P < .05).

CONCLUSIONS:

The PEWS systems have been an effective means of identifying deteriorating pediatric patients and reducing unnecessary RRT activations. The new system fosters collaboration and communication at the bedside to prevent acute deterioration, perform timely interventions, and ultimately improve patient safety and outcomes.

Approximately 27% of inpatient pediatric cardiopulmonary arrests occur outside the PICU, and of those cardiopulmonary arrests, 27% to 42% of children survive to discharge and 10% to 34% survive to 1 year post cardiac arrest.1,2  The Institute for Healthcare Improvement identified pediatric rapid response teams (RRTs) as an effective strategy to provide evaluation and intervention to prevent cardiac arrest and clinical deterioration.35  The Pediatric Early Warning Score (PEWS) system was first developed at The Royal Children’s Hospital in Brighton, England in 2005 as an evidence-based tool to assist RRTs.6 

The integration of early warning scoring tools is a relatively new concept to most pediatric institutions, with their development coming from experiences in adult settings.68  A large randomized controlled trial failed to reveal that the PEWS decreased mortality; however, there was a statistically significant reduction in clinical deterioration events.9  Early warning scoring systems have been modified across multiple clinical settings, including hematology and oncology, cardiology, and emergency medicine.1014  There are several early warning score methodologies with variations in scoring, leading to differences in reliability and validity.8,15  Early identification of a deteriorating pediatric patient requires collaboration between a dedicated multidisciplinary team and a systematic tool, such as the PEWS. Such elements help facilitate timely delivery of appropriate resources and staff.10,1618 

Pediatric physiology easily prompts age-appropriate vital sign changes due to anxiety, fever, or medication delivery, exemplifying the need for unique assessment tools. Furthermore, pediatric anatomy and physiology differ on the basis of age group and lead to a higher predisposition for rapid clinical deterioration than adults.4,10,19  The former pediatric RRT system used at our institution mirrored the adult system and triggered a mandatory RRT activation when 1 vital sign was out of parameter (Fig 1). Single-parameter trigger systems have been proven to have low sensitivity.20  As a result, there is an overuse of the system due to unnecessary RRT activations, leading to ineffective use of resources.

FIGURE 1

Vital sign–based RRT activation criteria.

FIGURE 1

Vital sign–based RRT activation criteria.

Close modal

We aimed to evaluate the impact of implementing an evidence-based PEWS system using the plan-do-study-act (PDSA) model. The goals of our quality improvement initiative were to improve the RRT system by assessing the rate of RRT activations and PICU transfers and effectively identify deteriorating patients by evaluating RRT interventions. Over 2 years, we sought to reduce the percentage of unnecessary RTT activations by 50% without increasing PICU transfers or compromising patient safety and timely evaluation.

Our military treatment facility is an academic medical center with a 16-bed pediatric inpatient ward and 6-bed PICU. The clinical team at our institution includes pediatric and intensive care nurses, medical technicians, residents, fellows, pediatricians, pediatric subspecialists, and pediatric intensivists. Family members have an active role in the RRT system and participate in family-centered rounds.

The first PDSA cycle included implementing the PEWS system in July 2016 to replace the vital sign–based system in our pediatric inpatient ward. Preimplementation vital sign–based system data were collected from October 2015 to June 2016, and postimplementation PEWS system data were collected from July 2016 to June 2017. The PEWS system was devised by the PICU nursing staff, with the assistance of other clinical providers, and was modified for our institution from the study by Akre et al.7  In the PEWS criteria, 3 patient domains were evaluated: behavior, cardiovascular, and respiratory (Fig 2). Depending on patient characteristics, point values (ranging in value from 0 to 3) were assigned to each domain.

FIGURE 2

PEWS activation criteria. Fio2, fraction of inspired oxygen.

FIGURE 2

PEWS activation criteria. Fio2, fraction of inspired oxygen.

Close modal

Clinical team members received comprehensive training on the PEWS system before implementation and were informed of changes in the RRT system through monthly department meetings, emails, and scheduled training updates. The PEWS was obtained every 4 hours at each patient assessment and charted in the electronic health record (EHR). The algorithm for the PEWS is outlined in a detailed flowchart (Fig 3). Subsequent evaluation of the patient during the RRT activation was completed by the PICU resident, PICU charge nurse, and respiratory therapist in collaboration with the patient’s primary team.

FIGURE 3

PEWS Flowchart. PRRT, pediatric rapid response team; RN, registered nurse.

FIGURE 3

PEWS Flowchart. PRRT, pediatric rapid response team; RN, registered nurse.

Close modal

A second PDSA cycle was implemented in July 2017 after we discovered an increased sensitivity to heart rate scoring due to the parameters. The upper limit for the heart rate parameter was 110 beats per minute for a 13-month-old toddler to a 6-year-old child. We proposed a modified Pediatric Early Warning Score (m-PEWS) that altered the toddler (13 months to 3 years) heart rate range to 70 to 130 beats per minute and the preschool-aged (4–6 years) heart rate range to 70 to 120 beats per minute on the basis of deviations from age-appropriate heart rate parameters.21  A sample of the most recent 20 RRT events in the PEWS group was analyzed to determine the utility of the proposed heart rate changes and to ensure that deteriorating patients in that age group would not be missed with the proposed change. The m-PEWS system data were collected from July 2017 to June 2018.

Data for each pediatric RRT activation were prospectively collected through chart review from all pediatric admissions to assess the impact of implementing the new systems. Data included the date of birth, date of admission, admission diagnosis, RRT activation criteria, interventions performed per RRT, PICU transfers, missed RRT activations, and hospital-wide data, including patient acuity, patient care days, and monthly discharge rates. Specific interventions performed during each RRT activation were collected and organized into categories: respiratory support, intravenous fluids, imaging (plain radiographs, ultrasound, etc), medications (acetaminophen, ibuprofen, respiratory treatments, pain control, etc), other (electrocardiograms, transfusions, temperature warming systems, etc), no intervention performed, and transfer to the PICU. Acuity was estimated for the inpatient pediatric unit for each intervention period by using nursing workload data from the Workload Management System for Nursing–Internet (WMSNi), a database that quantifies nursing contributions to clinical care.22  Patient acuity scores range from 2 to 6, and higher acuity scores translate into more nursing care requirements. The WMSNi analyzes data in the hospital units. Details regarding each patient’s acuity level during an RRT activation were not available.

Details of each RRT event were transcribed into a secure RRT database, deidentified, and reviewed by the PEWS implementation team. A collaborative chart review of each RRT event was performed to confirm the accuracy of the data collected. There were no deaths reported, and mortality was not assessed.

The primary outcome measure was to evaluate the percentage of unnecessary RRT activations and to decrease this by 50% over 2 years. Unnecessary RRT activations were defined as RRT activations that required no intervention. The increased number of unnecessary RRT activations for the vital sign parameter RRT system started the quality improvement initiative and was chosen for the primary outcome measure. Balancing measures to evaluate for delayed or missing RRT activations included the RRT activation rate, PICU transfer rate, missed RRT activation rate (all per 1000 patient care days), and total interventions performed per RRT activation. The aims were to see a decreasing RRT rate, a stable number of PICU transfers, a decreasing number of missed RRT activations, and a stable number of interventions performed per RRT activation. Missed RRT activations are defined as scenarios that met RRT activation criteria without an RRT activation. Missed RRT activations were identified through an EHR vital sign and PEWS monitoring system and investigated through a manual chart review and voluntary reporting from the staff. The RRT database was the source of data for the outcome and balancing measures.

Secondary outcome measures involved staff satisfaction and knowledge and were evaluated with satisfaction surveys. Mean Likert scale responses with scales from 1 to 5 were used to assess pre- and postintervention survey questions regarding knowledge, accuracy, perception, confidence in the RRT system, clinical autonomy, and team communication. Survey questions were partially derived from the study by Kaul et al18  and modified for our institution.

The data for this quality improvement initiative were in 2 formats for statistical analysis. The first set included monthly summaries of patient data identified as RRT aggregate data. These data were pooled together by month and intervention periods (vital sign, PEWS, and m-PEWS) to look at trends over time. A time series analysis was completed by using run charts to analyze the primary outcome and balancing measures. Baseline median values for the center lines were obtained from vital sign intervention period data. An analysis to identify special cause variation was completed by using the Institute for Healthcare Improvement’s quality improvement essentials tool kit to identify shifts, trends, runs, and astronomical data points.23  Continuous data, such as RRT data and interventions performed, were summarized by using the mean ± SD for parametric data or median with interquartile range (IQR) for nonparametric data and analyzed by using the Kruskal-Wallis test (with a Tukey test for post hoc corrections). The second set included categorical data from the surveys and was analyzed by using the χ2 test or Fisher’s exact test, as appropriate. P values <.05 were considered statistically significant.

Our institutional review board screening process determined that this quality improvement initiative was exempt from formal review because it was not considered human subjects research. There were no potential conflicts of interest.

Data for the pediatric RRT activations by using the vital sign–based, PEWS, and m-PEWS systems collected over the quality improvement initiative period are presented in Table 1. There were a total of 38, 51, and 34 RRT activations for the vital sign, PEWS, and m-PEWS periods, respectively. The age of the patients was not statistically different between the vital sign, PEWS, and m-PEWS intervention periods (median [IQR] 4.4 [1.8–6.9] vs 2 [1.5–6.8] [P = .17] vs 3.1 [1.2–6.5] [P = .37]). The PEWS values for the PEWS (5 [3–6]) and m-PEWS (4 [3–5]; P = .05) periods were similar. There was no statistically significant difference in admission diagnosis between the 3 groups. Patient acuity in the vital sign period (median [IQR] 3.5 [3.3–4.1]) was higher compared with the PEWS (3.4 [3.3–3.6]; P < .001) and m-PEWS periods (3.3 [3.2–3.4]; P = .03). Mean (± SEM) monthly patient care days increased from 193.1 ± 9.7 in the vital sign period to 213.3 ± 9.6 in the PEWS period (P = .31) and then to 235.3 ± 15.1 in the m-PEWS period (P = .08). Mean (± SEM) monthly hospital discharges decreased from 102.3 ± 2.4 in the vital sign period to 88.6 ± 2.3 in the PEWS period (P < .001) and then to 83.6 ± 2.9 in the m-PEWS (P = .34) period.

TABLE 1

Pediatric RRT Data

Vital Sign–Based System (October 2015 to June 2016)PEWS System (July 2016 to July 2017)m-PEWS System (July 2017 to June 2018)
Median age (IQR), y 4.4 (1.8–6.9) 2 (1.5–6.8) 3.1 (1.2–6.5) 
Median PEWS (IQR) — 5 (3–6) 4 (3–5)** 
Admission diagnosis, %    
 Neurologic 
 Cardiac 
 Respiratory 56 53 63 
 Gastrointestinal 13 
 Infectious 33 29 37 
 Hematology and oncology 
 Surgical 14 17 
Median acuity (IQR)a 3.5 (3.3–4.1) 3.4 (3.3–3.6)* 3.3 (3.2–3.4)** 
Monthly patient care days, mean (SEM) 193.1 (9.7) 213.3 (9.6) 235.3 (15.1) 
Monthly discharges, mean (SEM) 102.3 (2.4) 88.6 (2.3)* 83.6 (2.9) 
Total RRT activations 38 51 34 
Missed RRT activations 28 
Pediatric code blue events 
Vital Sign–Based System (October 2015 to June 2016)PEWS System (July 2016 to July 2017)m-PEWS System (July 2017 to June 2018)
Median age (IQR), y 4.4 (1.8–6.9) 2 (1.5–6.8) 3.1 (1.2–6.5) 
Median PEWS (IQR) — 5 (3–6) 4 (3–5)** 
Admission diagnosis, %    
 Neurologic 
 Cardiac 
 Respiratory 56 53 63 
 Gastrointestinal 13 
 Infectious 33 29 37 
 Hematology and oncology 
 Surgical 14 17 
Median acuity (IQR)a 3.5 (3.3–4.1) 3.4 (3.3–3.6)* 3.3 (3.2–3.4)** 
Monthly patient care days, mean (SEM) 193.1 (9.7) 213.3 (9.6) 235.3 (15.1) 
Monthly discharges, mean (SEM) 102.3 (2.4) 88.6 (2.3)* 83.6 (2.9) 
Total RRT activations 38 51 34 
Missed RRT activations 28 
Pediatric code blue events 

—, PEWS were not available for vital sign-based system.

a

The WMSNi was used to measure patient acuity.

*

P < .05 (vital sign–based system versus PEWS).

**

P < .05 (PEWS versus m-PEWS).

Data for interventions and measures for each period are shown in Table 2. For the percentage of unnecessary RRT activations, the run chart analysis revealed a decreasing shift in the percentage of unnecessary RRT activations after implementation of the PEWS and m-PEWS systems (Fig 4). The data point for quarter 7 may have been an astronomical data point. When examined by period, the percentage of unnecessary RRT activations decreased from a mean (± SEM) of 33% ± 9.9% to 15% ± 5% (P = .11) from the vital sign period to the PEWS period and then to 3.5% ± 2% (P = .003) in the m-PEWS period. Additionally, the number of RRT activations without intervention per 1000 patient care days decreased from 7.7 ± 2.2 to 0.7 ± 0.5 from the vital sign period to the m-PEWS period (P = .004).

TABLE 2

Pediatric RRT Interventions and Measures

Vital Sign–Based System (October 2015 to June 2016), Mean (SEM)PEWS System (July 2016 to June 2017), Mean (SEM)m-PEWS System (July 2017 to June 2018), Mean (SEM)
Interventions    
 RRT activations with intervention per 1000 patient care days    
  Respiratory support 8.3 (2) 11.1 (2.3) 8.2 (1.8) 
  Intravenous fluids 1.8 (0.9) 2 (1) 3.5 (1.1) 
  Imaging 3.1 (1.3) 0.7 (0.5) 1.4 (0.6) 
  Medications 6.8 (1.8) 6.5 (2.1) 5.4 (1.1) 
  Other 1.8 (0.9) 3.6 (1.2) 4.3 (1.3) 
  No intervention 7.7 (2.1) 3.5 (1.2) 0.7 (0.4)**,*** 
Outcome and balancing measures    
 Percentage of unnecessary RRT activations, % 33 (9) 15 (5) 3.5 (2)*** 
 Rate of RRT activations per 1000 patient care days 22.6 (3.1) 18.9 (2.8) 13.3 (2.7)*** 
 Rate of PICU transfers per 1000 patient care days 5.5 (2.2) 6.9 (1.6) 7.7 (1.6) 
 Rate of missed RRT activations per 1000 patient care days 16.5 (5.7) 2.2 (1.7)* 0.3 (0.2)*** 
 Total interventions per RRT activation 1.4 (0.3) 1.6 (0.3) 2.3 (0.2)**,*** 
Vital Sign–Based System (October 2015 to June 2016), Mean (SEM)PEWS System (July 2016 to June 2017), Mean (SEM)m-PEWS System (July 2017 to June 2018), Mean (SEM)
Interventions    
 RRT activations with intervention per 1000 patient care days    
  Respiratory support 8.3 (2) 11.1 (2.3) 8.2 (1.8) 
  Intravenous fluids 1.8 (0.9) 2 (1) 3.5 (1.1) 
  Imaging 3.1 (1.3) 0.7 (0.5) 1.4 (0.6) 
  Medications 6.8 (1.8) 6.5 (2.1) 5.4 (1.1) 
  Other 1.8 (0.9) 3.6 (1.2) 4.3 (1.3) 
  No intervention 7.7 (2.1) 3.5 (1.2) 0.7 (0.4)**,*** 
Outcome and balancing measures    
 Percentage of unnecessary RRT activations, % 33 (9) 15 (5) 3.5 (2)*** 
 Rate of RRT activations per 1000 patient care days 22.6 (3.1) 18.9 (2.8) 13.3 (2.7)*** 
 Rate of PICU transfers per 1000 patient care days 5.5 (2.2) 6.9 (1.6) 7.7 (1.6) 
 Rate of missed RRT activations per 1000 patient care days 16.5 (5.7) 2.2 (1.7)* 0.3 (0.2)*** 
 Total interventions per RRT activation 1.4 (0.3) 1.6 (0.3) 2.3 (0.2)**,*** 
*

P < .05 (vital sign–based system versus PEWS).

**

P < .05 (PEWS versus m-PEWS).

***

P < .05 (vital sign–based system versus m-PEWS).

FIGURE 4

Percentage of unnecessary RRT activations: run chart with quarterly data.

FIGURE 4

Percentage of unnecessary RRT activations: run chart with quarterly data.

Close modal

The time series analysis revealed that the RRT activation rate remained stable, without any identifiable trends or shifts, after the implementation of the PEWS and m-PEWS systems (Fig 5). When examined by period, the RRT activation rate decreased from 22.6 ± 3.1 RRT activations per 1000 patient care days to 18.9 ± 2.8 (P = .69) from the vital sign period to the PEWS period and then further decreased to 13.3 ± 2.7 in the m-PEWS period (P = .04). A run chart analysis revealed a shift to increasing PICU transfer rates (Fig 6). When the PICU transfer rate was analyzed by period, it increased from 5.5 ± 2.2 in the vital sign period to 6.9 ± 1.6 transfers per 1000 patient care days (P = .5) in the PEWS period and then to 7.7 ± 1.6 transfers per 1000 patient care days (P = .32) in the m-PEWS period. A run chart analysis for missed RRT activations revealed a significant shift and trend for decreasing rates after the implementation of the PEWS and m-PEWS systems. There were too few runs in the chart, with the majority of the data points below the median center line (Fig 7). When analyzed per period, the missed RRT activation rate decreased from 16.5 ± 5.7 RRT activations per 1000 patient care days in the vital sign period to 2.2 ± 1.7 in the PEWS period (P = .02), then further decreased to 0.3 ± 0.2 in the m-PEWS period (P = .005). A run chart analysis revealed that the total number of interventions performed per RRT activation remained stable over the intervention periods (Fig 8). When analyzed per period, the number of interventions per RRT activation increased from 1.4 ± 0.3 interventions per RRT activation to 1.6 ± 0.3 from the vital sign period to the PEWS period and then to 2.3 ± 0.2 in the m-PEWS period (P = .02).

FIGURE 5

Pediatric RRT activation rates: run chart with quarterly data.

FIGURE 5

Pediatric RRT activation rates: run chart with quarterly data.

Close modal
FIGURE 6

RRT PICU transfer rate: run chart with quarterly data.

FIGURE 6

RRT PICU transfer rate: run chart with quarterly data.

Close modal
FIGURE 7

Missed RRT activation rate: run chart with quarterly data.

FIGURE 7

Missed RRT activation rate: run chart with quarterly data.

Close modal
FIGURE 8

Total interventions per RRT activation: run chart with quarterly data.

FIGURE 8

Total interventions per RRT activation: run chart with quarterly data.

Close modal

A total of 67 vital sign presurveys (58%), 73 PEWS postsurveys (63%), and 51 m-PEWS postsurveys (44%) were collected. There was a total of 115 potential survey participants, including 40 pediatric residents, 26 staff pediatricians, 16 PICU nurses, and 33 pediatric ward staff. A total of 7 vital sign surveys, 14 PEWS postsurveys, and 9 m-PEWS postsurveys from the PICU nursing staff were incorrectly completed and, thus, were not included in the analysis.

Demographic data for survey results for physicians and nurses are outlined in Tables 3 and 4, respectively. Physician confidence in the nursing staff to correctly identify deteriorating children increased when comparing the vital sign parameters with the PEWS system, with mean (± SEM) scores of 3.0 ± 0.2 and 3.7 ± 0.2, respectively (P = .04). Physicians felt that the PEWS system infrequently missed important signs and symptoms of clinical deteriorating compared with the vital sign parameters, with mean (± SEM) scores of 2.7 ± 0.2 and 3.2 ± 0.1, respectively (P = .02). Nurses reported improved management and prioritization of ill patients when comparing the vital sign parameters with the PEWS system, with mean (± SEM) scores of 3.1 ± 0.3 vs 4.0 ± 0.1. (P = .003). Comparing the vital sign parameters with the PEWS system, nurses correctly responded to a question regarding an RRT activation scenario, with mean (± SEM) scores of 3.8 ± 0.3 vs 1.6 ± 0.2, respectively, (P < .001). Clinical autonomy increased when comparing the vital sign parameters with the PEWS system, with mean (± SEM) scores of 2.8 ± 0.3 vs 3.8 ± 0.2, respectively, (P = .009).

TABLE 3

Physician Satisfaction Survey Results

Vital Sign–Based System (October 2015 to June 2016), n = 25PEWS System (July 2016 to July 2017), n = 26m-PEWS System (July 2017 to June 2018), n = 26
Level of training, n (%)   
 PGY 1 — — 7 (27) 
 PGY 2 7 28) 8 (31) 6 (23) 
 PGY 3 6 (24) 8 (31) 6 (23) 
 Attending or fellow 12 (48) 10 (38) 7 (27) 
RRTs participated in, n (%)    
 None 6 (24) 2 (7) 2 (7) 
 <5 12 (48) 14 (54) 15 (58) 
 6–10 6 (24) 8 (31) 5 (19) 
 11–15 1 (4) 1 (4) 3 (12) 
 >15 — 1 (4) 1 (4) 
Code events participated in, n (%)    
 None 15 (60) 10 (38) 9 (35) 
 <5 9 (36) 12 (46) 15 (58) 
 6–10 — 3 (12) 2 (7) 
 11–15 — — — 
 >15 1 (4) 1 (4) — 
Survey question, mean score (SEM)    
 Clinical deterioration 3.2 (0.2) 3.6 (0.1) 3.5 (0.2) 
 Nurse confidence 3.0 (0.2) 3.7 (0.2)* 3.7 (0.2) 
 Physician confidence 3.8 (0.1) 4.1 (0.2) 4.2 (0.2) 
 Accuracy of RRT criteria 3.2 (0.1) 2.7 (0.2)* 2.7 (0.2) 
 Team communication 3.4 (0.1) 3.7 (0.2) 3.6 (0.2) 
Vital Sign–Based System (October 2015 to June 2016), n = 25PEWS System (July 2016 to July 2017), n = 26m-PEWS System (July 2017 to June 2018), n = 26
Level of training, n (%)   
 PGY 1 — — 7 (27) 
 PGY 2 7 28) 8 (31) 6 (23) 
 PGY 3 6 (24) 8 (31) 6 (23) 
 Attending or fellow 12 (48) 10 (38) 7 (27) 
RRTs participated in, n (%)    
 None 6 (24) 2 (7) 2 (7) 
 <5 12 (48) 14 (54) 15 (58) 
 6–10 6 (24) 8 (31) 5 (19) 
 11–15 1 (4) 1 (4) 3 (12) 
 >15 — 1 (4) 1 (4) 
Code events participated in, n (%)    
 None 15 (60) 10 (38) 9 (35) 
 <5 9 (36) 12 (46) 15 (58) 
 6–10 — 3 (12) 2 (7) 
 11–15 — — — 
 >15 1 (4) 1 (4) — 
Survey question, mean score (SEM)    
 Clinical deterioration 3.2 (0.2) 3.6 (0.1) 3.5 (0.2) 
 Nurse confidence 3.0 (0.2) 3.7 (0.2)* 3.7 (0.2) 
 Physician confidence 3.8 (0.1) 4.1 (0.2) 4.2 (0.2) 
 Accuracy of RRT criteria 3.2 (0.1) 2.7 (0.2)* 2.7 (0.2) 
 Team communication 3.4 (0.1) 3.7 (0.2) 3.6 (0.2) 

PGY, postgraduate year; —, not applicable.

*

P < .05 (vital sign–based system versus PEWS).

TABLE 4

Nursing Satisfaction Survey Results

Vital Sign–Based System (October 2015 to June 2016), n = 19PEWS (July 2016 to July 2017), n = 34m-PEWS (July 2017 to June 2018), n = 17
Position, n (%)    
 RN 12 (63) 19 (56) 6 (35) 
 LPN or LVN 2 (10.5) 5 (15) 3 (18) 
 Medical technician 4 (21) 10 (29) 8 (47) 
 RT 1 (5.5) — — 
Clinical experience in pediatrics, n (%)    
 <1 4 (21) 9 (26) 4 (23) 
 1–5 12 (63) 20 (59) 11 (65) 
 6–10 3 (16) 4 (12) 2 (12) 
 11–15 — 1 (3) — 
Survey question theme, mean score (SEM)    
 Nursing confidence 4.3 (0.2) 4.3 (0.1) 4.4 (0.2) 
 Clinical deterioration 4.3 (0.2) 4.4 (0.1) 4.4 (0.2) 
 RRT activation 3.8 (0.3) 1.6 (0.2)* 1.5 (0.3) 
 Nursing workload 2.5 (0.3) 2.2 (0.2) 2.1 (0.2) 
 Patient management 3.1 (0.3) 4.0 (0.1)* 4.2 (0.2) 
 Autonomy 2.8 (0.3) 3.8 (0.2)* 3.8 (0.2) 
 Team communication 3.3 (0.2) 3.6 (0.1) 3.8 (0.2) 
Vital Sign–Based System (October 2015 to June 2016), n = 19PEWS (July 2016 to July 2017), n = 34m-PEWS (July 2017 to June 2018), n = 17
Position, n (%)    
 RN 12 (63) 19 (56) 6 (35) 
 LPN or LVN 2 (10.5) 5 (15) 3 (18) 
 Medical technician 4 (21) 10 (29) 8 (47) 
 RT 1 (5.5) — — 
Clinical experience in pediatrics, n (%)    
 <1 4 (21) 9 (26) 4 (23) 
 1–5 12 (63) 20 (59) 11 (65) 
 6–10 3 (16) 4 (12) 2 (12) 
 11–15 — 1 (3) — 
Survey question theme, mean score (SEM)    
 Nursing confidence 4.3 (0.2) 4.3 (0.1) 4.4 (0.2) 
 Clinical deterioration 4.3 (0.2) 4.4 (0.1) 4.4 (0.2) 
 RRT activation 3.8 (0.3) 1.6 (0.2)* 1.5 (0.3) 
 Nursing workload 2.5 (0.3) 2.2 (0.2) 2.1 (0.2) 
 Patient management 3.1 (0.3) 4.0 (0.1)* 4.2 (0.2) 
 Autonomy 2.8 (0.3) 3.8 (0.2)* 3.8 (0.2) 
 Team communication 3.3 (0.2) 3.6 (0.1) 3.8 (0.2) 

LPN, licensed practical nurse; LVN, licensed vocational nurse; RN, registered nurse; RT, respiratory therapist; —, not applicable.

*

P < .05 (vital sign–based system versus PEWS).

This quality improvement initiative is one of the first in which a pediatric early warning scoring system is compared with a single-parameter vital sign trigger system. Early warning scoring systems have been proven to identify trends ranging from 11 to 24 hours preceding a cardiopulmonary event with a sensitivity of up to 95%.4,12,16,24,25  The vital sign parameter system led to a high incidence of false alarms, creating a sense of frustration and wasted resources. These results build on such existing literature, demonstrating the clinical utility of identifying deteriorating patients and reducing unwarranted RRT activations.

Implementing the PEWS and m-PEWS systems improved recognition of deteriorating patients by decreasing the percentage of unnecessary RRT activations from 33% to 3.5% over 2 years, allowing for more resources to be available for urgent intervention. The considerable reduction of unnecessary RRT activations resulted in an overall decreased RRT activation rate without significantly changing the PICU transfer rate. The missed RRT activation rate significantly decreased because of an evidence-based scoring system and appropriate RRT activations. The number of interventions performed per RRT activation increased for the m-PEWS period, but no shifts or trends were revealed on the run chart analysis. The increase in interventions is possibly due to decreasing unnecessary RRT activations (ie, those that did not require interventions), therefore correctly identifying patients requiring clinical intervention. We can conclude that the changes with the RRT system led to accurate identification of deteriorating patients without compromising care for patients requiring a higher level of care.

Keys to success for early warning scoring systems include appropriate and consistent education, cooperation from nursing and physician staff, and validated scoring tools.10,19,25  Integration of the early warning scoring tools into the EHR facilitates accurate documentation and adherence to the protocol. Clear and appropriate activation criteria and a simple algorithm facilitate immediate action.7  Additionally, early warning scoring systems have been shown to improve confidence and autonomy.16,25  In one study, 80% of providers reported improved confidence in identifying deteriorating children.6  This statistic aligned with our measures and resulted in physicians and nurses reporting increased confidence in managing deteriorating patients as well as nursing autonomy. Overall, the PEWS activation system has enhanced communication and collaboration between team members.

There were limitations to this quality improvement initiative. There was a small sample size of patients over a limited time in a single military treatment facility. The focus was on matching for the same period. However, the intervention groups were not matched to assess the same months of the year; this could have affected the acuity level, the rate of RRT activations, and the types of admission diagnoses. Additionally, interventions were organized into categories, and we did not account for multiple interventions per category. A retrospective power analysis revealed that our study was underpowered; therefore, some interventions did not reach statistical significance. The data available allowed us to analyze the RRT activations as a system. However, we lacked physiologic data or risk-based scoring systems to compare groups or assess acuity after transfer to the PICU. The data available on the WMSNi as an accurate measure of acuity are limited. The acuity level was statistically lower in the m-PEWS period than in the vital sign and PEWS periods; however, that difference between acuity scores of 3.5 and 3.3 may not be clinically relevant. We can conclude that the inpatient unit acuity was similar between intervention periods. Satisfaction surveys are accompanied by limitations, including the high turnover of residents, nurses, and technicians and recall bias associated with survey questions.

This quality improvement initiative has proven to be sustainable through the early identification and inclusion of key stakeholders in the system change process. To further advance patient care and safety, we will continue to collect data, with plans to improve on the basis of additional feedback and PDSA cycles. Future directions include implementing the PEWS system in the emergency department to guide patient acuity and disposition before admission. Furthermore, we anticipate an extension of the PEWS system for pediatric RRT activation to other military treatment facilities with pediatric care teams.

We acknowledge the patients and their families, as well as the pediatric care teams at Brooke Army Medical Center, who were involved and impacted by the quality improvement initiative. We also acknowledge James Aden, the statistician, for his contributions to the statistical analysis throughout the initiative.

Mrs O’Hara-Wood, Mr Slaughter, and Mrs Cox, initiated data collection and implementation of the Pediatric Early Warning Score; Ms McFarlan assisted with data collection, conceptualization, and design of the quality improvement initiative; Mrs Gibbons assisted with conceptualization, education of the nursing staff, and design of the quality improvement initiative; Dr Sam assisted with design of the quality improvement initiative, analysis, and revisions of the manuscript; Dr Penney assisted with data collection and analysis, drafted the initial manuscript, and reviewed the final manuscript; Dr Matos initiated data collection and implementation of the Pediatric Early Warning Score, coordinated and supervised data collection, and critically reviewed the manuscript for important intellectual content; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

     
  • EHR

    electronic health record

  •  
  • IQR

    interquartile range

  •  
  • m-PEWS

    modified Pediatric Early Warning Score

  •  
  • PDSA

    plan-do-study-act

  •  
  • PEWS

    Pediatric Early Warning Score

  •  
  • RRT

    rapid response team

  •  
  • WMSNi

    Workload Management Score for Nursing–Internet

1
Reis
AG
,
Nadkarni
V
,
Perondi
MB
,
Grisi
S
,
Berg
RA
.
A prospective investigation into the epidemiology of in-hospital pediatric cardiopulmonary resuscitation using the international Utstein reporting style
.
Pediatrics
.
2002
;
109
(
2
):
200
209
2
Go
AS
,
Mozaffarian
D
,
Roger
VL
, et al.;
American Heart Association Statistics Committee and Stroke Statistics Subcommittee
.
Heart disease and stroke statistics--2013 update: a report from the American Heart Association
.
Circulation
.
2013
;
127
(
1
):
e6
e245
3
Simmonds
TC
.
Best-practice protocols: implementing a rapid response system of care
.
Nurs Manage
.
2005
;
36
(
7
):
41
42, 58–59
4
Duncan
H
,
Hutchison
J
,
Parshuram
CS
.
The Pediatric Early Warning System score: a severity of illness score to predict urgent medical need in hospitalized children
.
J Crit Care
.
2006
;
21
(
3
):
271
278
5
Kotsakis
A
,
Lobos
AT
,
Parshuram
C
, et al.;
Ontario Pediatric Critical Care Response Team Collaborative
.
Implementation of a multicenter rapid response system in pediatric academic hospitals is effective
.
Pediatrics
.
2011
;
128
(
1
):
72
78
6
Monaghan
A
.
Detecting and managing deterioration in children
.
Paediatr Nurs
.
2005
;
17
(
1
):
32
35
7
Akre
M
,
Finkelstein
M
,
Erickson
M
,
Liu
M
,
Vanderbilt
L
,
Billman
G
.
Sensitivity of the pediatric early warning score to identify patient deterioration
.
Pediatrics
.
2010
;
125
(
4
):
e763
e769
8
Murray
JS
,
Williams
LA
,
Pignataro
S
,
Volpe
D
.
An integrative review of pediatric early warning system scores
.
Pediatr Nurs
.
2015
;
41
(
4
):
165
174
9
Parshuram
CS
,
Dryden-Palmer
K
,
Farrell
C
, et al.;
Canadian Critical Care Trials Group and the EPOCH Investigators
.
Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: the EPOCH randomized clinical trial
.
JAMA
.
2018
;
319
(
10
):
1002
1012
10
Demmel
KM
,
Williams
L
,
Flesch
L
.
Implementation of the pediatric early warning scoring system on a pediatric hematology/oncology unit
.
J Pediatr Oncol Nurs
.
2010
;
27
(
4
):
229
240
11
McLellan
MC
,
Gauvreau
K
,
Connor
JA
.
Validation of the children’s hospital early warning system for critical deterioration recognition
.
J Pediatr Nurs
.
2017
;
32
:
52
58
12
Gold
DL
,
Mihalov
LK
,
Cohen
DM
.
Evaluating the Pediatric Early Warning Score (PEWS) system for admitted patients in the pediatric emergency department
.
Acad Emerg Med
.
2014
;
21
(
11
):
1249
1256
13
Niu
X
,
Tilford
B
,
Duffy
E
, et al
.
Feasibility and reliability of pediatric early warning score in the emergency department
.
J Nurs Care Qual
.
2016
;
31
(
2
):
161
166
14
Breslin
K
,
Marx
J
,
Hoffman
H
,
McBeth
R
,
Pavuluri
P
.
Pediatric early warning score at time of emergency department disposition is associated with level of care
.
Pediatr Emerg Care
.
2014
;
30
(
2
):
97
103
15
Chapman
SM
,
Grocott
MPW
,
Franck
LS
.
Systematic review of paediatric alert criteria for identifying hospitalised children at risk of critical deterioration
.
Intensive Care Med
.
2010
;
36
(
4
):
600
611
16
Skaletzky
SM
,
Raszynski
A
,
Totapally
BR
.
Validation of a modified pediatric early warning system score: a retrospective case-control study
.
Clin Pediatr (Phila)
.
2012
;
51
(
5
):
431
435
17
Ennis
L
.
Paediatric early warning scores on a children’s ward: a quality improvement initiative
.
Nurs Child Young People
.
2014
;
26
(
7
):
25
31
18
Kaul
M
,
Snethen
J
,
Kelber
ST
,
Zimmanck
K
,
Maletta
K
,
Meyer
M
.
Implementation of the bedside paediatric early warning system (BedsidePEWS) for nurse identification of deteriorating patients
.
J Spec Pediatr Nurs
.
2014
;
19
(
4
):
339
349
19
Haines
C
,
Perrott
M
,
Weir
P
.
Promoting care for acutely ill children-development and evaluation of a paediatric early warning tool
.
Intensive Crit Care Nurs
.
2006
;
22
(
2
):
73
81
20
Smith
GB
,
Prytherch
DR
,
Schmidt
PE
,
Featherstone
PI
,
Higgins
B
.
A review, and performance evaluation, of single-parameter “track and trigger” systems
.
Resuscitation
.
2008
;
79
(
1
):
11
21
21
Engorn
B
,
Flerlage
J
, eds..
The Harriet Lane Handbook
, 20th ed.
Philadelphia, PA
:
Saunders
;
2015
22
Wolgast
KA
,
Taylor
K
,
Garcia
D
,
Watkins
M
.
Measuring nursing essential contributions to quality patient care outcomes
.
US Army Med Dep J
.
2011
;
74
81
23
Institute for Healthcare Improvement
.
Quality improvement essentials toolkit
.
2020
. Available at: www.ihi.org/resources/Pages/Tools/Quality-Improvement-Essentials-Toolkit.aspx?utm_campaign=QI-Toolkit-Promotion&utm_medium=Whiteboard-Video&utm_source=ihi. Accessed November 11, 2020
24
Parshuram
CS
,
Duncan
HP
,
Joffe
AR
, et al
.
Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children
.
Crit Care
.
2011
;
15
(
4
):
R184
25
Tucker
KM
,
Brewer
TL
,
Baker
RB
,
Demeritt
B
,
Vossmeyer
MT
.
Prospective evaluation of a pediatric inpatient early warning scoring system
.
J Spec Pediatr Nurs
.
2009
;
14
(
2
):
79
85

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose. The views expressed in this article are those of the authors and do not reflect the official policy or position of the US Army, the US Air Force, the US Department of Defense, or the US Government. Title 17 US Code 105 provides that “Copyright protection under this title is not available for any work of the United States Government.” Title 17 US Code 101 defines a US Government work as a work prepared by a military service member or employee of the US Government as part of that person’s official duties.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.