Decreasing Percentage of Infants With Very Low 5-Minute Apgar Scores at a Safety-Net Level III NICU

We performed this prospective quality improvement study in a 25-bed level III NICU in an academic safety-net hospital that serves an urban area with more than 3500 annual deliveries and nearly 400 annual NICU admissions. Our NICU staffing includes 4 neonatologists, 5 advanced practice providers (APPs), 3 pediatric pharmacists, and adult respiratory therapists (RTs) with hospital-wide responsibilities. Each month, 2 pediatric interns and 4 senior pediatric residents rotate through the NICU. Overnight, APPs and residents are in-house, and the neonatologists take calls from home but come in for deliveries of infants <28 weeks, infants requiring continuous positive airway pressure or more support, and per request.

We created a multidisciplinary team including neonatologists, nurses, and APPs and reviewed current processes and barriers. The group identified key interventions for improvement and created a key driver diagram (Fig 1).

At the start of our initiative, a neonatologist, residents, and an APP who also covered the well-baby nursery worked in the NICU during the day. During the night shift, the neonatologist was on call from home and an APP was intermittently present. Residents often attended high-risk deliveries at night alone. We identified this inconsistency of NICU-trained staff at deliveries as a major barrier to providing optimal quality of neonatal resuscitation as 62% of infants with very low 5-minute Apgar scores were born between 5 pm and 7 am.

We included all inborn infants of 23 weeks’ gestation or more and performed a detailed analysis on those with 5-minute Apgar scores <4 admitted to the NICU. We excluded outborn infants, infants younger than 23 weeks’ gestational age (GA), infants with 5-minute Apgar scores ≧4, those with major congenital anomalies, and infants whose parents elected for comfort care after delivery.

The pediatric department expanded NICU APP coverage to 24 hours per day to ensure a NICU-trained provider was available to respond to all critical deliveries, increasing the team size from 3 intermittent to 5 NICU-dedicated APPs.

Pediatric interns attend all deliveries, senior pediatric residents attend most deliveries, and all are NRP certified. To address inconsistent orientation to the delivery room, APPs led an orientation for all residents during their first week in NICU. The APPs provided hands-on experience for common delivery scenarios through low-fidelity simulations. The orientation occurred during the first week of the residents’ rotation. APPs educated residents and staff on recent updates to NRP, particularly the importance of securing an advanced airway before initiating chest compressions, using the catchphrase “intubation is best before you pump the chest.”¹⁰ 

We identified a lack of resuscitation supplies when called to deliveries outside of labor and delivery or operating rooms as another contributing factor. RTs joined the multidisciplinary team to develop improvement initiatives. The team created “Go Bags” stocked with necessary equipment and supplies for all deliveries of infants ≤32 weeks’ GA or 1500 g, emergent deliveries, and any deliveries that occurred off the labor and delivery floor (e.g., main operating room, medical or surgical intensive care units, emergency department, offsite areas such as the parking lot). The equipment in the “Go Bags” included a Neopuff with a circuit, a self-inflating bag, four round mask sizes, 4 sizes of RAM cannulas, a bulb syringe, surfactant administration supplies, and monitoring supplies. The hospital stocked equipment for umbilical line placement and medication administration in the neonatal code carts available throughout the hospital.

We achieved improvement by early 2017, but our percentage of infants with very low Apgar scores often increased above the center line often during the third quarter of the year (July through September), coinciding with the start of the new academic year. Because we are a teaching institution with rotating residents, we needed more education around neonatal resuscitation. We performed monthly multidisciplinary NICU mock codes starting in November 2018. Rotating residents, APPs, RTs, NICU nurses, and a pharmacist participated in a mock code facilitated by a neonatologist. This experience provided a fundamental learning opportunity for all team members, especially new residents and adult RTs. Mock codes were scheduled by chief residents who prioritized the attendance of interns and senior residents.

We manually tracked infants meeting inclusion criteria, then obtained data from the electronic medical record (EMR) including date and time of birth, birth weight, GA, sex, delivery mode, singleton or multiples, Apgar score at 1 and 5 minutes, initial pH and base excess on blood gas (cord gas or infant’s gas within the first hour if cord gas was not available), maternal race and ethnicity, maternal age, presence of major anomalies, which NICU providers were present at delivery, chest compressions done at resuscitation, chest compression done before intubation, disposition, other short-term outcomes (whole-body cooling, hypoxic-ischemic encephalopathy and severity, seizures, placement of gastrostomy tube, and use of mechanical ventilation), and long-term outcomes at 24 months (cerebral palsy, developmental delay, and speech delay) because 69% of our babies received continued care in our hospital system’s clinics. We manually abstracted data for the baseline period (January 2012-March 2015), the intervention period (April 2015-November 2018), and the sustainment period (December 2018-September 2021). One author (E.H.) queried the local children’s hospital EMR for additional follow-up data.

Two authors (C.S., E.H.) manually collected data for process and balancing measures of APP and pediatric resident attendance at deliveries and number of intubations. One author (E.H.) collected data for the process measure of monthly mock codes performed in the NICU.

The primary outcome measure was the percentage of infants with very low 5-minute Apgar scores in inborn infants 23 weeks’ GA or more. The secondary outcome measure was the percentage of infants with Apgar scores <4 who received chest compressions before securing an advanced airway.

The process measures were the overall percentage of deliveries with APP attendance in infants with 5-minute Apgar scores <4 and also attendance stratified by time of day (7 am-5 pm) and night (5 pm-7 am) to ensure 24-hour APP coverage. Another process measure was the performance of monthly multidisciplinary NICU mock codes.

The balancing measure was the attendance of pediatric residents at deliveries of infants with 5-minute Apgar scores <4. This measure was important because of the potential concern for decreased pediatric resident learning opportunities or experience as a competing interest with more APP presence. Another balancing measure was the percentage of infants intubated during resuscitation. Given the emphasis we placed on intubation before initiating chest compressions, there was concern that the team may intubate infants unnecessarily.

We plotted the mean percentage of infants with 5-minute Apgar scores <4, the percentage of deliveries attended by an APP, and the percentage of infants with Apgar scores <4 intubated in the delivery room on a Statistical Process Control (SPC) p chart. We plotted the number of infants with Apgar scores <4 between infants receiving chest compressions before intubation on an SPC g chart. We used 3 σ limits to set the upper and lower control limits. We created the SPC p charts were created using QI Charts, version 2.0.22 (Scoville Associates, Texas). We used standard SPC charting rules for determining special cause as evidence of improvement.¹³  Continuous variables were presented either as mean with SD or median with interquartile range and analyzed with either an analysis of variance or Kruskal-Wallis test depending on the normality of the distribution. We analyzed categorical variables with χ² and Fisher’s exact tests, and we performed statistical analyses with SPSS, version 22 (IBM SPSS, Armonk, New York).

The hospital Quality Improvement Review Committee reviewed the project and determined this project did not include any research on human subjects and did not require institutional review board review.

The number of infants with 5-minute Apgar scores <4 in each period were as follows: 67 of 1051 deliveries in the baseline period, 54 of 1329 deliveries in the intervention period, and 22 of 927 deliveries in the sustainment period. GA, birth weight, 1-minute Apgar scores, initial pH, initial base excess, maternal race, and ethnicity were similar throughout the phases. Fewer infants received chest compressions as part of their resuscitation during the sustainment period (18%) compared with the baseline (34%) and intervention (50%) periods (P = .03) (Table 1, Supplemental Table 3).

The mean percent of infants with Apgar score <4 at 5 minutes decreased from the baseline, and the intervention mean of 5.12% to 2.16% when we detected special cause (run of 8 consecutive points below the centerline) on the SPC chart during the sustainment period (Fig 2).

Infants with Apgar scores <4 in the baseline period were almost 8 times (relative risk, 7.9; 95% CI, 1.9-33.7) more likely to receive chest compressions before intubation compared with the intervention and sustainment periods (P = .002) as the percent of infants receiving chest compressions before intubation decreased from 21% to 3%. We plotted the number of infants with 5-minute Apgar scores <4 between infants receiving chest compressions before intubation on an SPC g chart (Fig 3).

There were no statistically significant differences in short and long-term outcomes among all three periods (Table 2, Supplemental Table 3).

The mean percentage of APPs attending deliveries of infants with Apgar scores <4 increased from 44% in the baseline period to 100% after PDSA cycle 1, remained at 100%, and we detected special cause (run of 8 consecutive points above the centerline) on the SPC chart in summer 2015. We plotted updated mean and control limits on the SPC chart after we detected special cause (Fig 4). During the baseline period, an APP was present for 16/26 (62%) deliveries during the day and 15/41 (37%) deliveries at night. During the intervention period, an APP was present for 28/28 (100%) during the day and 24/26 (92%) during the night. During the sustainment period, 22/22 (100%) of deliveries were attended by an APP regardless of the time of day.

Monthly multidisciplinary mock codes started in November 2018 and the team performed them every month (35/35, 100%).

Each month, 2 pediatric interns and four senior pediatric residents rotate through the NICU. Before consistent 24-hour APP coverage, residents attended all deliveries of infants with very low Apgar scores (67/67, 100%). After increased APP coverage, residents continued to attend all deliveries during the intervention and sustainment periods (54/54, 100% and 22/22, 100%, respectively).

There was no increase in the percentage of infants with 5-minute Apgar scores <4 intubated across the 3 study periods: baseline, 53/67 (79%); intervention, 45/54 (83%); and sustainment, 12/22 (55%) (P = .02). The percentage of infants intubated in the delivery room decreased toward the end of the study but special cause was not detected on the SPC p chart (Fig 5).

After 4 PDSA cycles, we achieved our SMART aims. We decreased the percentage of infants receiving a 5-minute Apgar score <4 from a mean of 5.12% in the baseline and intervention periods to 2.16% in the sustainment period, slightly below the national mean. Variability also decreased as demonstrated by data points closer to the reset center line than during the baseline and intervention periods. Additionally, the percentage of infants receiving chest compressions before intubation decreased from 21% during baseline to 3% in the intervention and sustainment periods (P = .002). Our team sustained these results for nearly 3 years.

The key lessons learned were that by ensuring consistent APP availability and, more importantly, by providing rigorous education through simulations and monthly multidisciplinary mock codes, we succeeded in improving the quality of resuscitation. NRP guidelines are not always strictly adhered to in tertiary centers, even with highly experienced resuscitation teams, but we achieved this through easily replicable educational interventions.¹⁴ 

APPs are a fast-growing health care workforce and have shown to be effective in improving the quality of care by improving workflow, access to care, clinical outcomes, and patient/parental satisfaction across all patient populations.^15–17  In addition, the integration of APPs improved neonatal resuscitation and stabilization through their provision of leadership and mentorship to other members of the NICU team similar to Follet et al’s findings.¹⁸ 

Because we are in an academic teaching institution, this may have impacted our ability to show special cause sooner because there was often an increase above the center line during the July to September quarter likely from the presence of new interns and residents. In addition to the APP’s presence at deliveries and their review of common delivery scenarios, monthly multidisciplinary mock codes ultimately enabled our team to reach our goals. Though these educational experiences were low fidelity, results have shown that low-fidelity versus high-fidelity simulations have comparable learning outcomes.^19–21  Moreover, if performed frequently, low-fidelity mock codes improve quality of resuscitation and team collaboration.²²  Thus, simulation and mock codes can be a low-cost, high-yield, and easily replicable intervention. Although learners consistently prefer high-fidelity simulations in an urban safety-net academic hospital with a limited budget, simple, low-cost simulations and frequent mock codes are an effective and realistic intervention. We saw sustained improvement including no infants with 5-minute Apgar scores <4 for the last 6 months of the sustainment period.

We also significantly decreased the percentage of infants receiving chest compressions before intubation with no occurrences since June 2018. This result is a difficult milestone to reach because other centers found high rates of compressions occurring before intubation.²³  The addition of consistent APP presence, repetitive education through frequent multidisciplinary team mock codes, and a new catchphrase improved the adherence to NRP guidelines without increasing unnecessary intubations.

Adding full-time APP coverage was not to replace residents or impact their rotation experience, but to improve patient care. Pediatric residents attended all deliveries, so the APPs did not impact the residents’ education. Resnick et al found that adding APPs positively impacted and increased educational opportunities for residents.²⁴  We did not collect residents’ perceptions about increased APP presence, which may have provided interesting feedback.

We did not find any statistically significant differences in short- and long-term outcomes throughout the period, likely related to the small sample size. However, several outcomes trended upward, possibly because of the retainment of lower GA and higher acuity infants since our NICU expanded to provide additional services in 2018. Additionally, the incorporation of EMR in April 2016 allowed accurate tracking of infant data, especially long-term outcomes because the majority of the infants remain under the care of a pediatrician in the same hospital system. Finally, we did not include long-term data for infants born after 2020 because we assessed long-term outcomes at 24 months, which may affect our sustainment period data.

There were several limitations to our project. First, this is a single-center experience, and our success may have been related to the smaller size of our delivery service and staff. Whether similar methods are generalizable to larger, higher delivery units is unclear. Second, we also chose to use quarterly data for the SPC charts because of the small sample size rather than average percentages of data within a study period to ensure we were measuring long-term longitudinal improvement in our outcome measures. This may have resulted in a delay in documenting improvement.

Third, we excluded infants with Apgar scores of ≥4 because documentation of delivery attendance was variable and unreliable before EMR’s availability in April 2016, well into the intervention period. We also considered the decrease in the percentage of infants with low Apgar scores potentially influenced by the Hawthorne effect because our quality improvement process was not a blinded intervention. However, though the Apgar score is subjective, with a small group of APPs attending all deliveries, we presumed that the interrater reliability was good.

Last, a relatively small monthly sample size of infants meeting inclusion criteria resulted in variability in results, as seen in the widespread upper and lower control limits in the SPC charts. Small sample sizes result in less stable estimates for the outcome variables and limit its reliability and replicability. This applies to our short- and long-term outcome variables in which no statistical significance was noted, so we did not perform multivariate regression analysis.

This quality improvement initiative demonstrated that after initial problem identification and system evaluation, our NICU matched the national benchmark of infants with 5-minute Apgar scores <4 and significantly decreased the percentage of infants receiving chest compressions before intubation. Ultimately, the rigorous education and team collaboration through simulations and multidisciplinary team mock codes were critical to our success.