OBJECTIVES:

Pediatric residents quickly lose neonatal resuscitation (NR) skills after initial training. Helping Babies Breathe is a skills-based curriculum emphasizing basic NR skills needed within the “Golden Minute” after birth. With this pilot study, we evaluated the feasibility of implementing a Golden Minute review and the impact on overall performance and bag-mask ventilation (BMV) skills in pediatric interns during and/or after their NICU rotation, with varying frequency and/or intensity of “just-in-place” simulation.

METHODS:

During their NICU rotation, interns at 1 delivery hospital received the Golden Minute module and hands-on simulation practice. All enrolled interns were randomly assigned to weekly retraining or no retraining for their NICU month and every 1- or 3-month retraining post-NICU for the remainder of their intern year, based on a factorial design. The primary measure was the score on a 21-item evaluation tool administered at the end of intern year, which was compared to the scores received by interns at another hospital (controls).

RESULTS:

Twenty-eight interns were enrolled in the intervention. For the primary outcome, at the end of intern year, the 1- and 3-month groups had higher scores (18.8 vs 18.6 vs 14.4; P < .01) and shorter time to effective BMV (10.6 vs 20.4 vs 52.8 seconds; P < .05 for both comparisons) than those of controls. However, the 1- and 3-month groups had no difference in score or time to BMV.

CONCLUSIONS:

This pilot study revealed improvement in simulated performance of basic NR skills in interns receiving increased practice intensity and/or frequency than those who received the current standard of NR training.

Widespread implementation of properly practiced neonatal resuscitation (NR) is an important opportunity to save lives.1,2  However, shortly after an NR course, health providers may retain knowledge but are often unable to perform the essential life-saving skills when needed in real life.39  With reductions in duty hours and NICU rotations, pediatric residents have few opportunities to master basic resuscitation skills, particularly bag-mask ventilation (BMV). Simulation provides critical learning opportunities to master resuscitation skills, particularly BMV,1018  considered 1 of the most important procedures in pediatric residency by the Accreditation Council on Graduate Medical Education. Regular practice of BMV skills should result in improved performance that will translate into better management of critically ill newborns and children requiring stabilization in any pediatric setting, including the average of 24 pediatric office emergencies occurring annually.15,1921 

In late 2015, the International Liaison Committee on Resuscitation Neonatal Task Force released the most recent treatment recommendations incorporated into the seventh edition of the Neonatal Resuscitation Program (NRP). They used the Grading of Recommendations Assessment, Development and Evaluation system to systematically review the evidence for current research gaps, including the frequency of resuscitation training or practices to retain skills.22  The studies exploring this theme exhibited low quality of evidence, revealed no difference in patient outcomes but did show some advantages in psychomotor performance, knowledge, and confidence when trainings occurred at least every 6 months.22  Therefore, the seventh edition of the NRP suggests that neonatal task training should occur more frequently than the current 2-year interval but otherwise makes no specific recommendations. The International Liaison Committee on Resuscitation emphasizes that evidence gaps regarding NR include understanding whether simulated delivery room experiences enhance retention of knowledge and competency, how retention and competency are assessed, and how frequently practice is required to retain skills.2224 

Although higher quality of evidence is needed to formulate stronger recommendations, a weak recommendation can be made that training should occur more frequently than yearly, although the exact frequency is unknown.25  Low-dose, high-frequency practice has been used in pediatric and adult cardiopulmonary resuscitation, in addition to global health settings, for retention of skills.2628  Simulation-based studies incorporating “just-in-time” (practice just before the skill is performed in real life) and “just-in-place” (practice in the location where the skill is performed in real life, such as a clinical unit, not a simulation laboratory) methodologies have revealed mixed results, including improvements in cardiopulmonary resuscitation skills and reduction in central line–associated infections2931  but no improvement in resident lumbar puncture success rate.32  Therefore, opportunity exists to further study and make definitive recommendations on how intensely and/or frequently practice should be performed to acquire and retain skills. Our aim was to test a cohort of pediatric interns with a schedule of recurring neonatal simulation practices by using just-in-place simulation administered at varying frequency and intensity to determine the most effective way to teach pediatric residents NR, with a goal of achievement and retention of the skill of BMV.

The study was approved by the institutional review boards of the freestanding children’s hospital and the 2 delivery hospitals where the study took place over the 2014–2015 academic year. Each year, 55 to 60 pediatric interns are accepted and may be grouped into categorical pediatrics (∼40), medicine/pediatrics (∼5–7), pediatrics/neurology (∼5), pediatrics/genetics (∼2), pediatrics/psychiatry and child psychiatry (∼3), and pediatrics/rehabilitation medicine (∼1) divisions. The pediatric interns had their initial NRP training in June 2014. Thereafter, the interns were assigned by the chief residents to do their initial NICU rotation at either delivery hospital 1 or delivery hospital 2 (Fig 1). The interns who rotated at delivery hospital 1 were the intervention group. At the time of their NICU rotation, they were approached by the study coordinator and enrolled into the study with written informed consent. The interns at delivery hospital 2 received the standard curriculum consisting of a tour of the delivery room and a review of resuscitation equipment at the beginning of the rotation, with no additional simulation included.

FIGURE 1

Study enrollment.

FIGURE 1

Study enrollment.

Close modal

We performed a pilot study using the teaching techniques and Action Plan of Helping Babies Breathe (HBB), First Edition,33  a basic skills-based neonatal resuscitation curriculum. Given the emphasis of the Accreditation Council on Graduate Medical Education on BMV and recognizing that the majority of infants respond to the initial steps of resuscitation and BMV,10,34  we used the methodology of HBB because of its demonstrated success in teaching BMV and the steps to improve ventilation.9,28,3538  The module focuses on the steps that must be performed within the “Golden Minute” after birth and emphasizes that the newborn should be breathing spontaneously or receiving effective BMV by the end of the first minute after birth.

The teaching occurred within 3 to 5 days of rotation start, during the residents’ normal 45-minute morning teaching period before rounds, to capture both day- and night-shift residents. The HBB Action Plan was used to illustrate the sequence of basic resuscitation interventions. Residents had the opportunity to practice the skills of drying, stimulation, and suction, BMV with a self-inflating bag, and the steps to improve ventilation. NeoNatalie mannequins (Laerdal Global Health, Stavanger, Norway), which simulate crying, breathing, and effective BMV with chest wall excursion, were used for practice. At the end of the session, each participant was individually assessed with an objective structured clinical evaluation (OSCE), described below.

In addition to the Golden Minute module at delivery hospital 1, advanced resuscitation and other positive-pressure ventilation devices were covered on additional days of standardized teaching. One day was spent on simulation involving advanced resuscitation practices including endotracheal intubation, chest compression, and epinephrine administration. The second day was spent with a respiratory therapist reviewing the setup and use of devices such as the T-piece resuscitator, continuous positive airway pressure, and flow-inflating bag-mask devices. At the second delivery hospital, a review of NR was performed during orientation, but there was no skills-based review or assessment done in a formalized fashion.

After the Golden Minute session, the residents were randomly assigned in blocks of 8 to 1 of 4 groups on the basis of a factorial design. They were randomly assigned to 2 levels of intensity of curriculum booster during their NICU rotation (none or weekly) and 2 levels of frequency of testing (1 vs 3 months) after their NICU month. All study residents were assessed with the OSCE and survey after the teaching sessions and immediately post-NICU rotation. During the NICU month, if randomly assigned to weekly boosters, the resident would meet with 1 of the evaluators weekly (an additional 2–3 times during the month) to perform the simulation and/or evaluation. After the NICU month, the resident was emailed to meet with an evaluator at a convenient date and location where the mannequin was brought directly to the resident for just-in-place simulation according to the appropriate interval. At these meetings, both the survey and OSCE evaluation were performed. These locations varied from patient care units or clinics where the residents were rotating on clinical service to the lecture hall after noon conference. The number of evaluations after the NICU month depended on when the resident was initially enrolled into the study (month of the resident’s NICU rotation) because the resident was managed for the remainder of intern year according to the randomized interval.

There were 3 evaluators for the study: the first was a neonatologist who is an NRP instructor and HBB master trainer, the second was an emergency medicine physician who is an HBB master trainer and simulation expert, and the third was a research coordinator who had been trained as an HBB master trainer. The evaluators were trained by the principal investigator in OSCE grading. Evaluators were assigned to meet with residents according to their availability, with an attempt to give each evaluator an equal number of residents to contact, meet, and evaluate.

The OSCE was a complex resuscitation that required the initial steps of resuscitation described above and BMV (Supplemental Information). It was modified from OSCE B in HBB, which has been evaluated in other global health settings.33,38  To pass, the learner had to reach a preset mastery level, defined as performing 15 of the 21 items correctly. Certain life-saving steps had to be correctly performed for the learner to pass, including recognition of a nonbreathing infant, assessment of chest movement with BMV, ventilation at 40 to 60 breaths per minute with a self-inflating bag, and steps to improve noneffective ventilation (assessment of correct mask size and placement, repositioning the airway and suction). If the resident did not pass, the simulation, scoring, and debriefing process were repeated until that was achieved. After each simulation and OSCE evaluation, the resident first had time to self-reflect on his or her performance and then obtained real-time immediate individual feedback on how he or she did by the evaluator. The OSCE and debriefing usually took ∼10 minutes.

Outcomes

Our primary outcome was OSCE score assessed at the end of the intern year (September 2015). Secondary outcomes included the OSCE score at the end of the NICU month, performance of effective BMV in OSCE simulation, time to initiate effective BMV, and passing OSCE on the first attempt. A global rating score was also added toward the end of the pilot year to provide additional information about performance. We also collected data on each step of the OSCE and whether it was correctly performed or not.

If the resident performed BMV or positive-pressure ventilation in the delivery room, they were asked to give their supervisor an evaluation form rating performance on the same steps of resuscitation as the OSCE. At the end-of-year OSCE, interns were surveyed about career choice and open-ended questions regarding the Golden Minute teaching, simulation scenario, mannequin, and frequency of simulation.

Data Analysis

Descriptive statistics were used to describe individual demographics, characteristics, delivery room attendance, and resuscitation skills. Regression models were used to compare skill retention across levels of curriculum intensity and frequency of practice. The control group was included in all end-of-year final OSCE models. For continuous outcomes, we conducted general linear models with an identity link function; for categorical outcomes, we conducted logistic regression to estimate the parameters and test the significance of the main effects (intensity of curriculum and frequency of practice) and interaction (through contrasts). Outcomes were also compared at the end of the NICU month between those who had weekly curriculum boosters to those with no boosters. We examined the association between intensive care as a career choice and passing the final OSCE on first attempt with Fisher’s exact test. Statistical significance was declared at P < .05. SAS version 9.4 (SAS Institute, Inc, Cary, NC) was used to conduct all analyses.

Twenty-eight interns who rotated at delivery hospital 1 and participated in Golden Minute teaching were consented for the study (Fig 1). Four interns withdrew after the NICU month (2 in each of the 1- and 3-month groups) and were not included in any analyses.

Twenty-four interns completed the study; 13 were randomly assigned to follow-up simulations monthly, whereas 11 were assigned to follow-up simulations every 3 months (Fig 1). Although a greater number of interns with interest in critical care specialties such as NICU, PICU, or emergency medicine were in the 1-month group, this was not statistically different (Table 1). Interestingly, a greater proportion of interns in both 1- and 3-month groups reported gaining experience in performing BMV in real-life situations on the newborn and/or NICU rotations than any other clinical rotations (emergency department, pediatric intensive care or other).

TABLE 1

Characteristics of Study Participants

CharacteristicFollow-up Training Every Month (N = 13)Follow-up Training Every 3 M (N = 11)Control (N = 14)
Participants with weekly boosters during NICU mo, n (%) 8 (61.5) 3 (27.3) — 
Days between OSCEs, mean (SD) 46.9 (25.2) 72.7 (19.4) — 
No. OSCEs, mean (SD) 7.8 (3.5) 4.8 (1.5) — 
Range 3–13 2–8 — 
Post-NICU mo n = 9 n = 9 — 
 Confidence in BMV, Scale: 1–7, median (IQR) 6 (5–6) 5 (5–6) — 
 Confidence in resuscitation, median (IQR) 5 (4–5) 5 (5–5) — 
Final survey confidence N = 13 n = 10a N = 14 
 Confidence in BMV, median (IQR) 6 (5–6) 5 (5–6) 5 (4–5) 
 Confidence in resuscitation, median (IQR) 5 (5–6) 5 (4–5) 4 (3–5) 
Career choice: neonatology, critical care, ED, PICU, mean (SD) 8 (61.5) 2 (20)a 3 (21.4) 
Clinical experiences by recall of residents N = 13 n = 10a N = 14 
 ≥1 BMV in delivery room NICU/normal newborn, n (%) 12 (92.3) 8 (80) 11 (78.6) 
 ≥1 BMV in emergency department, n (%) 2 (15.4) 3 (30) 4 (28.6) 
 ≥1 BMV in PICU, n (%) 3 (23.1) 1 (10) 
 ≥1 BMV in other rotations, n (%) 5 (38.5) 4 (40) 9 (64.3) 
CharacteristicFollow-up Training Every Month (N = 13)Follow-up Training Every 3 M (N = 11)Control (N = 14)
Participants with weekly boosters during NICU mo, n (%) 8 (61.5) 3 (27.3) — 
Days between OSCEs, mean (SD) 46.9 (25.2) 72.7 (19.4) — 
No. OSCEs, mean (SD) 7.8 (3.5) 4.8 (1.5) — 
Range 3–13 2–8 — 
Post-NICU mo n = 9 n = 9 — 
 Confidence in BMV, Scale: 1–7, median (IQR) 6 (5–6) 5 (5–6) — 
 Confidence in resuscitation, median (IQR) 5 (4–5) 5 (5–5) — 
Final survey confidence N = 13 n = 10a N = 14 
 Confidence in BMV, median (IQR) 6 (5–6) 5 (5–6) 5 (4–5) 
 Confidence in resuscitation, median (IQR) 5 (5–6) 5 (4–5) 4 (3–5) 
Career choice: neonatology, critical care, ED, PICU, mean (SD) 8 (61.5) 2 (20)a 3 (21.4) 
Clinical experiences by recall of residents N = 13 n = 10a N = 14 
 ≥1 BMV in delivery room NICU/normal newborn, n (%) 12 (92.3) 8 (80) 11 (78.6) 
 ≥1 BMV in emergency department, n (%) 2 (15.4) 3 (30) 4 (28.6) 
 ≥1 BMV in PICU, n (%) 3 (23.1) 1 (10) 
 ≥1 BMV in other rotations, n (%) 5 (38.5) 4 (40) 9 (64.3) 

—, not applicable.

a

One missing survey.

Post-NICU rotation, the 1- and 3-month groups averaged 7.8 and 4.8 OSCE practices with an average of 46.9 and 72.7 days between practices, respectively (Table 1). On the end-of-year OSCE, the 1- and 3-month groups had no significant difference in score, time to BMV, or achievement of mastery (Table 2). However, the 1- and 3-month groups had higher OSCE scores than the control group (18.8 vs 18.6 vs 14.4; P < .01 for both comparisons). For the interns that completed all the initial steps correctly, there was no difference in time to initiation of ventilation between the 1- and 3-month groups. Only 1 intern in the control group performed all the initial steps correctly and thus could not be included in this comparison. Furthermore, after the intent to ventilate, there was no difference in the time to achieving effective BMV (as noted when the chest started moving) between the 1- and 3-month groups; however, both the 1- and 3-month groups were able to achieve effective ventilation more quickly than the control group (P < .05 for both comparisons; Table 2). The global rating score was also significantly higher in both the 1- and 3-month groups than in the control group (P < .01 for both comparisons). After the NICU month to the final survey, there was no significant difference in confidence in BMV or resuscitation for either group (Table 1).

TABLE 2

Final End-of-Year Comparison of Continuous Outcomes Between 1- and 3-Month Groups: Follow-up and Control

Continuous OutcomesFollow-up Every Month (N = 13)Follow-up Every 3 Month (N = 11)Control (N = 14)1 vs 3 Month1 Month vs Control3 Month vs Control
Least-Squares Mean (95% CI)P
OSCE score 18.8 (17.6 to 19.9) 18.6 (17.5 to 19.8) 14.4 (13.3 to 15.4) .96a <.0001a <.0001a 
Time to effective ventilation(s) (among those who completed all 3 initial steps) 56.7 (38.6 to 74.9) 53.6 (34.2 to 72.9) b .80 — — 
Time between intent to bag and chest moving(s) 10.6 (−5.5 to 26.7) 20.4 (2.9 to 37.8) 52.8 (35.3 to 70.3) .68a .003a .03a 
Time to start ventilation(s) 39.2 (27.7 to 50.6) 33.5 (21.1 to 46.0) 46.9 (35.9 to 57.9) .59a .78a .24a 
Global rating scale 5.1 (4.4 to 5.9) 4.6 (3.8 to 5.4) 2.1 (1.4 to 2.8) .61a <.0001a <.0001a 
Continuous OutcomesFollow-up Every Month (N = 13)Follow-up Every 3 Month (N = 11)Control (N = 14)1 vs 3 Month1 Month vs Control3 Month vs Control
Least-Squares Mean (95% CI)P
OSCE score 18.8 (17.6 to 19.9) 18.6 (17.5 to 19.8) 14.4 (13.3 to 15.4) .96a <.0001a <.0001a 
Time to effective ventilation(s) (among those who completed all 3 initial steps) 56.7 (38.6 to 74.9) 53.6 (34.2 to 72.9) b .80 — — 
Time between intent to bag and chest moving(s) 10.6 (−5.5 to 26.7) 20.4 (2.9 to 37.8) 52.8 (35.3 to 70.3) .68a .003a .03a 
Time to start ventilation(s) 39.2 (27.7 to 50.6) 33.5 (21.1 to 46.0) 46.9 (35.9 to 57.9) .59a .78a .24a 
Global rating scale 5.1 (4.4 to 5.9) 4.6 (3.8 to 5.4) 2.1 (1.4 to 2.8) .61a <.0001a <.0001a 

—, not applicable.

a

Tukey-Kramer adjusted P value for multiple comparisons.

b

N = 1 in control group who completed all 3 initial steps; model testing follow-up groups only.

In comparing performance on the end-of-year OSCE, the 1-month group had a greater probability of passing on the first attempt compared to controls, odds ratio (OR) 9.9 (95% confidence interval [CI] 1.5 to 63.7); results were similar for the 3-month group compared to controls, OR 8.1 (95% CI 1.2 to 53.2). On the end-of-year OSCE, both the 1-month and 3-month groups had higher odds of completing all the initial steps before BMV compared with controls; this was OR 20.8 (95% CI 2.0 to 211.7) for the 1-month group and OR 22.7 (95% CI 2.1 to 244.8) for the 3-month group. However, in comparing the 1- and 3-month groups, there were no statistically significant differences in any of these outcomes.

For the subanalysis regarding performance at the NICU rotation end, interns who had weekly OSCEs (N = 11) had higher OSCE scores than those without (N = 10; Table 3; mean 19.7 vs 16.9; P < .01). There were no differences in time to start ventilation nor in the time between starting ventilation at the chest moving (a sign of effective BMV). Three interns did not complete OSCEs at the end of the NICU month and were not included in this analysis; all 3 were assigned to the group that received no weekly OSCE. Five interns did not complete the ideal number of weekly OSCEs desired before the end-of-month OSCE; that is, 2 interns had no weekly OSCEs performed, and 3 interns had 1 weekly OSCE performed. Two residents received real-life experience performing positive-pressure ventilation in the delivery room and had their supervisors complete assessment forms. In these 2 situations, the residents were able to successfully perform positive-pressure ventilation, including checking the equipment, providing BMV at the correct rate, and confirming that ventilation was effective.

TABLE 3

Subanalysis: Differences in Outcomes at the End of the NICU Month Rotation Between Residents With Weekly Curriculum Boosters Versus No Boosters

Continuous OutcomesWeekly Boosters (N = 11)No Boosters (N = 10)P
Least-Squares Mean (95% CI)
OSCE score 19.7 (19.0 to 20.4) 16.9 (16.2 to 17.6) <.0001 
Time between intent to bag and chest moving(s) 27.5 (9.5 to 45.6) 19.0 (−0.97 to 39.0) .51 
Time to start ventilation(s) 35.4 (28.1 to 42.6) 40.5 (32.9 to 48.1) .32 
Continuous OutcomesWeekly Boosters (N = 11)No Boosters (N = 10)P
Least-Squares Mean (95% CI)
OSCE score 19.7 (19.0 to 20.4) 16.9 (16.2 to 17.6) <.0001 
Time between intent to bag and chest moving(s) 27.5 (9.5 to 45.6) 19.0 (−0.97 to 39.0) .51 
Time to start ventilation(s) 35.4 (28.1 to 42.6) 40.5 (32.9 to 48.1) .32 

Retention of life-saving resuscitation skills is a challenge after resuscitation workshops. The NRP has been at the forefront in incorporating simulation-based training in teaching life support to health care providers.39  However, the seventh edition of the NRP is vague in specific recommendations for the frequency of practice or refresher training required to maintain skills.22  It is suggested in this pilot study that regularly scheduled just-in-place simulation both during and after the NICU rotation can assist pediatric interns in mastery of basic NR skills in simulated performance. Although the initial intensity of weekly practices allowed for improved performance in NR skills by the NICU rotation end, our small group sample sizes did not allow for analysis of intensity at the end of the study. However, in the long-term, the study confirmed that short, frequent practices, whether at 1 or 3 months, could improve skills over baseline.40  The lack of a difference in final OSCE scores between the 1- and 3-month groups suggests there may be a plateau after a certain number of practices indicating mastery and that a certain number of practices may cause “saturation” and allow for the resident to master the skill.

This study reveals some feasibility of implementing an ongoing system of practice within a busy pediatric residency program, challenged by duty restrictions, shift work, and off-site rotations. Although our original intention was to compare a practice simulation frequency of 1 versus 3 months for skill retention, our 2 groups did not have the ideal separation in days between practices because of challenges of manpower and rotating schedules. This speaks to the challenges of scheduled trainings for residents, which include extra staff to schedule, meet, and debrief with residents at the appointed time.

Furthermore, although study interventions were well received by residents, it is unknown whether the intervention impacted clinical care. Only 2 residents had documented attempts to practice their BMV skills in real-life delivery scenarios. Authors of a recent study performed in a US tertiary academic center observed >1100 deliveries over an 18-month period and found that 6% of infants received positive-pressure ventilation,41  which underscores challenges of ensuring that residents have sufficient experience at performing BMV. In fact, several residents revealed that our study was the only experience they had with BMV during their intern year.

Our study faced several challenges. Most significantly, because of personal issues, two evaluators were not readily available to meet with residents for their final OSCEs; therefore, the last OSCEs for many study participants were closer together than desired by the original design. The study was also challenged by waiting for residents to respond to e-mails to schedule sessions; many residents did not regularly check or respond to e-mail as a primary mode of communication.

Given the positive effect of ongoing practice, we have expanded the Golden Minute teaching to all residents and have enrolled all interns at both delivery hospitals into a larger, more comprehensive study, with analysis still underway. We have incorporated some work-arounds that were based on the challenges that arose during this pilot study, including more evaluators trained in HBB and the NRP to assist with the ongoing evaluations. We added contacting residents by text message to obtain a more immediate answer of an ideal time and place to meet. Given that our evaluators were unblinded to the randomization of study groups and could have preferentially scored those practicing more frequently, we assigned a research coordinator to assign evaluators to meet with the residents; this enabled the evaluators to remain blinded to frequency (1 or 3 months) of each resident’s random assignment.

There were some limitations to the study. First, ideally, we should have assessed the interns’ NR skills before the Golden Minute teaching. On the basis of our experience, this would have taken 10 minutes per resident, which on average would add 30 to 40 minutes to the initial training if 2 evaluations were required. This likely would have taken too much time away from patient care. Also, because the training mannequin was new to each resident and is introduced during the initial training session, we may have underestimated skill by pretesting the residents with an unfamiliar piece of equipment. Second, there may have been differences between the 2 delivery hospitals that could have been potential confounders in resident performance, such as the more standardized teaching on NR at hospital 1. Third, our modified OSCE has not been validated, but it was heavily based on the OSCE from HBB, which has been widely used and evaluated in global health settings.38  It is unknown how performance on the OSCE correlates to real-life clinical performance. Fourth, given that this was a pilot study, we used a convenience sample rather than having an a priori sample size calculation. Authors of future studies can use the results of our initial, baseline evaluations, and improvements in the intervention group to help create sample size estimates. Fifth, we recognize that our results from a single institution may not be generalizable to all centers. We were not able to schedule a time to meet with some residents within their assigned practice window because of availability issues on both the part of the evaluators and residents themselves. Although a lot of manpower was used to ensure residents were supervised for practices at the designated study interval, it is suggested in our study that brief 5- to 10-minute practices with BMV aid in skill retention; how and when this is accomplished may differ according to the structure of a residency program. As done in some low-resource settings, these could potentially be done as a self-motivated check at the beginning of a shift, which would make the realization of these practices much easier if no direct supervision were required.36,37  Sixth, we did not attempt to analyze for a correlation between resident confidence and OSCE score. Although confidence and/or comfort are lower-level educational outcomes, authors of future studies could attempt to assess whether these outcomes predict success in NR and/or BMV. If so, learners with lower levels of confidence or comfort may benefit from more intense or frequent training. Finally, given that the OSCE was used as both the simulation and assessment tool, it could be argued that the interns who practiced more frequently were more familiar with the assessment and therefore performed better. However, NR is based on an algorithm, and performing the life-saving interventions in that specific order on the basis of muscle memory obtained through simulation is exactly what we hope to achieve.

Simulation is an effective means to teach BMV, particularly when residents have few opportunities to perform this skill in the delivery room setting. More effective teaching of BMV skills in the neonatal setting may also help residents to perform the skill in other clinical pediatric settings and be pertinent to all residents, not just those entering neonatal-perinatal medicine. Just-in-place simulation with a low-fidelity mannequin enables residents to perform low-dose and high-frequency practice, allowing for better acquisition of the most essential NR skills. Future studies should be used to further evaluate the question of 1- vs 3-month practice after the NICU rotation and whether there is an ideal “dose” or threshold of practice that allows for mastery of skills in a larger population of pediatric residents.

We thank the pediatric residents at Cincinnati Children’s Hospital Medical Center who participated in the study. We also thank pediatric residency program directors Javier Gonzalez del Rey and Sue Poynter Wong and the pediatric chief residents for their support of the project.

FUNDING: Funded by an Association of American Medical Colleges Clinical Care Innovation Pilot Award and the Cincinnati Children’s Hospital Medical Center Perinatal Institute Pilot and Feasibility Award.

Dr Kamath-Rayne conceptualized and designed the study, collected data, and drafted the initial manuscript; Ms Taylor collected data and critically reviewed the manuscript; Ms Tabangin performed data analysis and critically reviewed the manuscript; Dr Geis designed the study, collected data, and critically reviewed the manuscript; and all authors approved the final manuscript as submitted.

1
Kamath-Rayne
BD
,
Griffin
JB
,
Moran
K
, et al
.
Resuscitation and obstetrical care to reduce intrapartum-related neonatal deaths: A MANDATE study
.
Matern Child Health J
.
2015
;
19
(
8
):
1853
1863
2
Lee
AC
,
Cousens
S
,
Wall
SN
, et al
.
Neonatal resuscitation and immediate newborn assessment and stimulation for the prevention of neonatal deaths: a systematic review, meta-analysis and Delphi estimation of mortality effect
.
BMC Public Health
.
2011
;
11
(
suppl 3
):
S12
3
Cordero
L
,
Hart
BJ
,
Hardin
R
,
Mahan
JD
,
Giannone
PJ
,
Nankervis
CA
.
Pediatrics residents’ preparedness for neonatal resuscitation assessed using high-fidelity simulation
.
J Grad Med Educ
.
2013
;
5
(
3
):
399
404
4
Cusack
J
,
Fawke
J
.
Neonatal resuscitation: are your trainees performing as you think they are? A retrospective review of a structured resuscitation assessment for neonatal medical trainees over an 8-year period
.
Arch Dis Child Fetal Neonatal Ed
.
2012
;
97
(
4
):
F246
F248
5
Ersdal
HL
,
Vossius
C
,
Bayo
E
, et al
.
A one-day “Helping Babies Breathe” course improves simulated performance but not clinical management of neonates
.
Resuscitation
.
2013
;
84
(
10
):
1422
1427
6
Patel
J
,
Posencheg
M
,
Ades
A
.
Proficiency and retention of neonatal resuscitation skills by pediatric residents
.
Pediatrics
.
2012
;
130
(
3
):
515
521
7
Singhal
N
,
Lockyer
J
,
Fidler
H
, et al
.
Helping Babies Breathe: global neonatal resuscitation program development and formative educational evaluation
.
Resuscitation
.
2012
;
83
(
1
):
90
96
8
Skidmore
MB
,
Urquhart
H
.
Retention of skills in neonatal resuscitation
.
Paediatr Child Health
.
2001
;
6
(
1
):
31
35
9
Tabangin
ME
,
Josyula
S
,
Taylor
KK
,
Vasquez
JC
,
Kamath-Rayne
BD
.
Resuscitation skills after Helping Babies Breathe training: a comparison of varying practice frequency and impact on retention of skills in different types of providers
.
Int Health
.
2018
;
10
(
3
):
163
171
10
Accreditation Council on Graduate Medical Education
.
ACGME program requirements for graduate medical education in pediatrics 2019. Available at: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320_Pediatrics_2019_TCC.pdf?ver=2019-03-28-090932-520. Accessed May 31, 2019
11
Cordero
L
,
Hart
BJ
,
Hardin
R
,
Mahan
JD
,
Nankervis
CA
.
Deliberate practice improves pediatric residents’ skills and team behaviors during simulated neonatal resuscitation
.
Clin Pediatr (Phila)
.
2013
;
52
(
8
):
747
752
12
Mileder
LP
,
Urlesberger
B
,
Szyld
EG
,
Roehr
CC
,
Schmölzer
GM
.
Simulation-based neonatal and infant resuscitation teaching: a systematic review of randomized controlled trials
.
Klin Padiatr
.
2014
;
226
(
5
):
259
267
13
Nadel
FM
,
Lavelle
JM
,
Fein
JA
,
Giardino
AP
,
Decker
JM
,
Durbin
DR
.
Teaching resuscitation to pediatric residents: the effects of an intervention
.
Arch Pediatr Adolesc Med
.
2000
;
154
(
10
):
1049
1054
14
Sawyer
T
,
Sierocka-Castaneda
A
,
Chan
D
, et al
.
Deliberate practice using simulation improves neonatal resuscitation performance
.
Simul Healthc
.
2011
;
6
(
6
):
327
336
15
Schroedl
CJ
,
Corbridge
TC
,
Cohen
ER
, et al
.
Use of simulation-based education to improve resident learning and patient care in the medical intensive care unit: a randomized trial
.
J Crit Care
.
2012
;
27
(
2
):
219.e7
219.e13
16
Stross
JK
.
Maintaining competency in advanced cardiac life support skills
.
JAMA
.
1983
;
249
(
24
):
3339
3341
17
Surcouf
JW
,
Chauvin
SW
,
Ferry
J
,
Yang
T
,
Barkemeyer
B
.
Enhancing residents’ neonatal resuscitation competency through unannounced simulation-based training
.
Med Educ Online
.
2013
;
18
:
1
7
18
van Schaik
SM
,
Von Kohorn
I
,
O’Sullivan
P
.
Pediatric resident confidence in resuscitation skills relates to mock code experience
.
Clin Pediatr (Phila)
.
2008
;
47
(
8
):
777
783
19
Klig
JE
,
O’Malley
PJ
.
Pediatric office emergencies
.
Curr Opin Pediatr
.
2007
;
19
(
5
):
591
596
20
Frush
K
;
American Academy of Pediatrics Committee on Pediatric Emergency Medicine
.
Preparation for emergencies in the offices of pediatricians and pediatric primary care providers
.
Pediatrics
.
2007
;
120
(
1
):
200
212
21
Yuknis
ML
,
Weinstein
E
,
Maxey
H
, et al
.
Frequency of pediatric emergencies in ambulatory practices
.
Pediatrics
.
2018
;
142
(
2
):
e20173082
22
Wyllie
J
,
Perlman
JM
,
Kattwinkel
J
, et al
;
Neonatal Resuscitation Chapter Collaborators
.
Part 7: neonatal resuscitation: 2015 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science with Treatment Recommendations
.
Resuscitation
.
2015
;
95
:
e169
e201
23
Ersdal
HL
,
Singhal
N
.
Resuscitation in resource-limited settings
.
Semin Fetal Neonatal Med
.
2013
;
18
(
6
):
373
378
24
Perlman
J
,
Kattwinkel
J
,
Wyllie
J
,
Guinsburg
R
,
Velaphi
S
;
Nalini Singhal for the Neonatal ILCOR Task Force Group
.
Neonatal resuscitation: in pursuit of evidence gaps in knowledge
.
Resuscitation
.
2012
;
83
(
5
):
545
550
25
International Liaison Committee on Resuscitation
.
Resuscitation training frequency. 2015. Available at: https://volunteer.heart.org/apps/pico/Pages/PublicComment.aspx?q=859. Accessed April 23, 2018
26
Sutton
RM
,
Niles
D
,
Meaney
PA
, et al
.
Low-dose, high-frequency CPR training improves skill retention of in-hospital pediatric providers
.
Pediatrics
.
2011
;
128
(
1
).
27
Sutton
RM
,
Niles
D
,
Meaney
PA
, et al
.
“Booster” training: evaluation of instructor-led bedside cardiopulmonary resuscitation skill training and automated corrective feedback to improve cardiopulmonary resuscitation compliance of Pediatric Basic Life Support providers during simulated cardiac arrest
.
Pediatr Crit Care Med
.
2011
;
12
(
3
):
e116
e121
28
Mduma
E
,
Ersdal
H
,
Svensen
E
,
Kidanto
H
,
Auestad
B
,
Perlman
J
.
Frequent brief on-site simulation training and reduction in 24-h neonatal mortality–an educational intervention study
.
Resuscitation
.
2015
;
93
:
1
7
29
Niles
D
,
Sutton
RM
,
Donoghue
A
, et al
.
“Rolling Refreshers”: a novel approach to maintain CPR psychomotor skill competence
.
Resuscitation
.
2009
;
80
(
8
):
909
912
30
Niles
DE
,
Nishisaki
A
,
Sutton
RM
, et al
.
Improved retention of chest compression psychomotor skills with brief “rolling refresher” training
.
Simul Healthc
.
2017
;
12
(
4
):
213
219
31
Scholtz
AK
,
Monachino
AM
,
Nishisaki
A
,
Nadkarni
VM
,
Lengetti
E
.
Central venous catheter dress rehearsals: translating simulation training to patient care and outcomes
.
Simul Healthc
.
2013
;
8
(
5
):
341
349
32
Kessler
D
,
Pusic
M
,
Chang
TP
, et al
;
INSPIRE LP Investigators
.
Impact of just-in-time and just-in-place simulation on intern success with infant lumbar puncture
.
Pediatrics
.
2015
;
135
(
5
).
33
Niermeyer
S
,
Keenan
W
,
Little
G
,
Singhal
N
;
American Academy of Pediatrics
;
Laerdal Global Health
.
Helping babies breathe: facilitator flip chart. 2010. Available at: http://internationalresources.aap.org/Resource/ShowFile?documentName=hbb_flipchart_english.pdf
34
Kamath-Rayne
BD
,
Berkelhamer
SK
,
Kc
A
,
Ersdal
HL
,
Niermeyer
S
.
Neonatal resuscitation in global health settings: an examination of the past to prepare for the future
.
Pediatr Res
.
2017
;
82
(
2
):
194
200
35
Kamath-Rayne
BD
,
Josyula
S
,
Rule
ARL
,
Vasquez
JC
.
Improvements in the delivery of resuscitation and newborn care after Helping Babies Breathe training
.
J Perinatol
.
2017
;
37
(
10
):
1153
1160
36
Kc
A
,
Wrammert
J
,
Clark
RB
, et al
.
Reducing perinatal mortality in Nepal using helping babies breathe
.
Pediatrics
.
2016
;
137
(
6
):
e20150117
37
Rule
ARL
,
Tabangin
M
,
Cheruiyot
D
,
Mueri
P
,
Kamath-Rayne
BD
.
The call and the challenge of pediatric resuscitation and simulation research in low-resource settings
.
Simul Healthc
.
2017
;
12
(
6
):
402
406
38
Seto
TL
,
Tabangin
ME
,
Taylor
KK
,
Josyula
S
,
Vasquez
JC
,
Kamath-Rayne
BD
.
Breaking down the objective structured clinical examination: an evaluation of the helping babies breathe OSCEs
.
Simul Healthc
.
2017
;
12
(
4
):
226
232
39
Halamek
LP
.
Simulation and debriefing in neonatology 2016: mission incomplete
.
Semin Perinatol
.
2016
;
40
(
7
):
489
493
40
Oermann
MH
,
Kardong-Edgren
SE
,
Odom-Maryon
T
.
Effects of monthly practice on nursing students’ CPR psychomotor skill performance
.
Resuscitation
.
2011
;
82
(
4
):
447
453
41
Niles
DE
,
Cines
C
,
Insley
E
, et al
.
Incidence and characteristics of positive pressure ventilation delivered to newborns in a US tertiary academic hospital
.
Resuscitation
.
2017
;
115
:
102
109

Competing Interests

POTENTIAL CONFLICT OF INTEREST: Dr Kamath-Rayne was the associate editor of Helping Babies Breathe, Second Edition, and serves on the American Academy of Pediatrics Helping Babies Survive planning group. The American Academy of Pediatrics did not contribute to the design or analysis of this study; the other authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data