OBJECTIVES

The rare event of handling critically ill children often challenge the emergency care team. Several studies have investigated effects of simulation-based team training to prepare for such events, but the body of evidence remains to be compiled. We performed a systematic review of the effects of simulation-based team training on clinical performance and patient outcome.

METHODS

From a search of MEDLINE, Embase, CINAHL, and Cochrane Library, we included studies of team training in emergency pediatric settings with reported clinical performance and patient outcomes. We extracted data using a predefined template and assessed risk of bias using the Cochrane risk-of-bias tool for randomized trials 2.0 and the Newcastle Ottawa Quality Assessment Scale.

RESULTS

We screened 1926 abstracts and included 79 studies. We identified 15 studies reporting clinical health care professional performance or patient outcomes. Four studies reported survival data, 5 reported time-critical clinical events, 5 reported adherence to guidelines, checklists or tasks, and 2 reported on airway management. Randomized studies revealed improved team performance in simulated reevaluations 2 to 6 months after intervention. A meta-analysis was impossible because of heterogeneous interventions and outcomes. Most included studies had significant methodological limitations.

CONCLUSIONS

Pediatric simulation-based team training improves clinical performance in time-critical tasks and adherence to guidelines. Improved survival was indicated but not concluded because of high risk of bias. Team performance and technical skills improved for at least 2 to 6 months. Future research should include longer-term measures of skill retention and patient outcomes or clinical measures of treatment quality whenever possible.

Pediatricians are expected to manage acutely ill children in high-stakes, time-sensitive situations. Pediatrics is a broad specialty with a wide range of subspecialties, patient ages, and complex clinical conditions. Therefore, specific acute situations are rare, and as a consequence, pediatricians are rarely exposed to them, emphasizing the importance of maintaining a coordinated and efficient emergency team response.1 

Simulation-based team training is defined as simulation beyond technical skills that aims to mimic real acute situations where performance is influenced by teamwork, coordination, team behavior, communication, leadership, decision-making, and situational awareness.24  The use of advanced medical simulation training in pediatrics has increased rapidly in recent years.2  However, little is known about the retention of attained skills, knowledge, and behavior and how these can transfer to clinical encounters and emergency situations and, ultimately, how they may improve patient safety and outcome.

We have previously reviewed simulation-based team training in neonatal resuscitation and concluded that it improves team performance.5  However, no firm conclusions on survival and other clinical outcomes could be reached. We were unable to identify a systematic review of simulation-based team training in the pediatric nonneonatal setting to answer the following questions: Does simulation-based team training improve the performance of the team? Does it improve patient outcome and safety? We therefore performed a systematic review to describe the current state of evidence and point out areas needing more research.

This review is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA).6  We registered the protocol at the International Prospective Register of Systematic Reviews repository (CRD42019128213; submitted June 24, 2019, and published September 6, 2019).

We included studies of all designs if they met the criteria stated below.

Population

Studies including health care professionals with emergency care for pediatric patients were included. Health care professionals included nurses, physicians, paramedics, and respiratory therapists with clinical responsibilities in hospital or prehospital settings. We excluded studies on pregraduate simulation training.

Intervention

Interventions included were simulation-based team training of pediatric emergencies in situ or in a training facility. We defined team training as ≥2 health care professionals in an acute clinical situation requiring a coordinated effort for a successful outcome.

Comparators

We included single-group studies comparing the performance of health care professionals before (comparator) and after (intervention) simulation training, ie, pre- versus postintervention designs. Studies of team skills retention with several evaluations over time (≥2) were also included. In intervention studies with ≥2 groups, either randomized or nonrandomized, the comparator group could be subjected to usual training or no training. We accepted comparators with different types or intensities of the simulation training.

Outcomes

The simulation training outcomes were evaluated according to the framework of Kirkpatrick’s 4 levels of training evaluation and transferring learning to behavior.7  Given our aim, we did not include reaction outcomes (level 1), which correspond to professionals’ evaluation of the simulation training. Learning outcomes (level 2) were included as self-reported changes in, eg, knowledge, attitude, confidence, preparedness, self-efficacy, and technical and nontechnical skills. Technical skills were defined as adequacy of the actions taken from a mediacal and techbical perspective; nontechnical skills were defined as decision-making and team interaction processes used during the team’s management of a scenario or a clinical situation. Behavior outcomes (level 3) were included as observed technical skills and team performance (nontechnical skills) in the simulation setting. Patient outcomes (level 4) were included as a change in clinical performance of health care professionals (eg, time to critical task, adherence to protocol) or a change in patient outcomes (eg, survival) of pediatric emergencies.

We performed a combined search aimed at identifying all articles on simulation-based team training in emergency pediatrics and neonatology, doing so because simulation training programs may include both pediatric (nonneonatal) and neonatal patients. An experienced medical librarian (H.L.) and a subject specialist (M.S.L.) devised a search strategy for PubMed to identify studies concerning neonatology and pediatrics and team-based simulation training, and emergencies. The search strategy was then adapted to Embase, CINAHL, and Cochrane Library. Studies were limited to English language; however, no limit was placed on year of publication. Details of the search strategy are available in Supplemental Tables 68. The final search was run on February 1, 2021. After data extraction, the articles were divided into a pediatric (nonneonatal) and a neonatal review.5  Two studies contained relevant data on both nonneonatal and neonatal simulations and were included in both reviews.8,9 

Two authors (S.T. and M.S.L.) independently screened titles and abstracts of all identified studies. Studies meeting or potentially meeting the inclusion criteria and studies with insufficient information were included for full-text review (see Fig 1 for the PRISMA flowchart10 ). Two authors (S.T. and M.S.L.) independently performed the full-text reviews, and any disagreements were resolved by discussion to consensus or by consulting a third author in cases of nonconsensus (T.B.H.). The study selection process was managed by the Covidence review platform (https://www.covidence.org), which facilitates co-reviewer blinded abstract and full-text screening, study inclusion, and resolution of conflicts.

FIGURE 1

PRISMA flowchart of the study selection process.

FIGURE 1

PRISMA flowchart of the study selection process.

Close modal

Two authors (S.T. and A.W.S.) extracted data by using a predefined template developed for the purpose. Authors of studies with missing or inadequate information were not contacted. The following information was extracted from the studies: author, year, country, simulation setting, study design, intervention, comparator, number of participants, profession of participants, participant/instructor ratio, simulation and debriefing duration, simulation frequency (if applicable), level of fidelity, in situ or simulation center, retest timing (if applicable), self-reported outcomes, observed simulation outcomes, observed clinical behavior outcomes, change in clinical performance of health care professionals outcomes, and change in patient outcomes.

Two authors (M.S.L. and A.W.S.) evaluated randomized studies according to the revised Cochrane risk-of-bias tool for randomized trials 2.0 (RoB 2).11  Separate score tools were applied for individually randomized parallel group trials and for cluster randomized trials according to study design. The same 2 authors evaluated nonrandomized studies according to the Newcastle Ottawa Quality Assessment Scale (NOS).11  We used the NOS with the adaptations and operational definitions for evaluating medical education research as suggested by Cook and Reed.12 

The risk-of-bias evaluation was presented for all randomized studies, regardless of outcome studied. For nonrandomized studies, the risk-of-bias evaluation was presented for studies reporting clinical performance or patient outcomes, regardless of study type, and for studies reporting team performance outcomes with a control group. The risk-of-bias evaluation was used for the synthesis and discussion of the results of this systematic review. The bias evaluation had no impact on study inclusion or exclusion.

We performed a narrative synthesis of the included studies because of lack of comparability in design and interventions and heterogeneous outcomes. No single summary measure was applicable. We focused the main synthesis on the hierarchical Kirkpatrick levels and high-quality studies. We presented all available studies for each level to provide full transparency (Table 1). In our reporting of the results, statistical significance was considered at the P < .05 level. Estimates and P values are reported in the tables.

TABLE 1

Characteristics of 79 Included Studies Sorted by Study Design, Outcome Kirkpatrick Level, and Year

Outcome Kirkpatrick Levela
SourceYearCountryDesignNo. of Participants234
Besbes et al17  2021 Tunisia Randomized 33   
Bragard et al13  2019 Belgium Randomized 16  
Mariani et al15  2019 United States Randomized 18  
Stellflug and Lowe21  2018 United States Randomized 94  
Fagan et al19  2018 United States Randomized 26   
Lemke et al20  2019 United States Randomized 30   
Bultas et al18  2014 United States Randomized 33   
Kurosawa et al14  2014 United States Randomized 40   
Sudikoff et al16  2009 United States Randomized 16   
Happel et al41  2015 United States Cohort NA   
McLaughlin et al38  2018 United States Cohort 149   
Shah et al22  2016 United States Cohort 250   
Qian et al29  2016 China Cohort NA   
Dowson et al39  2013 England Cohort 20   
Auerbach et al40  2011 United States Cohort 115   
Nishisaki et al35  2011 United States Cohort 265   
Nishisaki et al23  2010 United States Cohort 265   
Gilfoyle et al24  2007 Canada Cohort 15   
Lee et al42  2021 United States Time series NA   
Diaz and Dawson36  2020 United States Time series 112   
Hazwani et al34  2020 Saudi Arabia Time series NA   
Hazwani et al43  2020 Saudi Arabia Time series NA   
Colman et al37  2019 United States Time series 165  
Colman et al44  2019 United States Time series 165   
Sawyer et al25  2019 United States Time series 332   
Barni et al30  2019 Italy Time series 30   
Di Nardo et al26  2018 Italy Time series NA  
Yager et al45  2016 United States Time series NA   
Auerbach et al46  2014 United States Time series 398   
Su et al27  2014 United States Time series NA   
Theilen et al31  2013 United Kingdom Time series NA   
Andreatta et al33  2011 United States Time series 228   
van Schaik et al47  2011 United States Time series 61   
Falcone et al48  2008 United States Time series 160   
Brown et al49  2021 United States Pre/post NA  
Cory et al50  2020 United States Pre/post 144  
Karageorge et al51  2020 United States Pre/post 24  
Lutfi et al52  2019 United States Pre/post 367   
Saqe-Rockoff et al53  2019 United States Pre/post 43  
Bayouth et al54  2018 United States Pre/post 99   
Brown et al55  2018 United States Pre/post 30  
Emani et al56  2018 Asia Pre/post 23  
Ryan et al57  2019 United States Pre/post  
Katznelson et al58  2018 United States Pre/post NA   
Couloures et al59  2017 United States Pre/post 23  
Gilfoyle et al60  2017 United States Pre/post 300   
Martin et al32  2016 United States Pre/post 83 
Stone et al61  2014 United States Pre/post 60   
Chan et al62  2013 United States Pre/post 26  
Kennedy et al63  2013 United States Pre/post 26   
Patterson et al64  2013 United States Pre/post 289  
Patterson et al65  2013 United States Pre/post 218   
Burton et al28  2011 United States Pre/post 19 
Tofil et al66  2011 United States Pre/post 30  
Hunt et al 67  2007 United States Pre/post NA   
Tsai et al68  2006 Canada Pre/post 18   
Peterson et al69  2020 United States Pre/post 27   
Monachino et al70  2019 United States Pre/post 211   
Abulebda et al71  2018 United States Pre/post NA   
Bragard et al9  2018 Belgium Pre/post 16   
Cristallo et al72  2021 United States Pre/post 353   
Lind et al73  2018 United States Pre/post 39   
Kalidindi et al74  2018 United States Pre/post NA   
Raffaeli et al75  2018 Italy Pre/post 28   
Saavedra et al76  2018 United States Pre/post 85   
Whitfill et al77  2018 United States Pre/post NA   
Lehner et al78  2017 Germany Pre/post 18   
Wallace et al79  2017 United States Pre/post 260   
Ross et al8  2016 United States Pre/post 17   
Katznelson et al80  2014 United States Pre/post NA   
Figueroa et al81  2013 United States Pre/post 37   
Popp et al82  2012 United States Pre/post 18   
Stocker et al83  2012 United Kingdom Pre/post 219   
Straka et al84  2012 United States Pre/post 26   
Kane et al85  2011 United States Pre/post 65   
Allan et al86  2010 United States Pre/post 182   
Tofil et al87  2010 United States Pre/post 30   
von Arx and Pretzlaff88  2010 United States Pre/post 27   
Toback et al89  2006 United States Pre/post 97   
Outcome Kirkpatrick Levela
SourceYearCountryDesignNo. of Participants234
Besbes et al17  2021 Tunisia Randomized 33   
Bragard et al13  2019 Belgium Randomized 16  
Mariani et al15  2019 United States Randomized 18  
Stellflug and Lowe21  2018 United States Randomized 94  
Fagan et al19  2018 United States Randomized 26   
Lemke et al20  2019 United States Randomized 30   
Bultas et al18  2014 United States Randomized 33   
Kurosawa et al14  2014 United States Randomized 40   
Sudikoff et al16  2009 United States Randomized 16   
Happel et al41  2015 United States Cohort NA   
McLaughlin et al38  2018 United States Cohort 149   
Shah et al22  2016 United States Cohort 250   
Qian et al29  2016 China Cohort NA   
Dowson et al39  2013 England Cohort 20   
Auerbach et al40  2011 United States Cohort 115   
Nishisaki et al35  2011 United States Cohort 265   
Nishisaki et al23  2010 United States Cohort 265   
Gilfoyle et al24  2007 Canada Cohort 15   
Lee et al42  2021 United States Time series NA   
Diaz and Dawson36  2020 United States Time series 112   
Hazwani et al34  2020 Saudi Arabia Time series NA   
Hazwani et al43  2020 Saudi Arabia Time series NA   
Colman et al37  2019 United States Time series 165  
Colman et al44  2019 United States Time series 165   
Sawyer et al25  2019 United States Time series 332   
Barni et al30  2019 Italy Time series 30   
Di Nardo et al26  2018 Italy Time series NA  
Yager et al45  2016 United States Time series NA   
Auerbach et al46  2014 United States Time series 398   
Su et al27  2014 United States Time series NA   
Theilen et al31  2013 United Kingdom Time series NA   
Andreatta et al33  2011 United States Time series 228   
van Schaik et al47  2011 United States Time series 61   
Falcone et al48  2008 United States Time series 160   
Brown et al49  2021 United States Pre/post NA  
Cory et al50  2020 United States Pre/post 144  
Karageorge et al51  2020 United States Pre/post 24  
Lutfi et al52  2019 United States Pre/post 367   
Saqe-Rockoff et al53  2019 United States Pre/post 43  
Bayouth et al54  2018 United States Pre/post 99   
Brown et al55  2018 United States Pre/post 30  
Emani et al56  2018 Asia Pre/post 23  
Ryan et al57  2019 United States Pre/post  
Katznelson et al58  2018 United States Pre/post NA   
Couloures et al59  2017 United States Pre/post 23  
Gilfoyle et al60  2017 United States Pre/post 300   
Martin et al32  2016 United States Pre/post 83 
Stone et al61  2014 United States Pre/post 60   
Chan et al62  2013 United States Pre/post 26  
Kennedy et al63  2013 United States Pre/post 26   
Patterson et al64  2013 United States Pre/post 289  
Patterson et al65  2013 United States Pre/post 218   
Burton et al28  2011 United States Pre/post 19 
Tofil et al66  2011 United States Pre/post 30  
Hunt et al 67  2007 United States Pre/post NA   
Tsai et al68  2006 Canada Pre/post 18   
Peterson et al69  2020 United States Pre/post 27   
Monachino et al70  2019 United States Pre/post 211   
Abulebda et al71  2018 United States Pre/post NA   
Bragard et al9  2018 Belgium Pre/post 16   
Cristallo et al72  2021 United States Pre/post 353   
Lind et al73  2018 United States Pre/post 39   
Kalidindi et al74  2018 United States Pre/post NA   
Raffaeli et al75  2018 Italy Pre/post 28   
Saavedra et al76  2018 United States Pre/post 85   
Whitfill et al77  2018 United States Pre/post NA   
Lehner et al78  2017 Germany Pre/post 18   
Wallace et al79  2017 United States Pre/post 260   
Ross et al8  2016 United States Pre/post 17   
Katznelson et al80  2014 United States Pre/post NA   
Figueroa et al81  2013 United States Pre/post 37   
Popp et al82  2012 United States Pre/post 18   
Stocker et al83  2012 United Kingdom Pre/post 219   
Straka et al84  2012 United States Pre/post 26   
Kane et al85  2011 United States Pre/post 65   
Allan et al86  2010 United States Pre/post 182   
Tofil et al87  2010 United States Pre/post 30   
von Arx and Pretzlaff88  2010 United States Pre/post 27   
Toback et al89  2006 United States Pre/post 97   

NA, not available (information not provided in the original article); Pre/post, pre- and postintervention study.

a

Kirkpatrick level 2 (learning), level 3 (clinical performance), and level 4 (patient outcome).

We used a funnel plot to assess potential publication bias. We used the estimate from the main outcome of each included study. Ratio estimates (odds ratios, risk ratios, incidence rate ratios) were used directly, whereas a ratio was calculated for studies using continuous outcomes, eg, intervention mean divided by comparator mean. A ratio >1.0 favored the intervention group or the postintervention estimate. Eighteen studies did not report the number of participants or effect estimates and were therefore excluded from the funnel plot.

We assessed selective reporting by inspecting preregistered studies or analysis protocols when available; for randomized studies, this was part of the RoB 2 score. Furthermore, we compared the specified analysis from the methods section with the reported outcomes in each included study.

A total of 3262 studies were identified across databases. After duplicate removal, we screened titles and abstracts of 1926 studies. A total of 223 full-text articles were assessed for eligibility. Of these, 110 met the inclusion criteria common to pediatric and neonatal populations, and 79 provided relevant data for this pediatric review (Fig 1). An overview of the excluded full-text articles is provided in the Supplemental Table 9.

Included studies were all published between 2006 and 2021 (Table 1). Eighteen studies contained a control group, and 9 of these used random allocation. Sixteen studies had a time series intervention group design, and the remaining 45 had a pre- and postintervention group design. Several studies reported outcomes at >1 Kirkpatrick level. Fifteen studies reported patient outcome (level 4), 37 reported behavior outcome (level 3), and 47 reported learning outcome (level 2) (Table 1).

The risk-of-bias judgments of the 9 randomized studies are presented in Table 2. Five studies received an overall judgment of some concern,1317  and 4 received a high-risk judgment.1821  Only 2 studies were judged as low risk for the crucial randomization domain.15,17  All studies had low risk of bias in the intervention domain, reflecting no deviations in the allocated intervention groups and intention-to-treat analyses.

TABLE 2

Risk-of-Bias Judgment for Included Randomized Studies Using the Revised RoB 2

Subdomain Judgment of Risk-of-BiasOverall Judgment of Risk of Bias
SourceYearDesignDomain 1 RandomizationDomain 1b RecruitmentDomain 2 InterventionDomain 3 Missing OutcomeDomain 4 Measuring OutcomeDomain 5 Selected Results
Besbes et al17  2021 Randomized, 2 arms Low risk NA Low risk Low risk Some concern Some concern Some concern 
Bragard et al13  2019 Cluster randomized Some concern Low risk Low risk Low risk Low risk Some concern Some concern 
Mariani et al15  2019 Randomized, 2 arms Low risk NA Low risk Low risk Some concern Some concern Some concern 
Fagan et al19  2018 Randomized, 2 arms High risk NA Low risk Low risk High risk Some concern High risk 
Stellflug and Lowe21  2018 Cluster randomized High risk Some concern Low risk Some concern Low risk Some concern High risk 
Lemke et al20  2019 Cluster randomized Some concern Some concern Low risk High risk Low risk Some concern High risk 
Bultas et al18  2014 Randomized, 2 arms Some concern NA Low risk Some concern Some concern Some concern High risk 
Kurosawa et al14  2014 Randomized, 2 arms Some concern NA Low risk Low risk Low risk Some concern Some concern 
Sudikoff et al16  2009 Randomized, 2 arms Some concern NA Low risk Low risk Low risk Some concern Some concern 
Subdomain Judgment of Risk-of-BiasOverall Judgment of Risk of Bias
SourceYearDesignDomain 1 RandomizationDomain 1b RecruitmentDomain 2 InterventionDomain 3 Missing OutcomeDomain 4 Measuring OutcomeDomain 5 Selected Results
Besbes et al17  2021 Randomized, 2 arms Low risk NA Low risk Low risk Some concern Some concern Some concern 
Bragard et al13  2019 Cluster randomized Some concern Low risk Low risk Low risk Low risk Some concern Some concern 
Mariani et al15  2019 Randomized, 2 arms Low risk NA Low risk Low risk Some concern Some concern Some concern 
Fagan et al19  2018 Randomized, 2 arms High risk NA Low risk Low risk High risk Some concern High risk 
Stellflug and Lowe21  2018 Cluster randomized High risk Some concern Low risk Some concern Low risk Some concern High risk 
Lemke et al20  2019 Cluster randomized Some concern Some concern Low risk High risk Low risk Some concern High risk 
Bultas et al18  2014 Randomized, 2 arms Some concern NA Low risk Some concern Some concern Some concern High risk 
Kurosawa et al14  2014 Randomized, 2 arms Some concern NA Low risk Low risk Low risk Some concern Some concern 
Sudikoff et al16  2009 Randomized, 2 arms Some concern NA Low risk Low risk Low risk Some concern Some concern 

NA, not applicable.

The risk-of-bias scores for the 15 nonrandomized studies with clinical performance or patient outcomes (level 4) regardless of study type and the 1 nonrandomized study with a control group reporting team performance outcomes (level 3) are presented in Table 3. Two cohort studies received an overall score of 4 out of 6,22,23  and 8 studies received a score of ≤2, including 1 cohort study with low study retention.24 

TABLE 3

Risk of Bias for Included Nonrandomized Studies on the Basis of the NOS With Adaption for Educational Research

NOS Subdomain Risk-of-Bias ScoreOverall Assessment Score (0–6)
SourceYearDesignIntervention Group RepresentativeComparison Group SelectionComparison Group ComparabilityStudy RetentionOutcome Assessment
Shah et al22  2016 Cohort 
Qian et al29  2016 Cohort 
Nishisaki et al35  2011 Cohort 
Nishisaki et al23  2010 Cohort 
Gilfoyle et al24  2007 Cohort 
Diaz and Dawson36  2020 Time series 
Hazwani et al34,43  2020 Time series 
Colman et al37,44  2019 Time series 
Sawyer et al25  2019 Time series 
Barni et al30  2019 Time series 
Di Nardo et al26  2018 Time series 
Su et al27  2014 Time series 
Theilen et al31  2013 Time series 
Andreatta et al33  2011 Time series 
Martin et al32  2016 Pre/post 
Burton et al28  2011 Pre/post 
NOS Subdomain Risk-of-Bias ScoreOverall Assessment Score (0–6)
SourceYearDesignIntervention Group RepresentativeComparison Group SelectionComparison Group ComparabilityStudy RetentionOutcome Assessment
Shah et al22  2016 Cohort 
Qian et al29  2016 Cohort 
Nishisaki et al35  2011 Cohort 
Nishisaki et al23  2010 Cohort 
Gilfoyle et al24  2007 Cohort 
Diaz and Dawson36  2020 Time series 
Hazwani et al34,43  2020 Time series 
Colman et al37,44  2019 Time series 
Sawyer et al25  2019 Time series 
Barni et al30  2019 Time series 
Di Nardo et al26  2018 Time series 
Su et al27  2014 Time series 
Theilen et al31  2013 Time series 
Andreatta et al33  2011 Time series 
Martin et al32  2016 Pre/post 
Burton et al28  2011 Pre/post 

Pre/post, pre- and postintervention study.

We included 15 studies that reported change in clinical performance of health care professionals or change in clinical outcomes (Table 4). Only 4 of the studies had a control group, but none of these used random allocation. Four of the studies simulated extracorporeal membrane oxygenation (ECMO),2528  1 septic shock,29  1 anaphylaxis,30  1 seizure,22  2 deteriorating patients,31,32  2 cardiopulmonary arrest,33,34  2 airway management,23,35  and 2 closed-loop communication and teamwork skills.36,37 

TABLE 4

Summary of 15 Studies With Change in Clinical Performance or Patient Outcome (Kirkpatrick Level 4)

Source, YearSettingDesign(No. of Participants)InterventionComparatorParticipantsCase KeywordsOutcomes
Qian et al, 201629  3 tertiary hospitals Cohort (NA) High-fidelity simulation every 3 mo at 1 intervention hospital Evaluation after 1 y 2 control hospital straining as usual Chief resident, residents, nurses (n = 5) Septic shock, first-hour basic tasks Total compliance with first-hour critical tasks: intervention hospital, 61.7%; control hospitals, 23.1% (P = .004) 
Shah et al, 201622  Prehospital Cohort (250) Transports of patients having seizures Intervention training: 9-h course, didactic, hands-on, and 4 pediatric simulations Training as usual Paramedics, 250 transports with at least 1 paramedic who attended the course Seizure The course enhanced seizure protocol compliance prehospital, 250 transports of patients having seizuresCheck blood glucose: intervention, 72%; control, 66% (P = .35)Midazolam if euglycemic: intervention, 63%; control, 56% (P = .28) 
Nishisaki et al, 201135  Children’s hospital, tertiary PICU Cohort (265) 202 high-fidelity simulations from June 2007 to August 2008: 30-min hands-on, simulation, and debriefingActual intubations (n = 15) were observed in daytime Same hospital Nontrained residents Pediatric residents, EM residents, nurses, respiratory therapists Clinical performance and team behavior during airway management, including tracheal intubation The presence of ≥2 simulation-trained providers in real events (intubation) were associated with improved observed performance (P = .012) 
Nishisaki et al, 201023  Children’s hospital, tertiary PICU Cohort (265) 202 high-fidelity simulations from June 2007 to August 2008: 30-min hands-on, simulation, and debriefing 401 orotracheal intubations were evaluated: 220 preintervention and 181 during intervention Same hospital Nontrained residents Period before intervention Pediatric residents, EM residents, nurses, respiratory therapists Clinical performance and team behavior during airway management, including tracheal intubation No differences in first-attempt success or overall success compared with nontrained residents or presimulation period 
Diaz and Dawson, 202036  Children’s hospital Time series (112) 70 simulations in 13 wkFocus was on closed-loop communication after watching 3 videos of simulated resuscitation scenariosStaff participated in 2 scenarios 1 mo apart Same hospital Period 4 mo before and after simulation Retrospective chart reviews Pediatric attending physicians, fellows, assistants, nurses, therapists (n = 3–6) Closed-loop communication Retrospective chart review of Emergency Severity Index patient category 1 Significant reduction in medication errors (P = .017) after intervention Staff perception of closed-loop communication ability improved and were sustained after 1 mo 
Hazwani et al, 202034  Children’s hospital Time series (NA) Mock code simulation program Random time end place 2–3 times per month Same hospital Period before mock code simulation program (June 2015–June 2016) Pediatric cardiac arrest team (n = 6) Cardiopulmonary arrest Review of all records of pediatric cardiopulmonary arrest occurring in study periods Significant reduction in CPR initiation time (P = .031) Improvement of compliance with guidelines: initiation of basic life support (P = .019) and electrical therapy for shockable rhythm (P = 00007)No significant change in survival rate or mortality 
Colman et al, 201937  Pediatric critical care unit (PICU), children’s hospital Time series (165) Simulation-based team training where team performance was the primary learning objective3-mo intervention period 30 mandatory team training workshops, each 3 h and including 3 scenarios Same hospital Data collected on real-time events in the pre- (9 mo) and post intervention (9 mo) period Multidisciplinary teams (n = 5–6) Team leader identification, role assignment, direct and closed-loop communication, situational awareness, shared mental model and global assessment Assessment of 15 teamwork categories immediately following a real medical crisis in the postintervention period Significant improvement in 11 of 15 composite teamwork skills (P < .05) 
Sawyer et al, 201925  Children’s hospital in 3 ICUs (PICU, CICU, NICU) Time series (332) 29 simulations performed between February 2014 and October 2016 Each simulation 2-h high-fidelity in situ training with 30-min introduction, 30 min of simulation, and 60 min of postevent debriefing Same hospital Period before simulation (December 2012–January 2014) Multidisciplinary, ICU/ECPR providers (n = 11 ± 3) ECMO, ECPR Retrospective chart view of hospital code blue records Clinical parameters: 1, compliance with ECPR activation protocol (pre, 83%; post, 95%; P = .02); 2, ECPR activation time (pre, 7 min; post, 2 min; P < .01); 3, ECPR response time, deployment time, and cannulation time did not change significantly Clinical outcomes: 1, ECLS 24-h survival (pre, 45%; post, 56%; P = .56); 2, survival to discharge (pre, 27%; post, 38%; P = .38) 
Barni et al, 201930  Pediatric emergency department Time series (30) 1 intervention hospital with simulation training(2011–2016)4-h high-fidelity simulation Same hospital Period before simulation (2004–2010) Physicians, nurses (n = 6) Anaphylaxis, use of epinephrine according to guideline Increase in use of epinephrine from 2.4% to 10% of patients (P < .05) Increase in number of patients with allergy workup (36%–51%; P = .011) 
Di Nardo et al, 201826  Children’s hospital, PICU Time series (NA) High-fidelity simulation program (December 2013–December 2016) Training every 3 mo Same hospitalPeriod before simulation (2011–November 2013) PICU personnel: nurses, physicians, perfusionists ECMO, time to manage most common ECMO emergencies Median time to oxygenator change (pre, 5.30 min; post, 3.90 min; P = .02)Median time to manage air embolism (pre, 22 min; post, 15 min; P = .048)ECMO survival rate (pre, 50%; post, 57%; P = .72) 
Su et al, 201427  Children’s hospital Time series (NA) 8 high-fidelity simulations over 36 mo (April 2010–March 2013) Same hospital Period before simulation (February 2009–March 2010) Multidisciplinary, ICU/ECPR providers ECMO, ECPR Retrospective chart view ECMO deployment time, ie, time from the call to the start of ECMO flow (pre, 51 min; post, 40 min; P = .018) 
Theilen et al, 201331  Children’s hospital Time series (NA) 2-h training weekly Each member 4–10 times/year Concurrent with introduction of pMET Same hospital Period before pMET and team training Physicians, nurses (pMET members) Deteriorating patients, pediatric trauma, postoperative complications Time from warning sign to first response (pre, 4 h; post, 1.5 h; P < .001) Increased nursing observation (P = .03) and consultant review (P = .004)Time to PICU admission (pre, 10.5 h; post, 5 h; P = .024) Overall hospital mortality rate reduced in the same period (P < .001) 
Andreatta et al, 201133  Children’s hospital Time series (228) Randomly called mock codes at least monthly with video recording for immediate debriefing (2005–2009) Same hospital Change over time Junior and senior residents Cardiopulmonary arrest due to sepsis, respiratory distress, increased intracranial pressure, anaphylactic shock, cardiogenic shock Survival rates (who survived cardiopulmonary arrest and were subsequently discharged) increased from 33% to ∼50% in same period of 4 y 
Martin et al, 201632  Children’s hospital Pre/post (83) 2-h cues of deterioration simulation program (aim = evaluating nurses’ early recognition of deterioration) (all)1-h multidisciplinary mock code (optional educational offer) over 1 y None Nurses Deteriorating patients,respiratory distress, cardiac arrest, seizures, supraventricular tachycardia No significant change in number of PICU transfers by the number of mock simulations No significant change in accuracy of PEWS score by the number of mock simulations 
Burton et al, 201128  Children’s hospital Pre/post (19) 24 sessions of high-fidelity simulations4-h duration (May 2008–August 2009) None ECMO nurses and respiratory therapists ECMO Based on 26 cannulations during study period1: time from blood available to circuit ready did not improve2: initiation checklist (24 items) compliance improved from 17.14 at pretraining baseline to 23.23 (SD, 1.61) during training (P < .0001) 
Source, YearSettingDesign(No. of Participants)InterventionComparatorParticipantsCase KeywordsOutcomes
Qian et al, 201629  3 tertiary hospitals Cohort (NA) High-fidelity simulation every 3 mo at 1 intervention hospital Evaluation after 1 y 2 control hospital straining as usual Chief resident, residents, nurses (n = 5) Septic shock, first-hour basic tasks Total compliance with first-hour critical tasks: intervention hospital, 61.7%; control hospitals, 23.1% (P = .004) 
Shah et al, 201622  Prehospital Cohort (250) Transports of patients having seizures Intervention training: 9-h course, didactic, hands-on, and 4 pediatric simulations Training as usual Paramedics, 250 transports with at least 1 paramedic who attended the course Seizure The course enhanced seizure protocol compliance prehospital, 250 transports of patients having seizuresCheck blood glucose: intervention, 72%; control, 66% (P = .35)Midazolam if euglycemic: intervention, 63%; control, 56% (P = .28) 
Nishisaki et al, 201135  Children’s hospital, tertiary PICU Cohort (265) 202 high-fidelity simulations from June 2007 to August 2008: 30-min hands-on, simulation, and debriefingActual intubations (n = 15) were observed in daytime Same hospital Nontrained residents Pediatric residents, EM residents, nurses, respiratory therapists Clinical performance and team behavior during airway management, including tracheal intubation The presence of ≥2 simulation-trained providers in real events (intubation) were associated with improved observed performance (P = .012) 
Nishisaki et al, 201023  Children’s hospital, tertiary PICU Cohort (265) 202 high-fidelity simulations from June 2007 to August 2008: 30-min hands-on, simulation, and debriefing 401 orotracheal intubations were evaluated: 220 preintervention and 181 during intervention Same hospital Nontrained residents Period before intervention Pediatric residents, EM residents, nurses, respiratory therapists Clinical performance and team behavior during airway management, including tracheal intubation No differences in first-attempt success or overall success compared with nontrained residents or presimulation period 
Diaz and Dawson, 202036  Children’s hospital Time series (112) 70 simulations in 13 wkFocus was on closed-loop communication after watching 3 videos of simulated resuscitation scenariosStaff participated in 2 scenarios 1 mo apart Same hospital Period 4 mo before and after simulation Retrospective chart reviews Pediatric attending physicians, fellows, assistants, nurses, therapists (n = 3–6) Closed-loop communication Retrospective chart review of Emergency Severity Index patient category 1 Significant reduction in medication errors (P = .017) after intervention Staff perception of closed-loop communication ability improved and were sustained after 1 mo 
Hazwani et al, 202034  Children’s hospital Time series (NA) Mock code simulation program Random time end place 2–3 times per month Same hospital Period before mock code simulation program (June 2015–June 2016) Pediatric cardiac arrest team (n = 6) Cardiopulmonary arrest Review of all records of pediatric cardiopulmonary arrest occurring in study periods Significant reduction in CPR initiation time (P = .031) Improvement of compliance with guidelines: initiation of basic life support (P = .019) and electrical therapy for shockable rhythm (P = 00007)No significant change in survival rate or mortality 
Colman et al, 201937  Pediatric critical care unit (PICU), children’s hospital Time series (165) Simulation-based team training where team performance was the primary learning objective3-mo intervention period 30 mandatory team training workshops, each 3 h and including 3 scenarios Same hospital Data collected on real-time events in the pre- (9 mo) and post intervention (9 mo) period Multidisciplinary teams (n = 5–6) Team leader identification, role assignment, direct and closed-loop communication, situational awareness, shared mental model and global assessment Assessment of 15 teamwork categories immediately following a real medical crisis in the postintervention period Significant improvement in 11 of 15 composite teamwork skills (P < .05) 
Sawyer et al, 201925  Children’s hospital in 3 ICUs (PICU, CICU, NICU) Time series (332) 29 simulations performed between February 2014 and October 2016 Each simulation 2-h high-fidelity in situ training with 30-min introduction, 30 min of simulation, and 60 min of postevent debriefing Same hospital Period before simulation (December 2012–January 2014) Multidisciplinary, ICU/ECPR providers (n = 11 ± 3) ECMO, ECPR Retrospective chart view of hospital code blue records Clinical parameters: 1, compliance with ECPR activation protocol (pre, 83%; post, 95%; P = .02); 2, ECPR activation time (pre, 7 min; post, 2 min; P < .01); 3, ECPR response time, deployment time, and cannulation time did not change significantly Clinical outcomes: 1, ECLS 24-h survival (pre, 45%; post, 56%; P = .56); 2, survival to discharge (pre, 27%; post, 38%; P = .38) 
Barni et al, 201930  Pediatric emergency department Time series (30) 1 intervention hospital with simulation training(2011–2016)4-h high-fidelity simulation Same hospital Period before simulation (2004–2010) Physicians, nurses (n = 6) Anaphylaxis, use of epinephrine according to guideline Increase in use of epinephrine from 2.4% to 10% of patients (P < .05) Increase in number of patients with allergy workup (36%–51%; P = .011) 
Di Nardo et al, 201826  Children’s hospital, PICU Time series (NA) High-fidelity simulation program (December 2013–December 2016) Training every 3 mo Same hospitalPeriod before simulation (2011–November 2013) PICU personnel: nurses, physicians, perfusionists ECMO, time to manage most common ECMO emergencies Median time to oxygenator change (pre, 5.30 min; post, 3.90 min; P = .02)Median time to manage air embolism (pre, 22 min; post, 15 min; P = .048)ECMO survival rate (pre, 50%; post, 57%; P = .72) 
Su et al, 201427  Children’s hospital Time series (NA) 8 high-fidelity simulations over 36 mo (April 2010–March 2013) Same hospital Period before simulation (February 2009–March 2010) Multidisciplinary, ICU/ECPR providers ECMO, ECPR Retrospective chart view ECMO deployment time, ie, time from the call to the start of ECMO flow (pre, 51 min; post, 40 min; P = .018) 
Theilen et al, 201331  Children’s hospital Time series (NA) 2-h training weekly Each member 4–10 times/year Concurrent with introduction of pMET Same hospital Period before pMET and team training Physicians, nurses (pMET members) Deteriorating patients, pediatric trauma, postoperative complications Time from warning sign to first response (pre, 4 h; post, 1.5 h; P < .001) Increased nursing observation (P = .03) and consultant review (P = .004)Time to PICU admission (pre, 10.5 h; post, 5 h; P = .024) Overall hospital mortality rate reduced in the same period (P < .001) 
Andreatta et al, 201133  Children’s hospital Time series (228) Randomly called mock codes at least monthly with video recording for immediate debriefing (2005–2009) Same hospital Change over time Junior and senior residents Cardiopulmonary arrest due to sepsis, respiratory distress, increased intracranial pressure, anaphylactic shock, cardiogenic shock Survival rates (who survived cardiopulmonary arrest and were subsequently discharged) increased from 33% to ∼50% in same period of 4 y 
Martin et al, 201632  Children’s hospital Pre/post (83) 2-h cues of deterioration simulation program (aim = evaluating nurses’ early recognition of deterioration) (all)1-h multidisciplinary mock code (optional educational offer) over 1 y None Nurses Deteriorating patients,respiratory distress, cardiac arrest, seizures, supraventricular tachycardia No significant change in number of PICU transfers by the number of mock simulations No significant change in accuracy of PEWS score by the number of mock simulations 
Burton et al, 201128  Children’s hospital Pre/post (19) 24 sessions of high-fidelity simulations4-h duration (May 2008–August 2009) None ECMO nurses and respiratory therapists ECMO Based on 26 cannulations during study period1: time from blood available to circuit ready did not improve2: initiation checklist (24 items) compliance improved from 17.14 at pretraining baseline to 23.23 (SD, 1.61) during training (P < .0001) 

CICU, cardiac ICU; ECLS, extracorporeal life support; ECPR, extracorporeal cardiopulmonary resuscitation; EM, emergency medicine; NA, not available (information not provided in the original article); pMET, pediatric medical emergency team; PEWS, pediatric early warning sign; pre/post, pre- and postintervention study.

Survival and Mortality Outcome

Four studies reported survival or mortality (Table 4). Sawyer et al25  performed simulation-based team training in 3 ICUs in a children’s hospital and compared survival after extracorporeal cardiopulmonary resuscitation (CPR) before and during the simulation program. The authors found a 24-hour survival rate of 56% in the period of simulation compared with 45% in the period before simulation. They also found a survival to discharge rate of 38% compared with 27% before simulation. Di Nardo et al26  compared survival before and after an ECMO simulation program at a children’s hospital. They reported a 50% survival rate before intervention compared with 57% after intervention. Theilen et al31  explored an intervention with 2 hours of weekly simulation and introduction of a pediatric medical emergency team where each team member had simulation training 4 to 10 times per year. They compared the period before and after introduction of the team and simulation training. They found that overall mortality was significantly reduced. Finally, Andreatta et al33  performed unannounced simulation mock codes at a minimum of once every month. They reported cardiopulmonary arrest survival rates over a 4-year period, including at the introduction of the simulation program. Survival rates increased from 33% before to ∼50% after simulation was introduced, and the reduction correlated with the number of mock codes.

Time to Critical Task Outcomes

Six studies reported time to a specific clinical event as outcome (Table 4). Four of these studies investigated time to manage ECMO emergencies (eg, pump failure, circuit failure) and time to initiate ECMO (eg, activation, cannulation, ECMO call to flow initiation). Authors of 3 of the 4 studies reported clinically and statistically significant reductions in time to important ECMO tasks after initiating simulation-based team training in their own hospital compared with before.2527  However, 1 study investigated time from blood available to circuit ready and did not find a significant change.28 

Theilen et al31  introduced a pediatric medical emergency team at their hospital where each team member had simulation training 4 to 10 times per year. Compared with a period before introduction of the team and simulation training, they observed a reduction from 4 hours to 1.5 hours from warning sign to first response and a reduction from 10.5 hours to 5 hours in time to pediatric ICU admission. Hazwani et al34  initiated mock code simulations and reported reduced CPR initiation time compared with the period before in the same hospital.

Protocol Adherence Outcomes

Seven studies reported adherence to guidelines or compliance to checklists or tasks (Table 4). Qian et al29  conducted a cohort study that initiated simulation-based training at 1 hospital, using 2 other hospitals as control. They investigated compliance with first-hour critical tasks in septic shock and reported a significantly higher compliance rate of 62% at the intervention hospital compared with 23% at the control hospitals. Barni et al30  investigated the use of epinephrine for anaphylaxis according to guidelines in a pediatric emergency department. Compared with before the simulation intervention, they observed a significant increase in the use of epinephrine from 2.4% to 10%. Shah et al22  found that a simulation-based course slightly improved adherence to prehospital seizure protocol, ie, blood glucose measurement and midazolam administration. Authors of 2 studies investigating ECMO reported improvement in compliance with an activation protocol25  and an initiation checklist28  after starting simulation-based training. Hazwani et al34  found improved compliance with guidelines in the treatment of cardiopulmonary arrest after a mock code simulation program, and Diaz and Dawson36  found a reduction in medication errors after simulation that focused on closed-loop communication.

Airway Management Outcomes

Two studies reported airway management outcomes (Table 4). Nishisaki et al23  investigated the effect of simulation-based intubation training and skill refresher training in the beginning of an on-call period. They found no difference in first-attempt and overall success of intubation between residents receiving refresher training or not on the basis of real pediatric intubations. Nishisaki et al35  later investigated effects of simulation training on clinical performance and team behavior during intubations as evaluated by a trained observer in a PICU. The team performance was significantly higher in teams with ≥2 simulation-trained members (both technical and behavioral scores) compared with teams with <2 trained members.

Team Performance and Communication Outcomes

Diaz and Dawson36  evaluated closed-loop communication and found a significant reduction in medication errors. Colman et al37  evaluated composite teamwork skills and found significant improvement in 11 of 15 skills after a period of simulation training.

We included 37 studies with observed team performance outcomes. Twenty-nine of these had no control group and are not further described here (Table 1). Below, we describe the 8 studies with a control group, 7 of which used random allocation (Table 5).

TABLE 5

Summary of Eight Controlled Studies With Team Performance Outcomes (Kirkpatrick Level 3)

Source, YearSettingDesign(No. of Participants)InterventionComparatorParticipantsRetestTimingCaseKeywordsOutcomes
Bragard et al, 201913  Pediatric department, university hospital Randomized(16) 5 high-fidelity simulation sessions with debriefingVideo recorded at sessions 1 and 5 for comparison No training program, 2 simulations sessions without debriefing, video recorded for evaluation Residents (pediatric and emergency), nurses — 5 scenarios on arrhythmia due to hypovolemia, drug intoxication, hyperkalemia, drowning, and hypoxia Blinded evaluation of video recordingsEffect on leadership and communication skills (P = .038)Effect on knowledge and clinical skills (P = .026)No significant change in time to initiate CPR 
Mariani et al, 201915  5-hospital health system Randomized(18) 2 high-fidelity simulation sessions of 1 h, 5 mo apart Clinical duties as usual Pediatric nurses 9 mo Hypovolemic shock, respiratory distress, seizure No difference in skill competency during scenarios comparing intervention and control groups 
Stellflug and Lowe, 201821  Regional PALS course Randomized(94) Integration of high-fidelity simulation into PALS course (2 d) PALS (2 d) + low-fidelity simulation Nurses, physicians, respiratory therapists, paramedics 6 mo(written test) Cases from PALS course Better retention after high fidelity vs low fidelity (P = .05)Time to critical events (immediate postcourse, intervention vs control):Recognition (100 s vs 112 s)Intervention (141 s vs 159 s)Reassessment (154 s vs 187s) 
Lemke et al, 201920  Hospital’s simulationcenter Randomized(30) Rapid deliberate practicePre- and posttest were video recorded and scored for comparison Traditional debriefing PEM fellows, nurses, respiratory therapists 3 mo Pediatric resuscitation cases Improvement in the human factor subscore (eg, establishing roles in the team, communication, knowledge sharing) for the intervention group (P = .013)A trend toward improvement in airway/breathing and circulation scores, but not significant 
Bultas et al, 201418  Pediatric hospital Randomized(33) High-fidelity simulation (comparing teaching strategy for recognizing deteriorating pediatric patient) Simulation with traditional static mannequins Pediatric staff 6 mo Respiratory failure, cardiogenic shock Significant change in teamwork confidence score (P = .001)Significant change in skills and tasks performed during respiratory scenario (P = .012) 
Kurosawa et al, 201414  Children’s hospital, tertiary PICU Randomized(40) Reconstructed PALS with 6 in situ scenarios of 30 min in 6 mo for each participant in the intervention group Standard PALS training Nurses, respiratory therapists 6 mo 12 core PALS scenarios Skill performance (by clinical performance tool) significantly improved (P = .004), and behavioral performance (improvement in teamwork) significantly improved in each group after training, but no significant difference between intervention and control groups 
Sudikoff et al, 200916  University hospital Randomized(16) High-fidelity simulation trainingIntervention group: 1-d training, 25% didactic, 15% hands on, 60% simulationCross-over study Standard resident curriculum Pediatric residents, postgraduate year 2 2 and 4 mo Respiratory failure and airway management Mean global competency score improved, and improvement persisted for up to 4 mo (P < .05)Critical actions showed a trend toward but did not reach statistical significanceReduction in number of harmful actions 
Gilfoyle et al, 200724  Children’s hospital Cohort(15) Plenary session + 2 simulated scenariosTeam performance evaluated by checklist Nontrained residents Pediatric residents 6 mo Resuscitation team leadership Significant increase in checklist scores (leadership, communication, etc) between the 2 scenarios on day 1 (P < .05)Significant increase in checklist scores after 6 mo in the intervention group (P < .001) vs nontrained residents (same scenario as from day 1) 
Source, YearSettingDesign(No. of Participants)InterventionComparatorParticipantsRetestTimingCaseKeywordsOutcomes
Bragard et al, 201913  Pediatric department, university hospital Randomized(16) 5 high-fidelity simulation sessions with debriefingVideo recorded at sessions 1 and 5 for comparison No training program, 2 simulations sessions without debriefing, video recorded for evaluation Residents (pediatric and emergency), nurses — 5 scenarios on arrhythmia due to hypovolemia, drug intoxication, hyperkalemia, drowning, and hypoxia Blinded evaluation of video recordingsEffect on leadership and communication skills (P = .038)Effect on knowledge and clinical skills (P = .026)No significant change in time to initiate CPR 
Mariani et al, 201915  5-hospital health system Randomized(18) 2 high-fidelity simulation sessions of 1 h, 5 mo apart Clinical duties as usual Pediatric nurses 9 mo Hypovolemic shock, respiratory distress, seizure No difference in skill competency during scenarios comparing intervention and control groups 
Stellflug and Lowe, 201821  Regional PALS course Randomized(94) Integration of high-fidelity simulation into PALS course (2 d) PALS (2 d) + low-fidelity simulation Nurses, physicians, respiratory therapists, paramedics 6 mo(written test) Cases from PALS course Better retention after high fidelity vs low fidelity (P = .05)Time to critical events (immediate postcourse, intervention vs control):Recognition (100 s vs 112 s)Intervention (141 s vs 159 s)Reassessment (154 s vs 187s) 
Lemke et al, 201920  Hospital’s simulationcenter Randomized(30) Rapid deliberate practicePre- and posttest were video recorded and scored for comparison Traditional debriefing PEM fellows, nurses, respiratory therapists 3 mo Pediatric resuscitation cases Improvement in the human factor subscore (eg, establishing roles in the team, communication, knowledge sharing) for the intervention group (P = .013)A trend toward improvement in airway/breathing and circulation scores, but not significant 
Bultas et al, 201418  Pediatric hospital Randomized(33) High-fidelity simulation (comparing teaching strategy for recognizing deteriorating pediatric patient) Simulation with traditional static mannequins Pediatric staff 6 mo Respiratory failure, cardiogenic shock Significant change in teamwork confidence score (P = .001)Significant change in skills and tasks performed during respiratory scenario (P = .012) 
Kurosawa et al, 201414  Children’s hospital, tertiary PICU Randomized(40) Reconstructed PALS with 6 in situ scenarios of 30 min in 6 mo for each participant in the intervention group Standard PALS training Nurses, respiratory therapists 6 mo 12 core PALS scenarios Skill performance (by clinical performance tool) significantly improved (P = .004), and behavioral performance (improvement in teamwork) significantly improved in each group after training, but no significant difference between intervention and control groups 
Sudikoff et al, 200916  University hospital Randomized(16) High-fidelity simulation trainingIntervention group: 1-d training, 25% didactic, 15% hands on, 60% simulationCross-over study Standard resident curriculum Pediatric residents, postgraduate year 2 2 and 4 mo Respiratory failure and airway management Mean global competency score improved, and improvement persisted for up to 4 mo (P < .05)Critical actions showed a trend toward but did not reach statistical significanceReduction in number of harmful actions 
Gilfoyle et al, 200724  Children’s hospital Cohort(15) Plenary session + 2 simulated scenariosTeam performance evaluated by checklist Nontrained residents Pediatric residents 6 mo Resuscitation team leadership Significant increase in checklist scores (leadership, communication, etc) between the 2 scenarios on day 1 (P < .05)Significant increase in checklist scores after 6 mo in the intervention group (P < .001) vs nontrained residents (same scenario as from day 1) 

—, data not available; PALS, pediatric advanced life support; PEM, pediatric emergency medicine.

Retention of Skills

Seven of the 8 studies evaluated retention of technical and nontechnical skills using simulation-based retesting of teams. Authors of 6 of these 7 studies retested after 2 to 6 months and found significant improvements in intervention groups compared with control groups.14,16,18,20,21,24  Mariani et al15  performed a retest after 9 months and found no difference in skill competency between the groups. Stellflug and Lowe21  investigated random allocation to either high-fidelity or low-fidelity team training and reported better performance on predefined skills for the high-fidelity group when retested after 6 months. Lemke et al20  investigated random allocation to either rapid-cycle deliberate practice or traditional team training with debriefing and reported improvement in the human factor subscore (eg, team organization, communication, knowledge sharing) for the intervention group after 3 months. Bultas et al18  investigated a teaching strategy for recognizing a deteriorating pediatric patient, with random allocation to either high-fidelity simulation or traditional training, and reported significantly better teamwork and skills and tasks scores during a respiratory scenario in the intervention group at 6-month retest. Kurosawa et al14  performed a randomized trial of a reconstructed recertification of pediatric advanced life support with simulation once a month compared with standard recertification. They found significantly improved skills performance when retesting after 6 months compared with the control group. Sudikoff et al16  conducted a randomized cross-over study of emergency airway management and teamwork. They found improved global competency scores with regard to communication, leadership, and team skills, and the improvement persisted for up to 4 months. Gilfoyle et al24  conducted a workshop with simulation-based training and investigated team leadership skills in pediatric advanced resuscitation. They found a significant increase in checklist scores after 6 months in the intervention group compared with nontrained residents.

Team Performance Outcomes

Five of the 8 studies evaluated teamwork competencies such as leadership, communication, human factors, and teamwork confidence. In 4 of 5 studies, the intervention group had significantly better team performance scores than the control group.13,18,20,24 

We included 47 studies with self-reported knowledge or confidence. Thirty-eight of these had no control group and are not further described here (Table 1). Five of the studies had a control group with random allocation. Bragard et al13  performed a team training program on cardiac arrhythmias and found reduced perceived stress and improved satisfaction with communication, general skills, and communication skills compared with the control group. Stellflug and Lowe21  investigated random allocation to either high-fidelity or low-fidelity team training and found that the low-fidelity group declined more in ratings of their confidence levels on performing 19 different skills or tasks on retest at 6 months. Fagan et al19  investigated random allocation to recertification of pediatric advanced life support with or without simulation-based team training. Members of the intervention group reported higher perception of teamwork and more situational awareness. Mariani et al15  investigated the effect of repeated pediatric mock code simulations and found no difference in self-confidence between the groups. Besbes et al17  found that knowledge test scores improved significantly for both groups, although the improvement was significantly higher in the simulation group.

Four of the studies had a control group without random allocation. McLaughlin et al38  investigated the impact of team training on perceived confidence when handling pediatric trauma patients, and at 2 years follow-up, trained professionals reported less anxiety and greater confidence compared with professionals who did not receive the simulation training. Dowson et al39  investigated the impact of team training on pediatric nurses’ self-assessed clinical confidence and found a significant improvement. Auerbach et al40  compared repetitive pediatric simulation training (scenario-debriefing-scenario) to standard simulation training and found that the intervention group reported greater improvement in knowledge and skills. Happel et al41  evaluated the perceived impact on live critical events in a period with weekly simulation-based team training. The residents in the intervention group reported positive effects on their behaviors and confidence levels as well as feeling more prepared; however, these changes did not differ significantly from the control group.

Within each group of studies (according to Table 1), the funnel plot raised no major concern about publication bias (Fig 2). In the funnel plot, studies with and without control groups appeared similar across study size and effect ratio. We found no indication of selective reporting bias, as methods and results sections were coherent in all studies. Prepublished study protocols were unavailable on all randomized studies, which was reflected in the risk-of-bias score.

FIGURE 2

Funnel plot of 61 studies. Black circle, studies with a control group; white circle, studies with no control group and level 3 to 4 outcome; x, studies with no control group and level 2 outcome. Fourteen studies were excluded from the funnel plot because the number of participants was not specified.26,27,29,31,34,42,43,45,58,67,71,74,77,80  Four studies were excluded from the funnel plot because no effect estimate was specified.46,81,86,88 

FIGURE 2

Funnel plot of 61 studies. Black circle, studies with a control group; white circle, studies with no control group and level 3 to 4 outcome; x, studies with no control group and level 2 outcome. Fourteen studies were excluded from the funnel plot because the number of participants was not specified.26,27,29,31,34,42,43,45,58,67,71,74,77,80  Four studies were excluded from the funnel plot because no effect estimate was specified.46,81,86,88 

Close modal

We performed this systematic review to describe the current state of evidence in the evolving research field of simulation-based team training in pediatrics. Summarizing effects on patient outcomes (level 4), 2 important findings deserve emphasis.

First, 4 studies indicated improved survival after introducing simulation-based team training.25,26,31,33  However, given study design limitations, future studies with higher methodological standards would enable more robust conclusions. The authors of the 4 time series studies reported survival rates before and after introduction of a simulation program. The findings were prone to confounding from uncontrolled or unknown factors. For example, Theilen et al31  introduced a pediatric medical emergency team at the same time as the simulation program, which hampers conclusions related to effects of simulation-based training per se.

Second, authors of 4 studies designed with a control group (nonrandomized) indicated effects of simulation-based team training on clinical performance, eg, improved compliance to first-hour critical tasks in sepsis treatment,29  enhanced seizure protocol adherence,22  and improved intubation performance if ≥2 simulation-trained professionals were present.23,35  Overall, these controlled studies may indicate effects on treatment quality, but ambiguity and lack of precision preclude a strong conclusion. Authors of several studies with a time series design observed improved clinical performance after the introduction of simulation programs (Table 4). However, their conclusions were prone to bias from uncontrolled or unknown factors, eg, other changes during the period of intervention.

In considering team performance outcomes (Kirkpatrick level 3) and learning outcomes (Kirkpatrick level 2), 2 important findings should be noted. First, authors of several randomized studies found improved team performance when retesting 2 to 6 months after intervention.14,16,18,20,21,24  Studies with control groups revealed that team performance, eg, leadership, communication, human factors, and teamwork confidence, improved.13,18,20,24  Second, authors of most studies with random allocation showed improved perceived confidence, skill levels, and ability to communicate in emergency situations after the introduction of simulation-based team training. Overall, the findings from these controlled studies support improved team performance, knowledge, and confidence after simulation training.

This systematic review applied a comprehensive search strategy in 4 medical databases by an experienced medical librarian. We followed a prespecified study protocol registered in the Prospective Register of Systematic Reviews repository. Two reviewers independently screened and selected studies and performed data extraction and risk-of-bias scoring. Data were presented according to the PRISMA guidelines. The funnel plot in Fig 2 indicates no publication bias.

Meta-analysis was impossible because of heterogeneous study designs, interventions, and outcomes in the included studies. Instead, we performed a narrative synthesis structured by hierarchical outcome, including patient and clinical performance outcomes, team performance outcomes, and learning outcomes. Although this process is less standardized than a meta-analysis, we consider it reproducible and open to scrutiny. Most included studies had significant methodological limitations. Sixty-one of 79 studies did not include a control group, which is concerning in studies using self-reported or simulation-based end points. Most nonrandomized studies had inadequate adjustment for potential confounding factors, such as years of clinical experience, profession, team composition, age, and sex. Small sample size (<50 in 33 studies and not described in 16 studies) may likely have resulted in inadequate power and an inability to perform properly adjusted statistical analyses.

Measuring effects of simulation-based training on clinical outcomes should be preferred whenever possible. However, studying rare clinical end points, such as mortality or severe morbidity, requires large, multicenter studies. Therefore, other clinical outcomes may be more feasible to obtain and may serve as surrogates related to treatment quality, eg, time to critical tasks or guideline adherence. We acknowledge that real-life evaluation of training effects may be difficult and time consuming given the paucity and volatile nature of critical clinical events. We advocate the use of a control group when designing new studies and random allocation to intervention and control whenever possible. Study protocols should be registered to avoid selective reporting. When applying simulation-based evaluation of training effects, we recommend the use of video recordings and blinded assessment using accepted and validated protocols. Compared with planned simulation-based retesting, the use of unannounced simulated mock codes may mimic real clinical encounters more closely because clinical professionals often prepare for planned testing. We recommend that the theoretical framework and process of debriefing is described, as these are important parts of simulation-based training that may affect the learning outcome.

To our knowledge, this systematic review of simulation-based pediatric team training is the first with an aim to collectively describe all current available literature on clinical, team performance, and learning outcomes. We included 79 studies published over the past 15 years and conclude that pediatric simulation-based team training improves clinical performance in time-critical tasks and increases adherence to guidelines. Simulation-based training improves team performance and technical skills for at least 2 to 6 months. Furthermore, it may improve survival rates, but these results are affected by heterogenous interventions and outcomes and studies without a control group. Future research should include longer-term measures of retention of team performance and other skills and patient outcomes or clinical measures of treatment quality whenever possible.

The authors acknowledge and appreciate the scientific contribution of the authors of all articles included in this review. The authors thank the editor and reviewers for valuable comments on an earlier version of this manuscript.

Dr Thim designed and planned the study, registered the study with Prospective Register of Systematic Reviews, performed abstract and full-text screening and data extraction, and drafted the initial manuscript; Dr Lindhard designed and planned the study, helped to specify the literature search, performed abstract and full-text screening, data extraction, and risk-of-bias scoring, and critically revised the manuscript; Mr Laursen specified and performed the systematic literature search, maintained data in the Covidence review platform, wrote detailed documentation for publication, and critically revised the manuscript; Mr Schram performed data extraction and risk-of-bias scoring and critically revised the manuscript; Dr Paltved initiated the study, provided intellectual support for all parts of it, and critically revised the manuscript; Dr Henriksen designed and planned the study, helped to resolve inclusion and risk-of-bias scoring conflicts, and critically revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: The study was supported by Corporate HR, MidtSim, Central Region Denmark. The funding body had no influence on the design of the study; data collection, analysis, and interpretation; manuscript drafting; or conclusions.

CPR

cardiopulmonary resuscitation

ECMO

extracorporeal membrane oxygenation

NOS

Newcastle Ottawa Quality Assessment Scale

PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analysis

RoB 2

Cochrane risk-of-bias tool for randomized trials 2.0

1
Clerihew
L
,
Rowney
D
,
Ker
J
.
Simulation in paediatric training
.
Arch Dis Child Educ Pract Ed
.
2016
;
101
(
1
):
8
14
2
Ojha
R
,
Liu
A
,
Rai
D
,
Nanan
R
.
Review of Simulation in pediatrics: the evolution of a revolution
.
Front Pediatr
.
2015
;
3
:
106
3
Hilliard
R
,
Bannister
SL
,
Amin
H
,
Baird
B
.
Paediatric medical education: challenges and new developments
.
Paediatr Child Health
.
2009
;
14
(
5
):
303
309
4
Gaba
DM
.
The future vision of simulation in health care
.
Qual Saf Health Care
.
2004
;
13
(
suppl 1
):
i2
i10
5
Lindhard
MS
,
Thim
S
,
Laursen
HS
,
Schram
AW
,
Paltved
C
,
Henriksen
TB
.
Simulation-based neonatal resuscitation team training: a systematic review
.
Pediatrics
.
2021
;
147
(
4
):
e2020042010
6
Moher
D
,
Liberati
A
,
Tetzlaff
J
,
Altman
DG
;
PRISMA Group
.
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement
.
PLoS Med
.
2009
;
6
(
7
):
e1000097
7
Kirkpatrick
DL
,
Kirkpatrick
JD
.
Evaluating Training Programs: The Four Levels
. 3rd ed.
Oakland, CA
:
Berrett-Koehler Publishers
;
2006
8
Ross
J
,
Rebella
G
,
Westergaard
M
,
Damewood
S
,
Hess
J
.
Simulation training to maintain neonatal resuscitation and pediatric sedation skills for emergency medicine faculty
.
WMJ
.
2016
;
115
(
4
):
180
184
9
Bragard
I
,
Seghaye
M-C
,
Farhat
N
, et al
.
Implementation of a 2-day simulation-based course to prepare medical graduates on their first year of residency
.
Pediatr Emerg Care
.
2018
;
34
(
12
):
857
861
10
Liberati
A
,
Altman
DG
,
Tetzlaff
J
, et al
.
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration
.
PLoS Med
.
2009
;
6
(
7
):
e1000100
11
Ottawa Hospital Research Institute
.
The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses
.
Available at: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed August 28, 2021
12
Cook
DA
,
Reed
DA
.
Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale-Education
.
Acad Med
.
2015
;
90
(
8
):
1067
1076
13
Bragard
I
,
Farhat
N
,
Seghaye
M-C
, et al
.
Effectiveness of a high-fidelity simulation-based training program in managing cardiac arrhythmias in children: a randomized pilot study
.
Pediatr Emerg Care
.
2019
;
35
(
6
):
412
418
14
Kurosawa
H
,
Ikeyama
T
,
Achuff
P
, et al
.
A randomized, controlled trial of in situ pediatric advanced life support recertification (“pediatric advanced life support reconstructed”) compared with standard pediatric advanced life support recertification for ICU frontline providers*
.
Crit Care Med
.
2014
;
42
(
3
):
610
618
15
Mariani
B
,
Zazyczny
KA
,
Decina
P
, et al
.
Simulation for clinical preparedness in pediatric emergencies: a pilot study
.
J Nurses Prof Dev
.
2019
;
35
(
1
):
6
11
16
Sudikoff
SN
,
Overly
FL
,
Shapiro
MJ
.
High-fidelity medical simulation as a technique to improve pediatric residents’ emergency airway management and teamwork: a pilot study
.
Pediatr Emerg Care
.
2009
;
25
(
10
):
651
656
17
Besbes
H
,
Ouanes
I
,
Thabet
F
,
Sfar
E
,
Chouchane
C
,
Chouchane
S
.
High-fidelity simulation versus video-based learning in the management of pediatric septic shock: a pilot study
.
Eur J Pediatr
.
2021
;
180
(
2
):
487
493
18
Bultas
MW
,
Hassler
M
,
Ercole
PM
,
Rea
G
.
Effectiveness of high-fidelity simulation for pediatric staff nurse education
.
Pediatr Nurs
.
2014
;
40
(
1
):
27
32
,
42
19
Fagan
MJ
,
Connelly
CD
,
Williams
BS
,
Fisher
ES
.
Integrating team training in the pediatric life support program: an effective and efficient approach?
J Nurs Adm
.
2018
;
48
(
5
):
279
284
20
Lemke
DS
,
Fielder
EK
,
Hsu
DC
,
Doughty
CB
.
Improved team performance during pediatric resuscitations after rapid cycle deliberate practice compared with traditional debriefing: a pilot study
.
Pediatr Emerg Care
.
2019
;
35
(
7
):
480
486
21
Stellflug
SM
,
Lowe
NK
.
The effect of high fidelity simulators on knowledge retention and skill self efficacy in pediatric advanced life support courses in a rural state
.
J Pediatr Nurs
.
2018
;
39
:
21
26
22
Shah
MI
,
Carey
JM
,
Rapp
SE
, et al
.
Impact of high-fidelity pediatric simulation on paramedic seizure management
.
Prehosp Emerg Care
.
2016
;
20
(
4
):
499
507
23
Nishisaki
A
,
Donoghue
AJ
,
Colborn
S
, et al
.
Effect of just-in-time simulation training on tracheal intubation procedure safety in the pediatric intensive care unit
.
Anesthesiology
.
2010
;
113
(
1
):
214
223
24
Gilfoyle
E
,
Gottesman
R
,
Razack
S
.
Development of a leadership skills workshop in paediatric advanced resuscitation
.
Med Teach
.
2007
;
29
(
9
):
e276
e283
25
Sawyer
T
,
Burke
C
,
McMullan
DM
, et al
.
Impacts of a pediatric extracorporeal cardiopulmonary resuscitation (ECPR) simulation training program
.
Acad Pediatr
.
2019
;
19
(
5
):
566
571
26
Di Nardo
M
,
David
P
,
Stoppa
F
, et al
.
The introduction of a high-fidelity simulation program for training pediatric critical care personnel reduces the times to manage extracorporeal membrane oxygenation emergencies and improves teamwork
.
J Thorac Dis
.
2018
;
10
(
6
):
3409
3417
27
Su
L
,
Spaeder
MC
,
Jones
MB
, et al
.
Implementation of an extracorporeal cardiopulmonary resuscitation simulation program reduces extracorporeal cardiopulmonary resuscitation times in real patients
.
Pediatr Crit Care Med
.
2014
;
15
(
9
):
856
860
28
Burton
KS
,
Pendergrass
TL
,
Byczkowski
TL
, et al
.
Impact of simulation-based extracorporeal membrane oxygenation training in the simulation laboratory and clinical environment
.
Simul Healthc
.
2011
;
6
(
5
):
284
291
29
Qian
J
,
Wang
Y
,
Zhang
Y
,
Zhu
X
,
Rong
Q
,
Wei
H
.
A survey of the first-hour basic care tasks of severe sepsis and septic shock in pediatric patients and an evaluation of medical simulation on improving the compliance of the tasks
.
J Emerg Med
.
2016
;
50
(
2
):
239
245
30
Barni
S
,
Mori
F
,
Giovannini
M
,
de Luca
M
,
Novembre
E
.
In situ simulation in the management of anaphylaxis in a pediatric emergency department
.
Intern Emerg Med
.
2019
;
14
(
1
):
127
132
31
Theilen
U
,
Leonard
P
,
Jones
P
, et al
.
Regular in situ simulation training of paediatric medical emergency team improves hospital response to deteriorating patients
.
Resuscitation
.
2013
;
84
(
2
):
218
222
32
Martin
MG
,
Keller
LA
,
Long
TL
,
Ryan-Wenger
NA
.
High-fidelity simulation effect on nurses’ identification of deteriorating pediatric patients
.
Clin Simul Nurs
.
2016
;
12
(
6
):
228
239
33
Andreatta
P
,
Saxton
E
,
Thompson
M
,
Annich
G
.
Simulation-based mock codes significantly correlate with improved pediatric patient cardiopulmonary arrest survival rates
.
Pediatr Crit Care Med
.
2011
;
12
(
1
):
33
38
34
Hazwani
TR
,
Alosaimi
A
,
Almutairi
M
,
Shaheen
N
,
Al Hassan
Z
,
Antar
M
.
The impact of mock code simulation on the resuscitation practice and patient outcome for children with cardiopulmonary arrest
.
Cureus
.
2020
;
12
(
7
):
e9197
35
Nishisaki
A
,
Nguyen
J
,
Colborn
S
, et al
.
Evaluation of multidisciplinary simulation training on clinical performance and team behavior during tracheal intubation procedures in a pediatric intensive care unit
.
Pediatr Crit Care Med
.
2011
;
12
(
4
):
406
414
36
Diaz
MCG
,
Dawson
K
.
Impact of simulation-based closed-loop communication training on medical errors in a pediatric emergency department
.
Am J Med Qual
.
2020
;
35
(
6
):
474
478
37
Colman
N
,
Figueroa
J
,
McCracken
C
,
Hebbar
KB
.
Can simulation based-team training impact bedside teamwork in a pediatric intensive care unit?
J Pediatr Intensive Care
.
2019
;
8
(
4
):
195
203
38
McLaughlin
CM
,
Wieck
MM
,
Barin
EN
, et al
.
Impact of simulation-based training on perceived provider confidence in acute multidisciplinary pediatric trauma resuscitation
.
Pediatr Surg Int
.
2018
;
34
(
12
):
1353
1362
39
Dowson
A
,
Russ
S
,
Sevdalis
N
,
Cooper
M
,
De Munter
C
.
How in situ simulation affects paediatric nurses’ clinical confidence
.
Br J Nurs
.
2013
;
22
(
11
):
610
,
612
617
40
Auerbach
M
,
Kessler
D
,
Foltin
JC
.
Repetitive pediatric simulation resuscitation training
.
Pediatr Emerg Care
.
2011
;
27
(
1
):
29
31
41
Happel
CS
,
Lease
MA
,
Nishisaki
A
,
Braga
MS
.
Evaluating simulation education via electronic surveys immediately following live critical events: a pilot study
.
Hosp Pediatr
.
2015
;
5
(
2
):
96
100
42
Lee
MO
,
Schertzer
K
,
Khanna
K
,
Wang
NE
,
Camargo
CA
Jr
,
Sebok-Syer
SS
.
Using in situ simulations to improve pediatric patient safety in emergency departments
.
Acad Med
.
2021
;
96
(
3
):
395
398
43
Hazwani
TR
,
Harder
N
,
Shaheen
NA
, et al
.
Effect of a pediatric mock code simulation program on resuscitation skills and team performance
.
Clin Simul Nurs
.
2020
;
44
:
42
49
44
Colman
N
,
Figueroa
J
,
McCracken
C
,
Hebbar
K
.
Simulation-based team training improves team performance among pediatric intensive care unit staff
.
J Pediatr Intensive Care
.
2019
;
8
(
2
):
83
91
45
Yager
P
,
Collins
C
,
Blais
C
, et al
.
Quality improvement utilizing in-situ simulation for a dual-hospital pediatric code response team
.
Int J Pediatr Otorhinolaryngol
.
2016
;
88
:
42
46
46
Auerbach
M
,
Roney
L
,
Aysseh
A
, et al
.
In situ pediatric trauma simulation: assessing the impact and feasibility of an interdisciplinary pediatric in situ trauma care quality improvement simulation program
.
Pediatr Emerg Care
.
2014
;
30
(
12
):
884
891
47
van Schaik
SM
,
Plant
J
,
Diane
S
,
Tsang
L
,
O’Sullivan
P
.
Interprofessional team training in pediatric resuscitation: a low-cost, in situ simulation program that enhances self-efficacy among participants
.
Clin Pediatr (Phila)
.
2011
;
50
(
9
):
807
815
48
Falcone
RA
Jr
,
Daugherty
M
,
Schweer
L
,
Patterson
M
,
Brown
RL
,
Garcia
VF
.
Multidisciplinary pediatric trauma team training using high-fidelity trauma simulation
.
J Pediatr Surg
.
2008
;
43
(
6
):
1065
1071
49
Brown
KM
,
Mudd
SS
,
Perretta
JS
,
Dodson
A
,
Hunt
EA
,
McMillan
KN
.
Rapid cycle deliberate practice to facilitate “nano” in situ simulation: an interprofessional approach to just-in-time training
.
Crit Care Nurse
.
2021
;
41
(
1
):
e1
e8
50
Cory
MJ
,
Hebbar
KB
,
Colman
N
,
Pierson
A
,
Clarke
SA
.
Multidisciplinary simulation-based team training: knowledge acquisition and shifting perception
.
Clin Simul Nurs
.
2020
;
41
:
14
21
51
Karageorge
N
,
Muckler
VC
,
Toper
M
,
Hueckel
R
.
Using simulation with deliberate practice to improve pediatric ICU nurses’ knowledge, clinical teamwork, and confidence
.
J Pediatr Nurs
.
2020
;
54
:
58
62
52
Lutfi
R
,
Montgomery
EE
,
Berrens
ZJ
, et al
.
Improving adherence to a pediatric advanced life support supraventricular tachycardia algorithm in community emergency departments following in situ simulation
.
J Contin Educ Nurs
.
2019
;
50
(
9
):
404
410
53
Saqe-Rockoff
A
,
Ciardiello
AV
,
Schubert
FD
.
Low-fidelity, in-situ pediatric resuscitation simulation improves RN competence and self-efficacy
.
J Emerg Nurs
.
2019
;
45
(
5
):
538
544.e1
54
Bayouth
L
,
Ashley
S
,
Brady
J
, et al
.
An in-situ simulation-based educational outreach project for pediatric trauma care in a rural trauma system
.
J Pediatr Surg
.
2018
;
53
(
2
):
367
371
55
Brown
KM
,
Mudd
SS
,
Hunt
EA
, et al
.
A multi-institutional simulation boot camp for pediatric cardiac critical care nurse practitioners
.
Pediatr Crit Care Med
.
2018
;
19
(
6
):
564
571
56
Emani
SS
,
Allan
CK
,
Forster
T
, et al
.
Simulation training improves team dynamics and performance in a low-resource cardiac intensive care unit
.
Ann Pediatr Cardiol
.
2018
;
11
(
2
):
130
136
57
Ryan
A
,
Rizwan
R
,
Williams
B
,
Benscoter
A
,
Cooper
DS
,
Iliopoulos
I
.
Simulation training improves resuscitation team leadership skills of nurse practitioners
.
J Pediatr Health Care
.
2019
;
33
(
3
):
280
287
58
Katznelson
JH
,
Wang
J
,
Stevens
MW
,
Mills
WA
.
Improving pediatric preparedness in critical access hospital emergency departments: impact of a longitudinal in situ simulation program
.
Pediatr Emerg Care
.
2018
;
34
(
1
):
17
20
59
Couloures
KG
,
Allen
C
.
Use of simulation to improve cardiopulmonary resuscitation performance and code team communication for pediatric residents
.
MedEdPORTAL
.
2017
;
13
:
10555
60
Gilfoyle
E
,
Koot
DA
,
Annear
JC
, et al;
Teams4Kids Investigators and the Canadian Critical Care Trials Group
.
Improved clinical performance and teamwork of pediatric interprofessional resuscitation teams with a simulation-based educational intervention
.
Pediatr Crit Care Med
.
2017
;
18
(
2
):
e62
e69
61
Stone
K
,
Reid
J
,
Caglar
D
, et al
.
Increasing pediatric resident simulated resuscitation performance: a standardized simulation-based curriculum
.
Resuscitation
.
2014
;
85
(
8
):
1099
1105
62
Chan
S-Y
,
Figueroa
M
,
Spentzas
T
,
Powell
A
,
Holloway
R
,
Shah
S
.
Prospective assessment of novice learners in a simulation-based extracorporeal membrane oxygenation (ECMO) education program
.
Pediatr Cardiol
.
2013
;
34
(
3
):
543
552
63
Kennedy
JL
,
Jones
SM
,
Porter
N
, et al
.
High-fidelity hybrid simulation of allergic emergencies demonstrates improved preparedness for office emergencies in pediatric allergy clinics
.
J Allergy Clin Immunol Pract
.
2013
;
1
(
6
):
608
617.e1–14
64
Patterson
MD
,
Geis
GL
,
LeMaster
T
,
Wears
RL
.
Impact of multidisciplinary simulation-based training on patient safety in a paediatric emergency department
.
BMJ Qual Saf
.
2013
;
22
(
5
):
383
393
65
Patterson
MD
,
Geis
GL
,
Falcone
RA
,
LeMaster
T
,
Wears
RL
.
In situ simulation: detection of safety threats and teamwork training in a high risk emergency department
.
BMJ Qual Saf
.
2013
;
22
(
6
):
468
477
66
Tofil
NM
,
Benner
KW
,
Zinkan
L
,
Alten
J
,
Varisco
BM
,
White
ML
.
Pediatric intensive care simulation course: a new paradigm in teaching
.
J Grad Med Educ
.
2011
;
3
(
1
):
81
87
67
Hunt
EA
,
Heine
M
,
Hohenhaus
SM
,
Luo
X
,
Frush
KS
.
Simulated pediatric trauma team management: assessment of an educational intervention
.
Pediatr Emerg Care
.
2007
;
23
(
11
):
796
804
68
Tsai
T-C
,
Harasym
PH
,
Nijssen-Jordan
C
,
Jennett
P
.
Learning gains derived from a high-fidelity mannequin-based simulation in the pediatric emergency department
.
J Formos Med Assoc
.
2006
;
105
(
1
):
94
98
69
Peterson
E
,
Porter
M
,
Calhoun
A
.
Mixed-reality simulation for a pediatric transport team: a pilot study
.
Air Med J
.
2020
;
39
(
3
):
173
177
70
Monachino
A
,
Caraher
C
,
Ginsberg
J
,
Bailey
C
,
White
E
.
Medical emergencies in the primary care setting: an evidence based practice approach using simulation to improve readiness
.
J Pediatr Nurs
.
2019
;
49
:
72
78
71
Abulebda
K
,
Lutfi
R
,
Whitfill
T
, et al
.
A collaborative in situ simulation-based pediatric readiness improvement program for community emergency departments
.
Acad Emerg Med
.
2018
;
25
(
2
):
177
185
72
Cristallo
T
,
Walters
M
,
Scanlan
J
,
Doten
I
,
Demeter
T
,
Colvin
D
.
Multidisciplinary, In Situ simulation improves experienced caregiver confidence with high-risk pediatric emergencies
.
Pediatr Emerg Care
.
2021
;
37
(
9
):
451
455
73
Lind
MM
,
Corridore
M
,
Sheehan
C
,
Moore-Clingenpeel
M
,
Maa
T
.
A multidisciplinary approach to a pediatric difficult airway simulation course
.
Otolaryngol Head Neck Surg
.
2018
;
159
(
1
):
127
135
74
Kalidindi
S
,
Kirk
M
,
Griffith
E
.
In-situ simulation enhances emergency preparedness in pediatric care practices
.
Cureus
.
2018
;
10
(
10
):
e3389
75
Raffaeli
G
,
Ghirardello
S
,
Vanzati
M
, et al
.
Start a neonatal extracorporeal membrane oxygenation program: a multistep team training
.
Front Pediatr
.
2018
;
6
:
151
76
Saavedra
HR
,
Turner
JS
,
Cooper
DD
.
Use of simulation to improve the comfort of pediatric residents managing critically ill emergency department patients
.
Pediatr Emerg Care
.
2018
;
34
(
9
):
633
635
77
Whitfill
T
,
Gawel
M
,
Auerbach
M
.
A simulation-based quality improvement initiative improves pediatric readiness in community hospitals
.
Pediatr Emerg Care
.
2018
;
34
(
6
):
431
435
78
Lehner
M
,
Heimberg
E
,
Hoffmann
F
,
Heinzel
O
,
Kirschner
H-J
,
Heinrich
M
.
Evaluation of a pilot project to introduce simulation-based team training to pediatric surgery trauma room care
.
Int J Pediatr
.
2017
;
2017
:
9732316
79
Wallace
DM
,
Burnley
J
,
Langston
B
,
Russell
M
,
White
K
,
Stroud
MH
.
Education coupled with in-situ high fidelity simulation improves medical-surgical RN code blue comfort levels
.
J Ark Med Soc
.
2017
;
113
(
9
):
222
224
80
Katznelson
JH
,
Mills
WA
,
Forsythe
CS
,
Shaikh
S
,
Tolleson-Rinehart
S
.
Project CAPE: a high-fidelity, in situ simulation program to increase critical access hospital emergency department provider comfort with seriously ill pediatric patients
.
Pediatr Emerg Care
.
2014
;
30
(
6
):
397
402
81
Figueroa
MI
,
Sepanski
R
,
Goldberg
SP
,
Shah
S
.
Improving teamwork, confidence, and collaboration among members of a pediatric cardiovascular intensive care unit multidisciplinary team using simulation-based team training
.
Pediatr Cardiol
.
2013
;
34
(
3
):
612
619
82
Popp
J
,
Yochum
L
,
Spinella
PC
,
Donahue
S
,
Finck
C
.
Simulation training for surgical residents in pediatric trauma scenarios
.
Conn Med
.
2012
;
76
(
3
):
159
162
83
Stocker
M
,
Allen
M
,
Pool
N
, et al
.
Impact of an embedded simulation team training programme in a paediatric intensive care unit: a prospective, single-centre, longitudinal study
.
Intensive Care Med
.
2012
;
38
(
1
):
99
104
84
Straka
K
,
Burkett
M
,
Capan
M
,
Eswein
J
.
The impact of education and simulation on pediatric novice nurses’ response and recognition to deteriorating
.
J Nurses Staff Dev
.
2012
;
28
(
6
):
E5
E8
85
Kane
J
,
Pye
S
,
Jones
A
.
Effectiveness of a simulation-based educational program in a pediatric cardiac intensive care unit
.
J Pediatr Nurs
.
2011
;
26
(
4
):
287
294
86
Allan
CK
,
Thiagarajan
RR
,
Beke
D
, et al
.
Simulation-based training delivered directly to the pediatric cardiac intensive care unit engenders preparedness, comfort, and decreased anxiety among multidisciplinary resuscitation teams
.
J Thorac Cardiovasc Surg
.
2010
;
140
(
3
):
646
652
87
Tofil
NM
,
White
ML
,
Grant
M
, et al
.
Severe contrast reaction emergencies high-fidelity simulation training for radiology residents and technologists in a children’s hospital
.
Acad Radiol
.
2010
;
17
(
7
):
934
940
88
von Arx
D
,
Pretzlaff
R
.
Improved nurse readiness through pediatric mock code training
.
J Pediatr Nurs
.
2010
;
25
(
5
):
438
440
89
Toback
SL
,
Fiedor
M
,
Kilpela
B
,
Reis
EC
.
Impact of a pediatric primary care office-based mock code program on physician and staff confidence to perform life-saving skills
.
Pediatr Emerg Care
.
2006
;
22
(
6
):
415
422

Competing Interests

CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no conflicts of interest to disclose.

Supplementary data