Clinical knowledge and skills acquired during training programs like Helping Babies Breathe (HBB) and Essential Care for Every Baby (ECEB) decay within weeks or months. We assessed the effect of a peer learning intervention paired with mentorship on retention of HBB and ECEB skills, knowledge, and teamwork in 5 districts of Uganda.
We randomized participants from 36 Ugandan health centers to control and intervention arms. Intervention participants received HBB and ECEB training, a 1 day peer learning course, peer practice scenarios for facility-based practice, and mentorship visits at 2 to 3 and 6 to 7 months. Control arm participants received HBB and ECEB training alone. We assessed clinical skills, knowledge, and teamwork immediately before and after HBB/ECEB training and at 12 months.
Peer learning (intervention) participants demonstrated higher HBB and ECEB skills scores at 12 months compared with control (HBB: intervention, 57.9%, control, 48.5%, P = .007; ECEB: intervention, 61.7%, control, 49.9%, P = .004). Knowledge scores decayed in both arms (intervention after course 91.1%, at 12 months 84%, P = .0001; control after course 90.9%, at 12 months 82.9%, P = .0001). This decay at 12 months was not significantly different (intervention 84%, control 82.9%, P = .24). Teamwork skills were similar in both arms immediately after training and at 12 months (intervention after course 72.9%, control after course 67.2%, P = .02; intervention at 12 months 70.7%, control at 12 months 67.9%, P = .19).
A peer learning intervention resulted in improved HBB and ECEB skills retention after 12 months compared with HBB and ECEB training alone.
Neonatal care knowledge and skills acquired during conventional training courses decay soon after training.
This study documents the effects of a facility-based peer learning intervention and mentorship on neonatal care knowledge and skills retention among frontline health workers over a 1-year period.
Most newborn deaths in sub-Saharan Africa are preventable.1–4 In Uganda, neonatal mortality remains high at 27 deaths per 1000 live births,5 contributing to the highest proportion of deaths in children younger than 5 years old.5–7 Effective clinical management can prevent most neonatal deaths; however, this requires timely provision by skilled health providers to save newborn lives.2,8–16 Simulation-based courses like Helping Babies Breathe (HBB) and Essential Care for Every Baby (ECEB) support health provider knowledge and skills development.17–19 Unfortunately, retention of essential knowledge and skills among frontline health care providers after initial HBB and ECEB courses is poor.20–23 For example, in Rwanda, skills pass rates dropped to 43% 3 months after HBB training among health workers.20 The challenge of skills retention is not unique to HBB and ECEB; many areas are affected both in medicine and outside the field of medical practice.24–28 Best practices to promote the retention of neonatal knowledge and skill retention in constrained resources and especially rural settings are not well documented. Periodic onsite mentorship and repeat courses have proven to be hard to implement because of limited availability of experts.29 Costs of repeat courses are expensive to sustain at scale.30 High rates of skills decay have been documented soon after training.20,31–33 Retraining efforts for skills retention require frequent practice, ensure clinical care is not disrupted, and require no long absences from duty station by health providers traveling to a centralized location.32,33,34 This is especially relevant in rural Uganda, where facility staffing levels are minimal and patient volume is high.5,35,36
Peer learning is an educational strategy that promotes self-directed, peer-driven learning.37–41 In settings like undergraduate education, the benefit of peer-facilitated learning for both the peer learner and the peer facilitator is well documented.41–51 However, to date, less is documented about the efficacy of peer learning in neonatal knowledge and skill retention among frontline health workers in low- and middle-income countries. Peer-based learning is an integral part of the training methodology used in the Helping Babies Survive and Helping Mothers Survive skills courses. Understanding the effects of peer learning and mentorship on neonatal clinical care knowledge and skills will inform the design of human capacity development programs for clinical knowledge and skills retention in low-resource settings.
The purpose of this study was to assess the effectiveness of a package involving simulation-based training, a peer learning course followed by facility-based peer practice, and interval mentorship on the acquisition and retention of key neonatal care knowledge, skills, and teamwork among frontline health providers in Uganda.
Methods
Trial Setting and Design
We conducted a cluster randomized trial at 36 health facilities in 5 districts (Ntungamo, Isingiro, Rubirizi, Bushenyi, and Mbarara) in Southwestern Uganda. A health subdistrict (HSD) is a health administrative cluster at the county level. Each HSD was considered a cluster. The referral health facility in each county is called a Health Center IV, which in some cases is the District Hospital. Each HSD has several Health Center III (HC III) facilities (minor inpatient capacity) and HC II facilities (outpatient primary care only).
Study Sites
Twelve HSDs from within the 5 districts serving predominantly rural populations were selected for the study. The 12 HSDs were then randomly assigned to each of an intervention (n = 6) or control (n = 6) arm. During the study, selected HSDs did not have any ongoing or new maternal or newborn health interventions of similar natures.
Cluster Randomization
The HSD was chosen as a cluster to prevent contamination between study sites because there is significant staff interaction at sites within each HSD. A cluster unit was defined as 1 HC IV and 2 HC IIIs. Selected HSDs were itemized and entered into an MS Excel spreadsheet. A random number generator in Stata version 12 was used to assign each cluster to either the control or intervention arm by the study team.
Study Participants
We enrolled clinically active health workers from the selected HC IVs and HC IIIs whose primary responsibility was provision of maternal newborn and child health services including providing labor and delivery services. All study participants provided written informed consent at enrollment.
Interventions
Three interventions were implemented: (1) a novel peer learning course, (2) peer learning practice, and (3) a facility-based mentorship program.
Peer Learning Course
The peer learning course was collaboratively developed by the Simulation for Life staff at Mbarara University of Science and Technology (MUST) and the KidSIM Program at the University of Calgary. The peer learning course objectives included understanding and applying teamwork concepts and understanding the concept of peer learning and its importance to knowledge, skills, and health center improvement. The course used a low-fidelity simulation manikin (NeoNatalie) and peer learning cards. Peer learning cards were introduced via lecture, followed by faculty demonstration and peer learning practice (Supplemental Fig 7). Each card in the peer learning card set was structured in 4 sections: (1) a prebriefing section intended to help establish a safe learning environment, identify peer learner practice goals, and introduce the peer learner to the manikin (Supplemental Fig 8); (2) a key knowledge check question intended for peer learners to vocalize the answer(s); (3) a peer practice scenario; and (4) a knowledge review and debrief section. The debrief section was structured to use a modified PEARLS blended method debriefing, incorporating learner self-assessment (ie, plus-delta) as a core component of the debriefing framework (Supplemental Figs 9 and 10).54,55 In total, there were 5 peer learning cards.
Before implementation of the course, the development team pilot tested the peer learning course 3 times (July, August, and November 2017) to optimize course delivery, logistics, and participant experience. Pilot course participants were Helping Babies Survive trainers and newborn care providers (midwives and clinical officers) drawn from nonstudy sites. Participants provided feedback on course content, peer learning cards, and practice sessions. Key modifications resulting from the pilot courses included editing grammar to better align with the local context, improvements to the peer learning cards, and fine tuning of course facilitation methods.
Peer Learning Practice
From each intervention facility, 2 health providers, selected at the sole discretion of the facility leadership, were trained through the peer learning course as peer facilitators. These facilitators were expected to subsequently organize and provide peer practice sessions with their facility colleagues (called “peer providers”) at least once every month. The peer practice cards guided each peer practice session; sessions intended to facilitate passage of knowledge and skill sharing from peer facilitators to peer providers (see Fig 1 for structure of peer card design and steps for peer practice). Peer providers were allowed to be mentored by their peer facilitator colleague into becoming peer facilitators. Every peer learning practice session was registered in a practice log documenting date, participants, scenario(s) practiced and participant roles (peer facilitator, peer provider, observer).
All health providers in intervention and control sites received standard HBB and ECEB training. A package of simulation equipment was provided to all health facilities. This package included a NeoNatalie manikin, suction devices, resuscitation bag and mask, HBB and ECEB action plans, practice antibiotics, syringes, and other supplies relevant to every practice scenario card.
Facility-based Mentorship
Each intervention facility was visited by a clinical mentors twice: at 2 to 3 and 6 to 7 months after training. Visits were conducted by at least 1 pediatric simulation clinical specialist (from MUST or KidSIM). A mentor met with peer facilitators plus additional available facility staff for a practice session. Visits were standardized through clear objectives and a checklist to ensure demonstration, review, and practice of peer scenario cards by facility-based peer facilitators; formal debriefing was done with feedback provided to all participants. In the context of this study, mentors were skilled health providers who trained a cohort of frontline health workers (mentees) in HBB, ECEB, and peer learning methodology. During mentorship sessions, mentors worked side by side conducting peer practice sessions with their mentees. Mentees were first allowed to lead peer practice sessions. Mentors would perform corrective demonstrations of peer sessions when improvements were required, followed by return demonstrations. Performance strengths and gaps were rediscussed and documented at the peer learning mentorship exit meeting.
Additionally, between visits, WhatsApp (a mobile internet-based communication application) facilitated virtual communication with intervention facility participants to respond to questions and provide advance notice of upcoming mentorship visits. When planned mentorship visits failed to connect with 1 or more peer facilitator, another mentorship visit with a practice session involving that peer facilitator was arranged.
Outcome Measures
Our primary outcome measure was HBB and ECEB Objective Structured Clinical Assessment (OSCE) scores, demonstrating newborn resuscitation and care skills, using the standardized HBB (OSCE B) and ECEB (OSCE B) course skills checklists.19,54 Secondary outcomes were newborn care knowledge and team work skills. Knowledge was measured using multiple choice questions (MCQs; supplemental Figure 10) selected from the HBB 2.0 and ECEB courses.18,54 Team work performance was measured using the Clinical Teamwork scale (CTS), a 15-item scale with validity evidence supporting its use in simulation-based training.55 The CTS was applied during a standardized scenario to assess team performance of 2 study participants that were not from the same facility (but who were allocated to the same study arm).
Outcome measures were assessed at 3 time points: (1) at baseline before HBB and ECEB courses; (2) after HBB and ECEB courses, immediately on completing the 2 courses; and (3) at 12 months after HBB and ECEB courses. All assessments were centrally conducted at MUST by trained and calibrated raters to score OSCEs. Raters were trained using prerecorded HBB/ECEB OSCE scenarios performed by the study team (MUST/KidSIM staff). OSCE raters were 4 medical doctors not previously involved in the study. Rater calibration was considered complete when a κ value of 0.8 was achieved.
Sample Size
The sample size calculation was based on the percentage passing rate in the intervention and control groups. We assumed a 50% average mark in the control arm and 90% average mark in the intervention arm with a 40% difference in the percent of participants’ scores between the intervention versus control arms. The intracluster correlation was estimated to be 0.05. Using the formula from Hayes et al,56 with an estimated total cluster participant group size of 18 people per HSD, we required an estimated total of 110 health workers per arm to detect the desired difference with a power of 80% at 95% level of significance.
Statistical Methods and Data Analysis
Assessment scores and demographic data were entered into Microsoft Excel and exported to Stata 12. We summed up scores for each of the following: (1) HBB/ECEB OSCEs, (2) MCQ knowledge scores; and (3) CTS teamwork scores. Maximum possible scores on these scales were 29, 13, 25, and 15 for HBB, ECEB, MCQs, and CTS, respectively. Our data were uniformly distributed. We used t tests to compare mean scores at each time point. A P value less than .05 indicated significant differences between the intervention and control arms.
This study was approved by the Mbarara University research ethics committee and the Uganda National Council of Science and Technology.
Results
Study Participants
We enrolled a total of 125 participants (63 control, 62 intervention) from 36 health facilities within 12 HSDs. Participant characteristics (see Fig 2 for study flow diagram). Participant characteristics (Table 1) were comparable in control and intervention arms including previous participant exposure to HBB and ECEB training. Midwives composed the majority (79.2%) of study participants. The study started in January 2018 and ended in February 2019. No participant declined participation in the study.
Clinical Skills, OSCE Scores
Participants in the intervention and control arms had similar HBB and ECEB skills at baseline (HBB: intervention 22.5%, control 24.5%, P = .41; ECEB: intervention 24.2%, control 20.3%, P = .07). Skill scores are shown in Table 2. Figure 3 graphically represents the interarm, mean scores of ECEB skills over time. Figure 4 represents the interarm, mean scores of HBB skills over time.
These skills significantly increased immediately after HBB/ECEB training (HBB intervention arm: before course 22.5%, after course 71.3%, P < .001; HBB control arm: before course 24.5%, after course 72.9%, P < .001; ECEB intervention: before course 24.2%, after course 73%, P < .001; ECEB control arm: before course 20.3%, after course 66.7%, P < .001). At 12 months after training, HBB and ECEB skills had significantly decayed in both the intervention and control arms (Table 2). However, the peer learning (intervention) arm had significantly better skills at 12 months compared with controls in both HBB and ECEB (HBB: intervention at 12 months 57.9%, control at 12 months 48.5%, P = .007; ECEB: intervention at 12 months 61.7%, control at 12 months 49.9%, P = .004).
Knowledge Assessments (MCQs)
Newborn care knowledge increased after training in both intervention and control arms (intervention before course 75.9%, intervention after course 91.1%, P < .001; control before course 73.3%, after course 90.9%, P < .001). In both study arms, knowledge significantly decayed over 12 months with no significant interarm difference in knowledge after 12 months (intervention 84%, control 82.9%, P = .24) as represented in Fig 5.
Teamwork Skills
Teamwork scores at all intervals are shown in Table 3. There was a large difference in teamwork assessment scores on postassessments compared with preassessments in both intervention and control groups (Fig 6) (intervention before course 55.8%, after course 72.9%, P < .001; control before course 51.5%, control after course 67.2%, P < .001). Twelve months after initial training, teamwork scores remained high compared with baseline in intervention and in control groups with no significant interarm differences (intervention 70.7%, control 67.9%, P = .19).
Discussion
Our study has shown that the implementation of a simulation-based peer-to-peer learning intervention with purposeful mentorship significantly improves retention of HBB and ECEB skills among frontline health workers. The sustained impact of HBS courses has been affected, in part, by skills decay.20–23 The results of our facility-based and frontline-health worker-facilitated peer learning intervention presents a promising alternative to retraining and protracted mentorship. In the intervention arm, mentorship was conducted at 2 to 3 months and 6 months after training. This was followed by 6 months of peer learning without mentorship to the 12 month assessment. This study design was important in the interpretation of the impact of peer learning in the context of support supervision because it has been demonstrated that clinical care skills significantly decay from as early as 3 to 4 months after training.22,57 Though we did not assess participants’ skills by the end of the 6 month mentorship in both the intervention and control arms, previous HBB skills decay evidence makes it most likely that the better intervention skills performance at 12 months (6 months after mentorship), were most likely attributable to the peer learning intervention. This is supported by studies that demonstrated that frequent skills practice leads to reduced rates of skills decay. Most of these studies had shorter follow-up periods.58,59 A study in Nepal involved daily practice sessions to achieve skills retention at 6 months22 ; our study did not have this rigor of skills practice. Our reduced skills decay may be attributable to the structured peer feedback. The benefits of feedback are supported by studies of feedback interventions that resulted in better outcomes.45,60–63
Peer learning has been demonstrated to create an effective learning environment, increase participant motivation to read and learn more, boost confidence of both the peer leader and peer learner, and contribute to knowledge and skills acquisition.39,42,45,47,48 Most peer learning interventions have been trialed among students but are limited among in-service health providers.42,48,49,51 Our study adds to the growing literature for peer learning in in-service settings. Other in-service health provider–focused, peer-led interventions demonstrated slower skills decay compared with control as shown in our study.43,64,65
Our study demonstrated minimum benefit of peer learning on the retention of newborn knowledge, 12 months posttraining, among in-service health professionals. This finding is similar to a 12 month follow-up intervention among in-service health providers in Sudan.66 However, this contrasted with the neonatal care knowledge retention results in a simulation-based study that demonstrated skills retention to 10 months after newborn simulation scenario exposure among preservice medical and nursing students in Uganda.67 The differences in pattern of knowledge retention among in-service health professionals characterized by decay over time compared with that of students, characterized by retention over time, may be attributable to the fact that preservice student assessments tend to be largely knowledge based compared with the heavy skills requirements in daily clinical practice. Shorter follow-up studies among in-service health workers have demonstrated neonatal care knowledge retention.59,68,69
Teamwork assessment was done in teams of 2 health workers. The lack of differences in teamwork between the intervention and control arms might be related to the low number of providers in the team.
Mentorship was done to encourage peer practice and to address peer practice skills challenges common with adoption of new interventions. The likely contribution of mentorship in our study may be related to encouraging correct peer practice. We are reserved to claim mentorship benefits beyond what is stated here because of the 6-month period to endline with no mentorship.
Limitations
We did not conduct a skills and knowledge assessment at the 6-month mark when support supervision was ended. An assessment at this point could have been important to more accurately determine the approximate effect of mentorship combined with peer learning on knowledge and skills retention.
Conclusion
Peer learning with initial mentorship significantly decreases neonatal care skills decay among frontline health providers after HBB and ECEB courses. This program provides an illustrative example of how peer learning reinforced with mentorship provides a potentially sustainable option for maintaining a more skilled health care team in a low-resource, rural setting.
Drs Data, Cheng, and Bajunirwe and Ms Mirette conceptualized the study design, and drafted the initial manuscript; Drs Cherop, Singhal, Kyakwera, Brenner, Abesiga, Namata, Twine, and Wishart and Ms McIntosh participated in different aspects of study design, data collection, and reviewed and edited the manuscript; and all authors approved the final manuscript as submitted and agreed to be accountable for all aspects of the work.
This trial has been registered at www.clinicaltrials.gov (identifier NCT03252275).
FUNDING: F unding for this study was provided by the Laerdal Foundation for Acute Medicine (grant number: 40162), the Elma Foundation (grant number: 16-F0021), and the International Development Research Center, Canada (grant number: 108217-001). All funders had no role in any aspects of this research work.
CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no potential conflicts of interest to disclose.
