Violence is a global public health problem, and early childhood interventions are a core component of violence prevention programming. Interventions to support parents and teachers of young children can prevent violence against children by caregivers and prevent the early development of antisocial behavior. However, there is limited guidance on how to scale up these programs in low- and middle-income countries.
In this article, we describe how we applied implementation science principles in the design, implementation, evaluation, and initial scaling of 2 complementary early childhood, violence prevention, caregiver training programs in Jamaica: the Irie Classroom Toolbox (a teacher-training program) and the Irie Homes Toolbox (a parenting program).
We identified 7 implementation science principles most relevant to our work in scaling the Irie Toolbox programs and describe how these principles were operationalized in the Jamaican context. The principles are: (1) design programs for scale from the outset; (2) use learning cycles for quality improvement; (3) plan strategically for government agency adoption; (4) provide high-quality initial and ongoing training and regular supervision; (5) monitor implementation quality; (6) use flexible delivery modes; and (7) plan for program sustainment.
Through applying these principles to scale the Irie Toolbox programs, we aim to promote a consistent approach to reducing violence against children, reducing child behavior problems, and increasing caregiver and child competencies across both home and school contexts at the population level. The principles and processes described in this article are relevant to other behavior change interventions in early childhood development, education, and public health.
In low- and middle-income countries (LMIC), approximately two-thirds of 2- to 4-year-old children experience physical punishment or psychological aggression from caregivers at home.1,2 Violence in and around schools, including violence against children (VAC) by teachers, affects more than half a billion children each year.3 VAC leads to long-term negative consequences, including increased risk for physical and mental health problems, low academic attainment, and school dropout.4,5 VAC also leads to high global economic costs, with school violence alone costing an estimated US$11 trillion in lost wealth from loss of learning and school dropout.3,6
A key strategy for reducing VAC in early childhood is through caregiver support programs.7 Caregiver violence prevention programs can reduce child maltreatment and child behavior problems.8,9 However, few interventions have been implemented at scale and there is limited guidance on how to facilitate wide-scale dissemination, especially in LMIC. In addition, although there is a growing literature from LMIC on early childhood, violence prevention parenting interventions,10 the evidence for teacher training programs is limited, with studies conducted primarily in primary and secondary schools.11,12
In this article, we describe how we used implementation science principles in the design, implementation, evaluation, and initial scaling of 2 complementary early childhood, violence prevention programs through the existing early childhood education network in Jamaica, including: (1) a teacher training intervention (The Irie Classroom Toolbox [ICT]) and (2) a parenting intervention (The Irie Homes Toolbox [IHT]). Irie is a Jamaican term that means “good,” “pleasing,” and “at peace with oneself and the world.” The Irie Toolbox programs target caregivers of children aged 2 to 8 years and address 2 important public health problems: (1) violence against children and (2) child behavior problems. The aim of this program of work is to support the implementation of evidence-based, early childhood, violence prevention programs at the population level, across both home and school settings.
Jamaican Context
There is a recognized need for violence prevention programs in Jamaica, with 84% of Jamaican caregivers with children aged 2 to 4 years reporting use of corporal punishment within the past month,13 and between 88% and 100% of early childhood teachers using VAC over 2 days of observation.14,15 The high prevalence of VAC by teachers is found despite the legal ban against corporal punishment in Jamaican early childhood institutions, indicating that legislation needs to be accompanied by additional actions.16,17 There is no legal ban on parents’ use of corporal punishment. Interventions to prevent child behavior problems are also necessary. The prevalence of disruptive behavior disorders among 5- to 6-year-old children is 12%, with limited access to appropriate services.18 In addition, we previously reported that 21% of 3- to 6-year-old children had high levels of conduct problems by teacher report.19,20 Conduct problems place children at increased risk of developing behavior disorders and for academic underachievement, school dropout, and crime and violence in adulthood.21
There are also good opportunities for intervention. First, Jamaica is a pathfinder country in the Global Partnership to End Violence Against Children, and violence prevention is a national strategic priority.22 Second, more than 98% of 3- to 6-year-old children are enrolled in early childhood educational provision. Interventions integrated into these services are nonstigmatizing and conveniently located, maximizing accessibility and acceptability, with potential for population-level reach. Third, there is a strong organizational structure through the Early Childhood Commission (ECC) to support implementation of violence prevention programming. The ECC is the government body with oversight for early childhood institutions, has the responsibility for setting and maintaining standards, and provides ongoing professional development for teachers.
The IRIE Toolbox Programs
Evidence of Effectiveness in Jamaica
In an earlier efficacy trial, we found that training preschool teachers in classroom behavior management and how to promote children’s social-emotional competence reduced child conduct problems and increased child social skills at home and at school among preschool children with heightened levels of conduct problems (Table 1).19 Benefits were also found to teacher practices and the classroom atmosphere, with these benefits sustained at 6-month follow-up (Table 2).23 The preschool teacher training intervention led to significant benefits to children’s academic achievement, oral language, self-regulation, and school attendance in grade 1 of primary school (Table 2).24 This efficacy trial led to the development the ICT: an early childhood, teacher training, violence prevention program specifically for use within LMIC (Fig 1).20 We evaluated the ICT in an effectiveness trial in preschools and a small trial in grade 1 of primary school and found large reductions in teachers’ use of VAC and significant improvements in the quality of the classroom environment (Table 1).14,15 Benefits were sustained at 1-year follow-up (Table 2).15 Benefits were also found for child behavior, aspects of child school achievement, and teachers’ professional well-being and retention.14,15
Trials of Early Childhood Violence Prevention Interventions in Jamaica
| Study | Sample | Intervention | Short-Term Outcomes |
|---|---|---|---|
| Teacher training | |||
| Efficacy trial in 24 preschools.19,23 Children aged 3–6 y | 24 preschools randomly assigned to intervention (n = 12 preschools, 37 classrooms) or control (12 preschools, 36 classrooms). Three children with the highest level of conduct problems from each class participated in the evaluation of individual child outcomes (113 intervention, 112 control). | Teachers in schools allocated to the intervention were trained in classroom behavior management and how to promote young children social-emotional competence through: • 8 full-day teacher training workshops and • Four 1-h individual sessions of in-class support: once a month for 4 mo Intervention used a hybrid of the Incredible Years Teacher Training Program (50%) and the content of the Irie Classroom Toolbox (50%) | Teacher outcomes (measured through observation): • Increased teacher positive behaviors: ES = 3.35 (95% CI, 2.70–3.98) • Decreased teacher negative behaviors: ES = –1.29 (95% CI, –0.87 to –1.71) • Increased teacher warmth: ES = 2.03 (95% CI, 1.41–2.67) Child outcomes Observed class-wide child behavior: • Increased of class-wide appropriate behavior: ES = 0.73 (95% CI, 0.17–1.30) • Increased class-wide child interest and enthusiasm: ES = 0.98 (95% CI, 0.48–1.50) Individual child outcomes: • Reductions in observed conduct problems at school: ES = –0.42 (95% CI, –0.12 to –0.71) • Increased observed friendship skills at school: ES = 0.74 (95% CI, 0.41–1.40) • Decreased teacher-reported behavior difficulties: ES = –0.47 (95% CI, –0.18 to –0.76) • Increased teacher-reported social skills: ES = 0.59 (95% CI, 0.35– 0.84) • Decreased parent reported behavior difficulties: ES = –0.22 (95% CI, –0.03 to –0.42) • Increased child attendance: ES = 0.30 (95% CI, 0.05–0.55) |
| Pilot study in 14 primary schools.14 Children aged 6–7 y | 14 primary schools randomly assigned to intervention (n = 7) or control (n = 7). Teachers of grade 1 children participated in the study (27 intervention, 28 control). Four children from each class were randomly selected to participate in the evaluation of individual outcomes. | Teachers in schools allocated to the intervention were trained in selected content from the first three modules of the Irie Classroom Toolbox through: • 12 h of teacher training workshops • Eight 1-h individual sessions of in-class support | Teacher outcomes • Reductions in teachers’ use of violence against children through observation: ES = –0.73 (95% CI, –0.15 to –1.31) • No benefits to teachers’ professional well-being: ES = –0.11 (95% CI, –0.63 to 0.43) Classroom outcomes Quality of the classroom environment through observation (using the CLASS K-3) • Increased emotional support: ES = 1.22 (95% CI, 0.57–1.87) Child outcomesObserved class-wide child behavior: • No benefits to class-wide aggression: ES = –0.20 (95% CI, –0.73 to 0.33) or prosocial behavior: ES = 0.18 (95% CI, –0.36 to 0.72) Individual child outcomes • Marginal benefits to a composite of child self-regulation, language and math reasoning skills: ES = 0.25 (95% CI, –0.02 to 0.52), P = .07 • No significant benefit to child behavior difficulties or prosocial skills on the SDQ or to a composite of child academic achievement in reading, spelling, and phonics. |
| Effectiveness trial in 76 preschools.15 Children aged 3–6 y | 76 preschools randomly assigned to intervention (n = 38 preschools, 119 classrooms) or control (n = 38 preschools, 110 classrooms). Up to 12 children aged 4 y were randomly selected to participate in the evaluation of individual child outcomes (441 intervention, 424 control) | Teachers in schools allocated to the intervention were trained in the Irie Classroom Toolbox through: • Five full-day teacher training workshops (30 h) • Eight 1-h individual sessions of in-class support | Teacher outcomes • Intervention teachers used less violence against children (measured through observation) than control teachers: –67.12% (95% CI, –53.52 to –80.71). • Benefits to teachers’ professional well-being not significant: ES = 0.18 (95% CI, –0.03 to 0.39) Classroom outcomes Quality of the classroom environment through observation (using the CLASS Pre-K) • Increased emotional support: ES = 0.65 (95% CI, 0.43–0.88) • Increased classroom organization: ES = 0.49 (95% CI, 0.24–0.74) • Increased instructional support: ES = 0.61 (95% CI, 0.31–0.97) Child outcomes Observed class-wide child behavior: • No benefits to class-wide child aggression: ES = 0.07 (95% CI, –0.16 to 0.29) • Increased class-wide prosocial behavior: ES = 0.42 (95% CI, 0.17–0.71) Individual child outcomes • Increased child inhibitory control through direct testing: ES = 0.18 (95% CI, 0.05–0.32) • Fewer children in intervention schools in the clinical range for behavior difficulties: odds ratio, 0.46 (0.22–0.94) • No significant benefits to child behavior difficulties or prosocial skills on the SDQ |
| Parenting interventions | |||
| Efficacy trial in 18 preschools.26 Children aged 2–6 y | 18 preschools randomly assigned to intervention (n = 9) or control (n = 9) Minimum of 12 parents per school recruited into the study (115 intervention, 108 control) | Parents in intervention schools participated in the Irie Homes Toolbox program. Sessions were held once a week for 8 wk with groups of 6 parents on the preschool compound. | Parent outcomes • Reductions in parents’ use of violence against their child: ES = –0.29 (95% CI, –0.05 to 0.52) • Increases in parents’ involvement with their child: ES = 0.30 (95% CI, 0.03–0.57) Child outcomes • No main effects to parent- and teacher-reported child behavior difficulties on the SDQ • Significant reductions in parent-reported child behavior difficulties for children above the 50th percentile on initial behavior difficulties: ES = –0.36 (95% CI, –0.03 to –0.68) |
| Efficacy trial of virtual Irie Homes Toolbox.27 Children aged 2–6 y | 1113 parents (recruited via SMS) randomly assigned to intervention (n = 557) or control (n = 556) | Parents allocated to the intervention group were invited to participate in the Virtual Irie Homes Toolbox delivered over a 10-wk period consisting of: • Access to a data free app • 3 SMS messages/wk • Weekly 1-h virtual parenting sessions with groups of 8–9 parents • Weekly session e-summaries sent via WhatsApp | Parent outcomes • Reductions in parents’ use of violence against their child: ES = –0.12 (95% CI, –0.01 to –0.24) at posttest; ES = –0.13 (95% CI, –0.01 to –0.25) at 9-mo follow-up • Reductions in parents’ attitudes toward violence against children: ES = –0.20 (95% CI, –0.10 to –0.30) at posttest; ES = –0.14 (95% CI, –0.02 to –0.26) at 9- mo follow-up Child outcomes • Reductions to child emotional problems on the SDQ: ES = –0.17 (95% CI, –0.07 to –0.27) • No benefits to child conduct problems on the SDQ |
| Study | Sample | Intervention | Short-Term Outcomes |
|---|---|---|---|
| Teacher training | |||
| Efficacy trial in 24 preschools.19,23 Children aged 3–6 y | 24 preschools randomly assigned to intervention (n = 12 preschools, 37 classrooms) or control (12 preschools, 36 classrooms). Three children with the highest level of conduct problems from each class participated in the evaluation of individual child outcomes (113 intervention, 112 control). | Teachers in schools allocated to the intervention were trained in classroom behavior management and how to promote young children social-emotional competence through: • 8 full-day teacher training workshops and • Four 1-h individual sessions of in-class support: once a month for 4 mo Intervention used a hybrid of the Incredible Years Teacher Training Program (50%) and the content of the Irie Classroom Toolbox (50%) | Teacher outcomes (measured through observation): • Increased teacher positive behaviors: ES = 3.35 (95% CI, 2.70–3.98) • Decreased teacher negative behaviors: ES = –1.29 (95% CI, –0.87 to –1.71) • Increased teacher warmth: ES = 2.03 (95% CI, 1.41–2.67) Child outcomes Observed class-wide child behavior: • Increased of class-wide appropriate behavior: ES = 0.73 (95% CI, 0.17–1.30) • Increased class-wide child interest and enthusiasm: ES = 0.98 (95% CI, 0.48–1.50) Individual child outcomes: • Reductions in observed conduct problems at school: ES = –0.42 (95% CI, –0.12 to –0.71) • Increased observed friendship skills at school: ES = 0.74 (95% CI, 0.41–1.40) • Decreased teacher-reported behavior difficulties: ES = –0.47 (95% CI, –0.18 to –0.76) • Increased teacher-reported social skills: ES = 0.59 (95% CI, 0.35– 0.84) • Decreased parent reported behavior difficulties: ES = –0.22 (95% CI, –0.03 to –0.42) • Increased child attendance: ES = 0.30 (95% CI, 0.05–0.55) |
| Pilot study in 14 primary schools.14 Children aged 6–7 y | 14 primary schools randomly assigned to intervention (n = 7) or control (n = 7). Teachers of grade 1 children participated in the study (27 intervention, 28 control). Four children from each class were randomly selected to participate in the evaluation of individual outcomes. | Teachers in schools allocated to the intervention were trained in selected content from the first three modules of the Irie Classroom Toolbox through: • 12 h of teacher training workshops • Eight 1-h individual sessions of in-class support | Teacher outcomes • Reductions in teachers’ use of violence against children through observation: ES = –0.73 (95% CI, –0.15 to –1.31) • No benefits to teachers’ professional well-being: ES = –0.11 (95% CI, –0.63 to 0.43) Classroom outcomes Quality of the classroom environment through observation (using the CLASS K-3) • Increased emotional support: ES = 1.22 (95% CI, 0.57–1.87) Child outcomesObserved class-wide child behavior: • No benefits to class-wide aggression: ES = –0.20 (95% CI, –0.73 to 0.33) or prosocial behavior: ES = 0.18 (95% CI, –0.36 to 0.72) Individual child outcomes • Marginal benefits to a composite of child self-regulation, language and math reasoning skills: ES = 0.25 (95% CI, –0.02 to 0.52), P = .07 • No significant benefit to child behavior difficulties or prosocial skills on the SDQ or to a composite of child academic achievement in reading, spelling, and phonics. |
| Effectiveness trial in 76 preschools.15 Children aged 3–6 y | 76 preschools randomly assigned to intervention (n = 38 preschools, 119 classrooms) or control (n = 38 preschools, 110 classrooms). Up to 12 children aged 4 y were randomly selected to participate in the evaluation of individual child outcomes (441 intervention, 424 control) | Teachers in schools allocated to the intervention were trained in the Irie Classroom Toolbox through: • Five full-day teacher training workshops (30 h) • Eight 1-h individual sessions of in-class support | Teacher outcomes • Intervention teachers used less violence against children (measured through observation) than control teachers: –67.12% (95% CI, –53.52 to –80.71). • Benefits to teachers’ professional well-being not significant: ES = 0.18 (95% CI, –0.03 to 0.39) Classroom outcomes Quality of the classroom environment through observation (using the CLASS Pre-K) • Increased emotional support: ES = 0.65 (95% CI, 0.43–0.88) • Increased classroom organization: ES = 0.49 (95% CI, 0.24–0.74) • Increased instructional support: ES = 0.61 (95% CI, 0.31–0.97) Child outcomes Observed class-wide child behavior: • No benefits to class-wide child aggression: ES = 0.07 (95% CI, –0.16 to 0.29) • Increased class-wide prosocial behavior: ES = 0.42 (95% CI, 0.17–0.71) Individual child outcomes • Increased child inhibitory control through direct testing: ES = 0.18 (95% CI, 0.05–0.32) • Fewer children in intervention schools in the clinical range for behavior difficulties: odds ratio, 0.46 (0.22–0.94) • No significant benefits to child behavior difficulties or prosocial skills on the SDQ |
| Parenting interventions | |||
| Efficacy trial in 18 preschools.26 Children aged 2–6 y | 18 preschools randomly assigned to intervention (n = 9) or control (n = 9) Minimum of 12 parents per school recruited into the study (115 intervention, 108 control) | Parents in intervention schools participated in the Irie Homes Toolbox program. Sessions were held once a week for 8 wk with groups of 6 parents on the preschool compound. | Parent outcomes • Reductions in parents’ use of violence against their child: ES = –0.29 (95% CI, –0.05 to 0.52) • Increases in parents’ involvement with their child: ES = 0.30 (95% CI, 0.03–0.57) Child outcomes • No main effects to parent- and teacher-reported child behavior difficulties on the SDQ • Significant reductions in parent-reported child behavior difficulties for children above the 50th percentile on initial behavior difficulties: ES = –0.36 (95% CI, –0.03 to –0.68) |
| Efficacy trial of virtual Irie Homes Toolbox.27 Children aged 2–6 y | 1113 parents (recruited via SMS) randomly assigned to intervention (n = 557) or control (n = 556) | Parents allocated to the intervention group were invited to participate in the Virtual Irie Homes Toolbox delivered over a 10-wk period consisting of: • Access to a data free app • 3 SMS messages/wk • Weekly 1-h virtual parenting sessions with groups of 8–9 parents • Weekly session e-summaries sent via WhatsApp | Parent outcomes • Reductions in parents’ use of violence against their child: ES = –0.12 (95% CI, –0.01 to –0.24) at posttest; ES = –0.13 (95% CI, –0.01 to –0.25) at 9-mo follow-up • Reductions in parents’ attitudes toward violence against children: ES = –0.20 (95% CI, –0.10 to –0.30) at posttest; ES = –0.14 (95% CI, –0.02 to –0.26) at 9- mo follow-up Child outcomes • Reductions to child emotional problems on the SDQ: ES = –0.17 (95% CI, –0.07 to –0.27) • No benefits to child conduct problems on the SDQ |
CI, confidence interval; CLASS, Classroom Assessment Scoring System; ES, effect size; SDQ: Strengths and Difficulties Questionnaire.
Follow-up Studies of Early Childhood, Teacher-Training, Violence Prevention Programs
| Study | Sample | Longer Term Outcomes |
|---|---|---|
| Follow up of efficacy trial in 24 preschools.23,24 Children aged 3–6 y on enrollment | 24 preschools randomly assigned to intervention (n = 12 preschools, 37 classrooms) or control (12 preschools, 36 classrooms). Follow-up of teachers at 6 mo Teachers were followed up 6 mo after the end of the intervention. Follow-up of individual children in grade 1 of primary school Five children from each class with the highest level of initial conduct problems in preschool were followed up in grade 1 of primary school (181 intervention, 183 control). | Follow-up 6 mo after the end of intervention in the 24 preschools Observed teacher practices • Increased teacher positive behaviors: ES = 2.70 (95% CI, 2.00–3.41) • Decreased teacher negative behaviors: ES = –0.98 (95% CI, –0.52 to –1.44) • Increased teacher warmth: ES = 0.91 (95% CI, 0.46–1.43) Observed class-wide child behavior: • Increased class-wide appropriate behavior: ES = 0.50 (95% CI, 0.03–0.97) • Increased class-wide child interest and enthusiasm: ES = 0.78 (95% CI, 0.03–1.53) Individual child outcomes (measured in grade 1 of primary school): Child behavior • No significant benefits to observed conduct problems at school: ES = –0.13 (95% CI, –0.32 to 0.05) • Marginal benefits to teacher-reported conduct problems: (ES = –0.16 [95% CI, –0.35 to 0.02]) and social skills: ES = 0.19 (95% CI, –0.01 to 0.38) • No benefits to parent-reported conduct problems (ES = 0.10 [95% CI, –0.08 to 0.30]) and social skills: ES = –0.07 (95% CI, –0.27 to 0.14) Child school achievement, self-regulation, and school attendance: • Increased academic achievement: ES = 0.23 (95% CI, 0.04–0.42) • Increased oral language skills: ES = 0.28 (0.08–0.48) • Increased self-regulation: ES = 0.25 (0.07–0.43) • Increased child attendance: ES = 0.30 (95% CI, 0.10–0.49) |
| Follow up of effectiveness trial in 76 preschools.15 | 76 preschools randomly assigned to intervention (n = 38 preschools, 119 classrooms) or control (n = 38 preschools, 110 classrooms). Teachers were followed up 1 y after the end of the intervention | Teacher outcomes • Intervention teachers used less violence against children (measured through observation) than control teachers: –53.86% (95% CI, –71.08 to –36.65). • Improved teachers’ professional well-being: ES = 0.26 (95% CI, 0.03–0.48) • Increased teacher retention in intervention schools: 88% intervention vs 77% control teachers, P = .03 Classroom outcomes Quality of the classroom environment on the CLASS Pre-K • Increased emotional support: ES = 0. 50 (95% CI, 0.20–0.79) • Increased classroom organization: ES = 0.42 (95% CI, 0.14–0.69) • No significant benefits for instructional support: ES = 0.29 (95% CI, –0.06 to 0.57) Child outcomes Observed class-wide child behavior: • No benefits to class-wide child aggression: ES = –0.14 (95% CI, –0.42 to 0.16) or prosocial behavior: ES = 0.22 (95% CI, –0.08 to 0.53) |
| Study | Sample | Longer Term Outcomes |
|---|---|---|
| Follow up of efficacy trial in 24 preschools.23,24 Children aged 3–6 y on enrollment | 24 preschools randomly assigned to intervention (n = 12 preschools, 37 classrooms) or control (12 preschools, 36 classrooms). Follow-up of teachers at 6 mo Teachers were followed up 6 mo after the end of the intervention. Follow-up of individual children in grade 1 of primary school Five children from each class with the highest level of initial conduct problems in preschool were followed up in grade 1 of primary school (181 intervention, 183 control). | Follow-up 6 mo after the end of intervention in the 24 preschools Observed teacher practices • Increased teacher positive behaviors: ES = 2.70 (95% CI, 2.00–3.41) • Decreased teacher negative behaviors: ES = –0.98 (95% CI, –0.52 to –1.44) • Increased teacher warmth: ES = 0.91 (95% CI, 0.46–1.43) Observed class-wide child behavior: • Increased class-wide appropriate behavior: ES = 0.50 (95% CI, 0.03–0.97) • Increased class-wide child interest and enthusiasm: ES = 0.78 (95% CI, 0.03–1.53) Individual child outcomes (measured in grade 1 of primary school): Child behavior • No significant benefits to observed conduct problems at school: ES = –0.13 (95% CI, –0.32 to 0.05) • Marginal benefits to teacher-reported conduct problems: (ES = –0.16 [95% CI, –0.35 to 0.02]) and social skills: ES = 0.19 (95% CI, –0.01 to 0.38) • No benefits to parent-reported conduct problems (ES = 0.10 [95% CI, –0.08 to 0.30]) and social skills: ES = –0.07 (95% CI, –0.27 to 0.14) Child school achievement, self-regulation, and school attendance: • Increased academic achievement: ES = 0.23 (95% CI, 0.04–0.42) • Increased oral language skills: ES = 0.28 (0.08–0.48) • Increased self-regulation: ES = 0.25 (0.07–0.43) • Increased child attendance: ES = 0.30 (95% CI, 0.10–0.49) |
| Follow up of effectiveness trial in 76 preschools.15 | 76 preschools randomly assigned to intervention (n = 38 preschools, 119 classrooms) or control (n = 38 preschools, 110 classrooms). Teachers were followed up 1 y after the end of the intervention | Teacher outcomes • Intervention teachers used less violence against children (measured through observation) than control teachers: –53.86% (95% CI, –71.08 to –36.65). • Improved teachers’ professional well-being: ES = 0.26 (95% CI, 0.03–0.48) • Increased teacher retention in intervention schools: 88% intervention vs 77% control teachers, P = .03 Classroom outcomes Quality of the classroom environment on the CLASS Pre-K • Increased emotional support: ES = 0. 50 (95% CI, 0.20–0.79) • Increased classroom organization: ES = 0.42 (95% CI, 0.14–0.69) • No significant benefits for instructional support: ES = 0.29 (95% CI, –0.06 to 0.57) Child outcomes Observed class-wide child behavior: • No benefits to class-wide child aggression: ES = –0.14 (95% CI, –0.42 to 0.16) or prosocial behavior: ES = 0.22 (95% CI, –0.08 to 0.53) |
CI, confidence interval; CLASS, Classroom Assessment Scoring System; ES, effect size; SDQ, Strengths and Difficulties Questionnaire
The IHT is an early childhood, violence prevention parenting program developed to complement the ICT that includes content on understanding child behavior, positive discipline strategies, emotional self-regulation, and child-led play (Fig 1).25 It is designed to be delivered by ICT-trained preschool teachers, who are credible to parents, and have knowledge and expertise in using the strategies with children at school. In a small efficacy trial, the IHT was effective at reducing VAC by parents, increasing parent involvement with their child, and reducing conduct problems for children with heightened levels of conduct problems at baseline (Table 1).26 In response to the COVID-19 pandemic, we adapted the IHT for virtual delivery. In a randomized trial, the virtual IHT led to reductions in parents’ use of VAC and improved parent attitudes to violence with benefits sustained at 9-month follow-up.27
Beginning to Scale
The Irie Toolbox programs are designed to be integrated into the existing early childhood education services in Jamaica and delivered and supervised by existing staff. Within the ECC, each educational region has a senior development officer and senior inspector who each have responsibility for a team of 8 to 10 development officers and 6 to 8 inspectors, respectively. These field officers work at the district level with approximately 50 to 65 preschools each. Each region also has a community relations officer who coordinates parenting programs in the region. We are training and supporting these government staff members to implement the programs. Table 3 provides information on these activities. The ICT has been integrated into ongoing teacher training initiatives and is being scaled up nationally in Jamaica. The IHT is a more recent program, and our next step is to train ECC officers, who have been trained in the IHT, to train and support early childhood teachers to deliver the program.
Training Government Staff to Implement Early Childhood, Violence Prevention Programs
| Activity | Training of Government/Educational Staff by Research Team | Government Staff Delivered Intervention |
|---|---|---|
| Irie Classroom Toolbox | ||
| National dissemination of a 1-d training for primary school teachers in grades 1 to 3 | • We trained technical staff from the Ministry of Education, Jamaica to deliver a 1-d training for primary school teachers. • Technical staff were trained through 2 full-day workshops | • These technical staff trained all government teachers of grades 1, 2, and 3 as part of a national training initiative • Over a 2-y period, approximately 5000 teachers participated in the 1-d workshop • Teachers were trained in groups of 20–30 participants |
| Training ECC middle managers in the Irie Classroom Toolbox | • We trained 16 middle managers of the ECC (the senior inspector and senior development officer in each of the 8 educational regions) to deliver the Irie Classroom Toolbox training. • Middle managers attended 4-d residential training followed by monthly 1-d trainings for 6 mo. | • Middle managers worked in pairs within their region to deliver the Irie Classroom Toolbox training to a group of 20–25 preschool teachers over 2 school year. • More than 160 preschool teachers participated in this training. • Monthly supervision was provided by the research team. |
| ECC middle managers train ECC field officers in the Irie Classroom Toolbox | • After the middle managers had delivered the Irie Classroom Toolbox training to 1 group of teachers, we trained them to train the ECC field officers (development officers and inspectors) within their region to deliver the training with preschool teachers. • Middle managers attended 3 d of train-the-trainer workshops. | • Middle managers worked in pairs to train the ECC field officers to deliver the Irie Classroom Toolbox to teachers within their district. • All ECC field officers (n = 100) participated in 3 full-day training workshops with 10–16 field officers in each group. • Supervision was provided by the research team. Because of COVID-19, the teacher training was postponed. |
| Training ECC field officers in selected content from the Virtual Irie Classroom Toolbox | • We trained 42 ECC field officers (4–6 officers per educational region) to conduct virtual training sessions from the Irie Classroom Toolbox. • Officers were trained in groups of 14 participants for a total of 24 h of training. • Officers were trained to conduct 4 individual sessions. Before each conducting each session, officers attended two 3-h virtual training sessions (1 demonstration session and 1 practice session). | • ECC field officers worked in pairs to deliver the training to 2 groups of 20 preschool teachers within their educational region. • Preschool teachers attended 2-h virtual training sessions once a month for 4 mo. • A total of 840 preschool teachers participated in the training. • The middle managers supervised the teacher-training workshops |
| Irie Homes Toolbox | ||
| Training preschool teachers to implement the Irie Homes Toolbox | • We trained 2 teachers from 9 preschools (18 teachers) to deliver the Irie Homes Toolbox to parents of children in their school. • Teachers were trained through 2 full-day workshops. | • Preschool teachers delivered the Irie Homes Toolbox to 6–8 parents in their school (approximately 60 parents each school term). • The research team provided weekly or fortnightly supervision for the first round of implementation. |
| Training ECC field officers to deliver the Virtual Irie Homes Toolbox | • We trained 6 community relations officers and 24 field officers from the ECC to implement the Virtual Irie Homes Toolbox. • Officers were trained virtually with 15 officers per group • Training consisted of three 4-h virtual workshops before intervention implementation followed by weekly 2–3 h trainings for 9 wk during intervention implementation. | • Community relations officers worked with 4 groups of parents and field officers worked with 2 groups of parents. • Parents attended 1-h virtual sessions weekly for 10 wk in groups of 8–10 parents. • A total of 557 parents were recruited to participate in the training. |
| Activity | Training of Government/Educational Staff by Research Team | Government Staff Delivered Intervention |
|---|---|---|
| Irie Classroom Toolbox | ||
| National dissemination of a 1-d training for primary school teachers in grades 1 to 3 | • We trained technical staff from the Ministry of Education, Jamaica to deliver a 1-d training for primary school teachers. • Technical staff were trained through 2 full-day workshops | • These technical staff trained all government teachers of grades 1, 2, and 3 as part of a national training initiative • Over a 2-y period, approximately 5000 teachers participated in the 1-d workshop • Teachers were trained in groups of 20–30 participants |
| Training ECC middle managers in the Irie Classroom Toolbox | • We trained 16 middle managers of the ECC (the senior inspector and senior development officer in each of the 8 educational regions) to deliver the Irie Classroom Toolbox training. • Middle managers attended 4-d residential training followed by monthly 1-d trainings for 6 mo. | • Middle managers worked in pairs within their region to deliver the Irie Classroom Toolbox training to a group of 20–25 preschool teachers over 2 school year. • More than 160 preschool teachers participated in this training. • Monthly supervision was provided by the research team. |
| ECC middle managers train ECC field officers in the Irie Classroom Toolbox | • After the middle managers had delivered the Irie Classroom Toolbox training to 1 group of teachers, we trained them to train the ECC field officers (development officers and inspectors) within their region to deliver the training with preschool teachers. • Middle managers attended 3 d of train-the-trainer workshops. | • Middle managers worked in pairs to train the ECC field officers to deliver the Irie Classroom Toolbox to teachers within their district. • All ECC field officers (n = 100) participated in 3 full-day training workshops with 10–16 field officers in each group. • Supervision was provided by the research team. Because of COVID-19, the teacher training was postponed. |
| Training ECC field officers in selected content from the Virtual Irie Classroom Toolbox | • We trained 42 ECC field officers (4–6 officers per educational region) to conduct virtual training sessions from the Irie Classroom Toolbox. • Officers were trained in groups of 14 participants for a total of 24 h of training. • Officers were trained to conduct 4 individual sessions. Before each conducting each session, officers attended two 3-h virtual training sessions (1 demonstration session and 1 practice session). | • ECC field officers worked in pairs to deliver the training to 2 groups of 20 preschool teachers within their educational region. • Preschool teachers attended 2-h virtual training sessions once a month for 4 mo. • A total of 840 preschool teachers participated in the training. • The middle managers supervised the teacher-training workshops |
| Irie Homes Toolbox | ||
| Training preschool teachers to implement the Irie Homes Toolbox | • We trained 2 teachers from 9 preschools (18 teachers) to deliver the Irie Homes Toolbox to parents of children in their school. • Teachers were trained through 2 full-day workshops. | • Preschool teachers delivered the Irie Homes Toolbox to 6–8 parents in their school (approximately 60 parents each school term). • The research team provided weekly or fortnightly supervision for the first round of implementation. |
| Training ECC field officers to deliver the Virtual Irie Homes Toolbox | • We trained 6 community relations officers and 24 field officers from the ECC to implement the Virtual Irie Homes Toolbox. • Officers were trained virtually with 15 officers per group • Training consisted of three 4-h virtual workshops before intervention implementation followed by weekly 2–3 h trainings for 9 wk during intervention implementation. | • Community relations officers worked with 4 groups of parents and field officers worked with 2 groups of parents. • Parents attended 1-h virtual sessions weekly for 10 wk in groups of 8–10 parents. • A total of 557 parents were recruited to participate in the training. |
ECC, Early Childhood Commission.
As described previously and in Tables 1 and 2, the Irie Toolbox programs have proven effectiveness in reducing violence against children at home and at school and in reducing child behavior difficulties. In the following sections, we focus on how we have used 7 key implementation science principles, drawn from literature related to scaling-up interventions,28 from the initial design phase through to initial scaling of the Irie Toolbox programs.
Principle 1: Design for Scale
Scalable interventions need to be designed for scale from the outset.29 This requires a focus on acceptability, relevance, feasibility, and effectiveness of the content and delivery method, involving the beneficiaries, facilitators, supervisors, and the government agency responsible for scaling.30,31
To design the Irie Toolbox programs, we chose to transport evidence-based content and methods of delivery rather than transporting an existing program. Although transported programs can be effective in new contexts,32 issues with cost and ownership are barriers to scaling and gaining government buy-in. In addition, in previous work, we found that using locally developed content and methods reduced the amount of support teachers required for effective implementation.23 To inform intervention design, we used the Active Implementation Framework, which states successful programs require an effective innovation, implemented well, within an enabling context.33 Table 4 illustrates key factors within each category and associated strategies we used to design the ICT and IHT. These factors are relevant for scaling interventions in LMIC across public health,34 early childhood development,35 and education programming.36
Designing Scalable Interventions Using the Active Implementation Framework32
| Requirements for an Effective, Scalable Intervention | Strategies Used to Design the Irie Toolbox Program | |
|---|---|---|
| Effective innovation | • Evidence-based content and process of delivery | • Incorporate common core components of evidence-based, violence prevention programs • Use evidence-based behavior change techniques in intervention delivery • Develop a theory of change for the interventions |
| • Acceptable, feasible, and relevant for participants, facilitators, and supervisors | • Collaborate with end-users and key stakeholders in developing the intervention from the outset • Operationalize the content and process of delivery for the Jamaican context based on extensive formative research including: Interviews with end users and key stakeholders to identify perceived needs and potential enablers and barriers to program uptake Observations of caregiver-child interaction and the caregiving environment Iterative piloting of the intervention in a diverse sample of preschools | |
| • Structured, manualized intervention | • Develop detailed facilitator manuals and associated training resources • Develop resources for participant parents/teachers to support implementation (eg, content summaries, homework assignments, materials to use with children) | |
| Effective implementation | • In-built flexibility: program can be delivered differently according to preferences and needs | • Develop a modular intervention that can be delivered flexibly depending on staff and participant availability and participants’ needs • Develop resources that can be used in face-to-face and virtual training to permit blended delivery |
| • High quality training and ongoing support of frontline facilitators | • Develop clear training and supervision protocols (eg, documenting content, methods, frequency, duration, group size) • Provide sufficient training and support for staff responsible for training and supervision, with ongoing quality monitoring • Provide opportunities for training new staff | |
| • Availability of technical tools to support implementation | • Develop training and supervisory manuals and quality monitoring tools and train senior staff in their use | |
| • In-built monitoring and evaluation system improvements | • Collect timely monitoring and evaluation data that is easy to use • Build in iterative feedback loops to ensure lessons learnt inform program revisions | |
| Enabling context | • Alignment with mission and goals of the implementing organization | • Form a collaborative group with the Early Childhood Commission to explicitly map how the Irie Toolbox aligns with their Strategic Plan, the Early Childhood Curriculum, the Operational Standards for Early Childhood Institutions, and the Inspection Documents. |
| • Availability of frontline workers and supervisory staff: sufficient staff to implement the program and fit with existing duties | • Develop the intervention so that it can be feasibly delivered using the existing organizational structure and existing staff of the Early Childhood Curriculum and Ministry of Education. |
| Requirements for an Effective, Scalable Intervention | Strategies Used to Design the Irie Toolbox Program | |
|---|---|---|
| Effective innovation | • Evidence-based content and process of delivery | • Incorporate common core components of evidence-based, violence prevention programs • Use evidence-based behavior change techniques in intervention delivery • Develop a theory of change for the interventions |
| • Acceptable, feasible, and relevant for participants, facilitators, and supervisors | • Collaborate with end-users and key stakeholders in developing the intervention from the outset • Operationalize the content and process of delivery for the Jamaican context based on extensive formative research including: Interviews with end users and key stakeholders to identify perceived needs and potential enablers and barriers to program uptake Observations of caregiver-child interaction and the caregiving environment Iterative piloting of the intervention in a diverse sample of preschools | |
| • Structured, manualized intervention | • Develop detailed facilitator manuals and associated training resources • Develop resources for participant parents/teachers to support implementation (eg, content summaries, homework assignments, materials to use with children) | |
| Effective implementation | • In-built flexibility: program can be delivered differently according to preferences and needs | • Develop a modular intervention that can be delivered flexibly depending on staff and participant availability and participants’ needs • Develop resources that can be used in face-to-face and virtual training to permit blended delivery |
| • High quality training and ongoing support of frontline facilitators | • Develop clear training and supervision protocols (eg, documenting content, methods, frequency, duration, group size) • Provide sufficient training and support for staff responsible for training and supervision, with ongoing quality monitoring • Provide opportunities for training new staff | |
| • Availability of technical tools to support implementation | • Develop training and supervisory manuals and quality monitoring tools and train senior staff in their use | |
| • In-built monitoring and evaluation system improvements | • Collect timely monitoring and evaluation data that is easy to use • Build in iterative feedback loops to ensure lessons learnt inform program revisions | |
| Enabling context | • Alignment with mission and goals of the implementing organization | • Form a collaborative group with the Early Childhood Commission to explicitly map how the Irie Toolbox aligns with their Strategic Plan, the Early Childhood Curriculum, the Operational Standards for Early Childhood Institutions, and the Inspection Documents. |
| • Availability of frontline workers and supervisory staff: sufficient staff to implement the program and fit with existing duties | • Develop the intervention so that it can be feasibly delivered using the existing organizational structure and existing staff of the Early Childhood Curriculum and Ministry of Education. |
Principle 2: Use Learning Cycles for Quality Improvement
Intervention design, implementation and dissemination involve dynamic, iterative learning cycles.29,37 Quality improvement is an ongoing process that requires structured methods to document, analyze, and use information on program implementation from all stakeholders.
To inform adaptations of the Irie Toolbox programs, we embed process and qualitative evaluations within all implementation activities.14,20,25,38,39 Data collected include: reach (numbers enrolled and attendance), user satisfaction, quality of implementation at the level of the facilitator (using facilitator- and supervisor-completed checklists) and at the level of the teacher/parent (using homework assignments, participant feedback, and facilitator observations), and documenting enablers, barriers, and suggestions for improvement from the perspectives of beneficiary parents/teachers, facilitators, supervisors, and the ECC executive and research teams. These data are combined with data from our impact evaluations to make ongoing revisions to the program content (see Table 5 for examples). Developing additional content risks making interventions more complex, thus decreasing scalability.29,40 We mitigate these risks by developing flexible programs with a combination of core and optional modules.
Dynamic Adaptation During Initial Implementation
| Rationale for Adaptation | Source of the Evidence | Adaptations to Content |
|---|---|---|
| • It was challenging for parents and teachers to problem-solve what strategies were most appropriate to deal with different situations and different child behaviors. • Understanding situations from the child’s perspective was also difficult. | Facilitator reflections on training/in-class support sessions. Participant feedback during training/in-class support sessions. | • We designed additional content to ensure participants developed a better understanding of young children’s behavior and the reasons for child “misbehavior.” • We placed greater emphasis on children’s emotions and thoughts when discussing child behaviors. |
| • Self-identifying as an Irie parent or an Irie teacher was a powerful motivator for parents and teachers to adopt the strategies introduced through the programs. | Facilitator reflections on training/in-class support sessions. Qualitative interviews with participants and facilitators | • We included explicit references to being an Irie parent/Irie teacher throughout the program (eg, in discussions, problem-solving activities, practice activities, homework assignments). • Parents/teachers sign an “I am an Irie parent/teacher” oath on completion of the program. |
| • Some parents and teachers viewed corporal punishment as severe physical discipline only. Terms such as “touch,” “brush off” are used to describe corporal punishment that involves slapping. • Similarly, psychological aggression (eg, threatening to hit a child, yelling) is not perceived as violence. | Facilitator reflections on training/in-class support sessions. Participant feedback during training/in-class support sessions. Qualitative interviews with participants and facilitators | • We have included clear definitions of violence against children using common terminology used in Jamaica. • We explicitly link the use of positive discipline, and refraining to use corporal punishment and psychological aggression, as the behavior of an Irie parent/teacher. |
| • Although the frequency of teachers’ use of violence against children reduced by 67% in our effectiveness trial, 72% of teachers continued to use corporal punishment and 55% continued to use psychological aggression. • Teachers reported resorting to violence when they were frustrated by children’s behavior. | Quantitative and qualitative evaluation of intervention trials | • We have designed additional content to increase teachers’ executive function skills and self-regulatory capacities. • This includes strengthening the content around individual goal setting, and training teachers in “calming down” techniques (eg, stop, think and problem solve). |
| • Principal support was an important factor in setting school-wide expectations, promoting teachers’ use of the strategies and promoting the programs with parents. | Facilitator reflections on training/in-class support sessions. Qualitative interviews with participants and facilitators | • We developed four 90-min principal-training modules focusing on how to: (1) support teachers, (2) engage parents in the programs, and (3) create an “Irie school.” |
| • Teachers needed additional guidelines and support for integrating the strategies into everyday teaching and learning activities | Facilitator reflections on training/in-class support sessions. Participant feedback during training/in-class support sessions. | • We prepared structured lesson plans, using content from the Jamaican early childhood curriculum, to guide teachers in how to use key strategies during everyday lessons. |
| • Although the ICT led to benefits to the quality of instructional support in early childhood classrooms with an effect size of 0.61 SD, scores remained in the low range and the benefits were not sustained at 1-year follow-up. | Quantitative evaluation of the ICT | • We developed an additional module for the ICT to promote teachers’ use of appropriate instructional support strategies. • This included: (1) how to promote children’s critical thinking skills, (2) how to give affirmative and corrective feedback, and (3) language facilitation skills. These skills were introduced through interactive reading activities and then generalized to everyday teaching and learning activities. |
| Rationale for Adaptation | Source of the Evidence | Adaptations to Content |
|---|---|---|
| • It was challenging for parents and teachers to problem-solve what strategies were most appropriate to deal with different situations and different child behaviors. • Understanding situations from the child’s perspective was also difficult. | Facilitator reflections on training/in-class support sessions. Participant feedback during training/in-class support sessions. | • We designed additional content to ensure participants developed a better understanding of young children’s behavior and the reasons for child “misbehavior.” • We placed greater emphasis on children’s emotions and thoughts when discussing child behaviors. |
| • Self-identifying as an Irie parent or an Irie teacher was a powerful motivator for parents and teachers to adopt the strategies introduced through the programs. | Facilitator reflections on training/in-class support sessions. Qualitative interviews with participants and facilitators | • We included explicit references to being an Irie parent/Irie teacher throughout the program (eg, in discussions, problem-solving activities, practice activities, homework assignments). • Parents/teachers sign an “I am an Irie parent/teacher” oath on completion of the program. |
| • Some parents and teachers viewed corporal punishment as severe physical discipline only. Terms such as “touch,” “brush off” are used to describe corporal punishment that involves slapping. • Similarly, psychological aggression (eg, threatening to hit a child, yelling) is not perceived as violence. | Facilitator reflections on training/in-class support sessions. Participant feedback during training/in-class support sessions. Qualitative interviews with participants and facilitators | • We have included clear definitions of violence against children using common terminology used in Jamaica. • We explicitly link the use of positive discipline, and refraining to use corporal punishment and psychological aggression, as the behavior of an Irie parent/teacher. |
| • Although the frequency of teachers’ use of violence against children reduced by 67% in our effectiveness trial, 72% of teachers continued to use corporal punishment and 55% continued to use psychological aggression. • Teachers reported resorting to violence when they were frustrated by children’s behavior. | Quantitative and qualitative evaluation of intervention trials | • We have designed additional content to increase teachers’ executive function skills and self-regulatory capacities. • This includes strengthening the content around individual goal setting, and training teachers in “calming down” techniques (eg, stop, think and problem solve). |
| • Principal support was an important factor in setting school-wide expectations, promoting teachers’ use of the strategies and promoting the programs with parents. | Facilitator reflections on training/in-class support sessions. Qualitative interviews with participants and facilitators | • We developed four 90-min principal-training modules focusing on how to: (1) support teachers, (2) engage parents in the programs, and (3) create an “Irie school.” |
| • Teachers needed additional guidelines and support for integrating the strategies into everyday teaching and learning activities | Facilitator reflections on training/in-class support sessions. Participant feedback during training/in-class support sessions. | • We prepared structured lesson plans, using content from the Jamaican early childhood curriculum, to guide teachers in how to use key strategies during everyday lessons. |
| • Although the ICT led to benefits to the quality of instructional support in early childhood classrooms with an effect size of 0.61 SD, scores remained in the low range and the benefits were not sustained at 1-year follow-up. | Quantitative evaluation of the ICT | • We developed an additional module for the ICT to promote teachers’ use of appropriate instructional support strategies. • This included: (1) how to promote children’s critical thinking skills, (2) how to give affirmative and corrective feedback, and (3) language facilitation skills. These skills were introduced through interactive reading activities and then generalized to everyday teaching and learning activities. |
ICT, Irie Classroom Toolbox.
The data are also used to inform revisions to program delivery. For example, for the ICT, we found that training needs differed according to the educational level of the teachers. A less intensive intervention led to similar reductions in VAC for fully qualified grade 1 teachers as a more intensive intervention implemented with paraprofessional preschool teachers.14,15
Principle 3: Plan Strategically for Government Agency Program Adoption
Factors influencing program adoption by government agencies include stakeholder relationships, the demand for the innovation, the fit between the innovation and the context, the presence of program champions, and the human and financial resource capacities of the organization, in addition to robust evidence of program effectiveness.36,37
The most important influences for the adoption of the Irie Toolbox programs were: (1) close alignment of the program with the ECC strategic plan; (2) good fit with the ECC organizational structure; (3) availability of all required resources for parents/teachers, facilitators and supervisors; (4) long-standing collaborative relationships between the research and ECC team; and (5) support from external partners (eg, UNICEF Jamaica). To gain government buy-in, we presented videos of the programs in action, feedback from participating parents and teachers, and infographics showing program materials. These resources proved more persuasive than the evidence of effectiveness alone. Framing the programs as promoting caregiver and child competencies, rather than only as violence prevention, was also important.
As the ECC adopted the programs, we prioritized quality implementation over rapid scale-up by beginning implementation on a small scale in each region, thus building capacity that could be leveraged in future rounds of implementation and resolving any initial difficulties before wider rollout.
Principle 4: Ensure High-Quality Initial and Ongoing Training and Regular Supervision for Program Facilitators
Program facilitators require training in new knowledge and skills alongside ongoing supervision to help build these skills in their everyday practice.41 Effective training programs model the collaborative, interactive methods and the focus on positive relationships, and require structured training and supervision guides.42,43
The Irie Toolbox programs involve addressing participants’ skills, cognitions, and emotions, challenging long-held beliefs and social norms relating to VAC, and dealing with resistance. Facilitators often share these beliefs and social norms and hence we provide opportunity for them to experience the program as participants before learning to train others. This also promotes understanding of the content and the rationale for the training techniques used. These include techniques to promote participants’ skills (ie, demonstration, practice, scaffolding), motivation (ie, positive feedback, collaborative problem-solving, goal setting), and opportunities to use the strategies (ie, peer support, provision of resources).44 We use short, regular trainings where possible, rather than a long period of initial training because they prevent cognitive overload, provide timely opportunities for practice, allow for group problem-solving around implementation barriers, and help to build skills over time. Some training techniques are relatively easy for facilitators to learn (ie, demonstration, practice, giving positive feedback, encouraging participation), whereas other techniques (ie, prompting, scaffolding, collaborative problem-solving) develop over time. Hence, we focus initially on the “easier” techniques before focusing on techniques that require advanced facilitation skills. We use structured training and supervision manuals that are valued by government staff, although there is usually an initial adjustment phase before staff are comfortable using a training script.
Supportive field supervision of a program facilitator is used to promote ongoing quality implementation. Frequency of supervision differs according to the educational level of the facilitators with professional staff requiring less supervision than paraprofessionals. In addition, we recommend more frequent supervision during the initial implementation, with reduced supervision after 1 round of implementation. Where possible, we train supervisors as facilitators first and supervise them as they deliver the program. Then, we train in the additional skills required to supervise the program.
Principle 5: Monitor Implementation Quality
High-quality implementation of early childhood interventions predicts increased participant engagement, better caregiver and child outcomes, and is more difficult to sustain as programs scale up.45–48 Maintaining implementation quality requires measurements that are reliable, valid, feasible for use by supervisory staff, and with easy-to-extract data to inform ongoing improvements.
For the Irie Toolbox programs, quality assessments include facilitator records (including participant attendance, session duration, and content checklists) and observational assessments of facilitators’ core competencies by supervisors. Although using independent observers would be a more rigorous method of measuring quality, embedding quality assessments into ongoing supervision is a pragmatic approach when going to scale. In addition to reducing costs, supervisor-completed assessments serve a dual purpose in guiding supervisors to provide appropriate and timely individual support and providing aggregated data to inform wider training needs. We also promote reflective practice by encouraging facilitators to complete self-evaluation forms and discussing these reflections during supervision.
Principle 6: Use Flexible Delivery Modes
Virtual interventions are attractive to policy makers and can promote awareness raising and population-level behavior change49,50 or be used as a supplement to face-to-face programming to promote sustainability.51 However, there is also some evidence of negative effects; thus, careful testing is required before wide-scale implementation.52,53
We adapted the Toolbox programs for virtual delivery because of COVID-19–related school closures. We illustrate the process using the IHT. The virtual IHT consisted of 4 components: (1) weekly, 1-hour, virtual IHT sessions for 10 weeks conducted by ECC officers; (2) 3 SMS messages per week providing information, tips, and encouragement54,55 ; (3) access to a data-free app with weekly uploaded content including 1 to 2 demonstration videos and an Irie Challenge (homework); and (4) weekly session e-summaries sent via WhatsApp. Although the virtual IHT was effective at reducing VAC by parents (Table 2), we identified several implementation challenges. Only 222/557 parents (40%) downloaded the app; attendance at virtual sessions was lower than previously found for face-to-face sessions (46% versus 69% attendance rate); and poor internet connectivity among facilitators and parents was a limiting factor. In addition, the beneficiary parents were more educated and more likely to be employed than parents in previous studies, suggesting that the virtual intervention was less attractive and/or accessible for more disadvantaged parents. However, 442 of 557 parents (79%) attended at least one virtual session and 292 parents (52%) attended 5 or more (out of 10) indicating reasonable take-up of the virtual groups. In addition, 444 of 499 parents (89%) reported reading the SMS and WhatsApp messages. In future studies, SMS and WhatsApp may be effective as a supplement to the Toolbox programs to sustain positive caregiving practices after the end of the initial training.51 Demonstration videos could be disseminated via health centers, schools, and/or other community venues and via social and broadcast media, rather than via an app. This may support efforts to change social norms related to VAC.7 These approaches need to be tested.
Principle 7: Plan for Program Sustainment
Government services adapt to changes in the political landscape, and program sustainment requires ongoing flexibility and adaptation with continued attention to program acceptability, relevance, feasibility, and effectiveness, including a program lead within the government agency.56–58
For the Irie Toolbox programs, implementation is through one government sector (education), and hence planning for sustainment is less complex than for multisectoral programs. Sustainment involves working with the ECC to fully integrate the programs into their routine activities, embedding quality monitoring into routine supervisory visits and inspections, and including Irie Toolbox competencies in operational guidelines. For example, the ECC are integrating the ICT into ongoing teacher training initiatives, the ICT has been aligned with the operational guidelines for early childhood institutions, and we have developed a simple evaluation of teacher practices, adapted from our more complex research measurements,23 to be used by ECC inspectors as part of their routine visits. A key challenge is the multiple and diverse training needs and competing priorities within the ECC that reduce resource availability to sustain specific programs over time. We work closely with the ECC training manager, advising on appropriate adaptations to the delivery mode of the programs to ensure continued fit within ECC structures and integration into ECC activities.
Through applying these principles to scale the Irie Toolbox programs, we aim to promote a consistent approach to reducing violence against children and child behavior problems and increasing caregiver and child competencies, across both home and school contexts, at the population level. In this article, we only provide a descriptive summary of the implementation processes. In future studies, we will use the Reach, Effectiveness, Adoption, Implementation, Maintenance framework to evaluate the implementation of the Irie Toolbox programs at scale.59 This framework includes assessments of individual-level impact and institutional- or setting-level impact and thus measures the extent to which evidence-based interventions can be implemented effectively in real-word settings and lead to improvements in public health. We will integrate the Reach, Effectiveness, Adoption, Implementation, Maintenance framework into a stepped-wedge cluster randomized trial design that is appropriate for service delivery interventions delivered at the level of the cluster (in this case, preschools).60 We will embed factorial experiments within the larger trial to test implementation strategies to identify the optimal level and mode of support required for program facilitators. We will also evaluate the effectiveness of flexible delivery modes for program participants including blended delivery models (combining face-to-face with virtual delivery), and tiered interventions with different intensity interventions delivered according to participants’ needs. Identifying successful models for scaling up early childhood, violence prevention programs in LMIC is a critical component in reducing the large public health, societal, and economic burden associated with violence.
Dr Baker-Henningham and Mss Bowers and Francis conceptualized and drafted the initial manuscript and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
This article describes how we applied implementation science principles in the design, implementation, evaluation, and scale-up of 2 early childhood, caregiver training, violence prevention interventions in Jamaica.
FUNDING: This work was funded by MRC/Wellcome Trust/NIHR/UK Aid, Grand Challenges Canada, Early Learning Partnership, World Bank, and UNICEF Jamaica. The funders had no role in the writing of the manuscript or the decision to submit for publication.
CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no potential conflicts of interest to disclose.
Comments