Adolescent Mental Health Program Components and Behavior Risk Reduction: A Meta-analysis

A protocol for this systematic review was agreed with the World Health Organization as the version of record (see Supplemental Information). We present findings relating to universal interventions only in this article (programs that are targeted at the whole adolescent population and are designed to benefit everyone, not only specific at-risk groups).

We included (1) randomized controlled trials (RCTs) of psychosocial interventions (2) with adolescent participants between the ages of 10 to 19 (3) in which trial interventions had the primary or secondary aims of promoting mental health or preventing mental disorders, reducing risk behaviors, or reducing self-harm and suicide; additionally, (4) the programs were aimed at the whole adolescent population and were designed to benefit everyone regardless of setting or delivery and (5) published between January 2000 and February 2018 in any language. Studies in which authors compared outcomes between groups who received an intervention and those who received usual or no care and/or those who received a different intervention were included. We included studies if the mean age was between 10 and 19 years or >50% of the participants were between 10 and 19 years old. Outcomes included positive mental health (mental well-being, resilience, coping, emotional regulation), depressive and anxious symptomatology, violence perpetration and bullying, and alcohol and other substance use. We included different time points and coded outcomes according to short (<2 months after intervention completion), medium (2–6 months), and long-term (>6 months).

We searched Medline, PsycINFO, Scopus, Embase, and Applied Social Sciences Index Abstracts and followed references of reviews. Duplicate abstracts were removed, and the remaining abstracts were assessed against inclusion criteria by 2 independent reviewers. Any disagreements were resolved by discussion between the 2 reviewers or resolved by the arbitration of a third reviewer. Subsequently, full-text reports were accessed and assessed. Pairs of reviewers working independently completed this screening process. Data were extracted by using a standardized form and included trial characteristics, setting, sampling, population characteristics, intervention details, outcome measures, study quality (assessed by using the Cochrane risk-of-bias tool), and treatment effects. In addition, each intervention was coded according to the presence of specific practice components. Details were gathered directly from the study publications and directly from intervention manuals when available. We relied on authors’ explicit description of components whenever possible; for example, the presence of “stress management” would not be inferred from a coping skills intervention unless the authors discussed stress specifically. In many cases, authors expounded on program elements in tables or figures. Program content components were coded according to a system based on the work of Boustani et al¹⁶ in which the PracticeWise Clinical Coding System¹⁷ was used to identify common practices across a range of prevention programs. We also added other program components relating to theoretically relevant methods.¹⁸ Finally, on the basis of the PracticeWise recommendations and as implemented by Brown et al¹⁹ in a similar activity, we recorded other frequently occurring components as free text and ultimately integrated them as new codes into the framework (Table 1). Some of these included decision-making,²⁰ conflict resolution,²¹ mindfulness,^22,23 and alcohol and drug education.²⁴ 

For reporting and analysis, we categorized all universally delivered programs into face-to-face, digital, or combined modality interventions. Face-to-face interventions consisted of all interventions delivered in schools, communities, or health centers; digital and combined modality interventions consisted of interventions that were solely digitally delivered content or digital content in combination with other modes of delivery.

Effect estimates from included studies were converted to standardized mean differences by using available published formulas.²⁶ A common problem in meta-analyses of complex interventions is that study authors report multiple effect estimates from the same domain (ie, conceptually exchangeable and thus equally valid) in respect to an outcome and often report outcomes from multiple time points. To address this, we used a robust variance estimation meta-analysis model²⁷ to include all relevant information from included studies. We estimated all models using random effects, given high anticipated levels of statistical heterogeneity and an intercorrelation parameter of 0.8, which is standard, to estimate how closely effect estimates within a study are related. Given the number and diversity of components we sought to analyze, we estimated a series of single-predictor meta-regression models. Predictors were entered into models as the study-level mean of a component. In standard 2-arm trials and trials in which components were binary, this variable took on the value of 1 or 0. In multiarm trials in which the ≥2 active arms differed as to the presence of a component, the variable took on the value of the proportion of effect estimates with a specific component. We estimated all models first with effect estimates corresponding to ≤2 months of follow-up and then with effect estimates over all follow-up times. We noted when models could not provide usable evidence because of model instability. We did not formally test publication bias given that these tests are not understood in the context of robust variance estimation meta-analysis. In assessing differences in effect sizes, we used standard thresholds of 0.2 for small effect size, 0.5 for medium effect size, and 0.8 for large effect size.²⁸ 

We identified 14 600 records through database searches and hand searching, of which 158 were suitable and reported data suitable for components analysis (Fig 1).

The characteristics of studies that met the inclusion criteria and contributed data to the components analysis are summarized in Table 2, and all included studies are listed in Supplemental Information 2. The average intervention duration was 13.88 hours for face-to-face interventions and 6.05 hours for digital interventions (see Supplemental Information 4 and 5). The average number of components per intervention was 5.4 for face-to-face interventions and 5.9 for digital interventions (further details in Supplemental Information 4 and 5).

In general, risk of bias was low across most categories, with the exception of allocation concealment and random sequence generation (see Fig 2 and Supplemental Information 3 for full details). In the majority of studies, it was unclear who had been responsible for randomization as well as how the randomization sequence was generated. It was also unclear if this sequence was protected sufficiently to prevent the research team from predicting the next treatment allocation during the process. Furthermore, in many studies, blinding of participants and outcome assessment was not possible because of the study design, particularly in school-based settings in which whole schools or specific classes were allocated to the intervention status. Outcome data assessment largely presented a low risk of bias, but approximately one-third of studies had unclear risk of attrition or other biases. Almost 90% of studies had a low risk of bias for selective reporting.

The overall effect sizes of universally delivered interventions on each of the study outcomes are reported in Table 3. Self-harm and suicide were not included in the analyses reported here because there were only 2 universally delivered interventions located in which these outcomes were reported. For face-to-face interventions in the short-term, there were significant differences between intervention and control groups for positive mental health and depression and anxiety symptoms. Across all time points, there were significant differences between intervention and control groups for positive mental health, depression and anxiety symptoms, and violence, aggression, and bullying. For digital or combined modality interventions, only depression and anxiety outcomes improved in the short-term, but this was not evident across all time points. For all time points, there were significant differences between intervention and control for positive mental health and substance use. All overall significant effect sizes were small to moderate and indicated beneficial effects of interventions. There were no differences for face-to-face interventions for substance use at any time point or for violence, aggression, and bullying in the short-term. There were no differences for digital and combined modality interventions for short-term positive mental health and substance use outcomes, depression and anxiety beyond the short-term, or for aggression, violence, and bullying across any time point.

Seven intervention components predicted only positive effects, that is, their presence was associated with more successful programs (see Tables 4 and 5). These were interpersonal skills, emotional regulation, alcohol and drug education, mindfulness, problem solving, assertiveness training, and stress management. The presence of interpersonal skills was most consistently associated with larger effect sizes, yielding improved effects for positive mental health, depression and anxiety prevention, and prevention of substance use. Emotional regulation was associated with greater effectiveness in improving positive mental health and greater reductions in depressive and anxious symptomatology. Alcohol and drug education predicted positive outcomes for non–alcohol- and non–drug-related outcomes, namely, positive mental health in face-to-face interventions and aggression in digital interventions. The remaining components were associated with larger effect sizes in 1 outcome category only. Mindfulness was associated with a decrease in anxiety and depression symptoms in face-to-face interventions. Problem solving was associated with a decrease in depression and anxiety symptoms in digital and combined interventions. Assertiveness and stress management predicted larger effect sizes for the prevention of substance use in digital interventions. See Supplemental Information 2 for full details of the presence of program components in interventions.

Six practice components revealed mixed results across the different outcomes; these components were conflict resolution, coping skills, goal setting, relaxation, skills to resist peer pressure, and self-efficacy training. In face-to-face interventions, conflict resolution predicted larger effects for substance use but smaller effects for depression and anxiety symptoms. Coping skills content did not predict any outcomes for face-to-face interventions but predicted diminished effectiveness for positive mental health and stronger effectiveness for substance use for digital interventions. Goal setting was predictive of smaller program effects for depression and anxiety in face-to-face interventions but larger effects for digital substance use interventions. Relaxation was associated with smaller effect sizes for digital positive mental health outcomes but stronger effectiveness for substance use. Skills to resist peer pressure predicted larger effects for violence outcome in digital interventions but smaller effects for aggression and positive mental health outcomes in face-to-face interventions. For depression and anxiety symptoms, self-efficacy predicted smaller effect sizes for face-to-face interventions but larger effects for digital interventions.

Across all meta-regressions, 6 components were associated with either attenuated effectiveness or minimal difference in effectiveness, depending on the outcome; these components were activity monitoring and scheduling, anger management, civic responsibility, communication skills, decision-making, and insight building. Communication skills and activity monitoring and scheduling were associated with smaller effect sizes for depression and anxiety outcomes in face-to-face interventions. Digital and combined interventions that included civic responsibility were less effective at reducing depressive and anxious symptomatology. In face-to-face interventions, the inclusion of decision-making activities was associated with smaller effect sizes on positive mental health and depressive and anxious symptomatology. Insight building predicted a smaller effect size for positive mental health when included in digital and combined interventions.

A final set of practice components that did not have a clear relationship to effectiveness in either direction included cognitive restructuring, mental health literacy, self-monitoring, social skills, support networking, and behavioral activation.

This is the first global review of active components present in interventions that are aimed to improve adolescent health across a range of interrelated mental health outcomes. With the results of this review, we indicate, first, that universally delivered interventions can improve adolescent mental health and reduce risk behavior and, second, that there are several content-related program components that are associated with larger or smaller effect sizes. Of these components, however, only 3 predicted positive effects across multiple outcomes: interpersonal skills training, emotional regulation, and alcohol and drug education. This finding reflects those in a review by Singla et al²⁵ which also found that interpersonal and emotional elements had the strongest associations with overall effectiveness across mental health interventions delivered by lay health workers in low- and middle-income countries (LMICs).

Developing skills to improve interpersonal relationships is highly relevant for improving adolescent mental health outcomes, and our findings indicate that including these skills in multioutcome interventions designed to promote mental health and prevent mental disorders and risk behaviors is a valuable strategy. Previous research has revealed that poor-quality relationships consistently predict poor mental health outcomes for adolescents,^29,30 whereas positive relationships are associated with better mental health outcomes.³¹ In this review, intervention content commonly included verbal and nonverbal communication skills¹⁶ and was often combined with broader social skills training focusing on how an individual engages in a social setting or larger group.^32,–34 

Activities used to develop emotional regulation skills were common in interventions that were aimed to reduce depression and promote positive mental health, as well as those that were aimed to reduce aggression. Intervention programs that included emotional regulation encompassed whole-class interventions, cognitive behavioral interventions,³⁵ antibullying interventions,³⁶ and guided expressive writing interventions,³⁷ as well as more broadly focused, integrated interventions.³⁸ Many yoga and mindfulness-based interventions also employed an emotional regulation component, as practitioners guided adolescents through meditative sessions in which observing as well as engaging with emotions was encouraged.^23,39 

Alcohol and drug education predicted larger effect sizes for mental health promotion and interventions addressing violence. This term covered a broad range of topics, including facts about alcohol, cannabis, and other illicit drugs, discussion about the risks of using illegal substances, social influences associated with alcohol use,^32,40,–42 media influences and pressures to use substances,^43,–45 and parent education about engaging their children in conversations about alcohol.^24,46,47 Certain interventions also took a harm minimization approach, teaching adolescents about less harmful ways to use alcohol or ways to reduce risk for themselves or others.^48,–50 Delivery methods also differed. For example, in 1 digital intervention, participants are walked through the consequences of a virtual night of binge drinking.⁵¹ However, the reasons for the effect of alcohol and drug use on outcomes beyond substance use are unknown. It may be because of shared risk and protective factors between these outcomes and shared pathways to effective prevention between different types of outcomes that have been “triggered” by teaching adolescents drug use prevention content.

For “Helping Adolescents Thrive,” it is evident that intervention content that is strongly centered on interpersonal and emotional skills is most likely to be effective across multiple outcome domains. It is also possible that the active components identified in this review, particularly interpersonal skills and emotional regulation, may have effects that extend beyond our defined scope of mental health outcomes to broader mental health domains. For example, in other research, improvements in emotional regulation have been shown to reduce risky sexual behavior during adolescence^52,–54 because these skills may help adolescents develop stronger and more equitable relationships.⁵² 

Given that the evidence base is almost entirely from high-income countries (HICs), it will be essential to track implementation efforts if and when these interventions are adapted for use in LMICs to ensure that they are implemented in a culturally and contextually valid and appropriate manner.⁷ Specifically, developing an intervention package on the basis of these findings will require active engagement with adolescents, particularly in low-resource settings, to translate relevant evidence-based principles into feasible and acceptable intervention programs that appeal to and effectively engage adolescents. Pursuing a user-centered design approach by employing multiple stages of engagement and prototyping with adolescents, their parents, their teachers, and other community stakeholders to coproduce the intervention package⁵⁵ will significantly strengthen the development of the program and its adaptability to different settings.

Again, although this was a global review, the publications eligible for inclusion were overwhelmingly based in HICs. In studies from LMICs, adapted versions of evidence-based interventions from HICs are often used, which may affect the validity and reliability of their results. In addition, studies with randomized designs are more likely to be used to evaluate research program interventions, whereas quasi-experimental and other designs are often used for real-life interventions, meaning that restricting our screening to RCTs only may have limited the applicability of these findings to nonresearch settings.⁵⁶ 

The program components approach depends on the quality of reporting in publications. Brown et al¹⁹ note that essential details required to understand content and implementation are often missing from these publications. In the review by Singla et al,²⁵ the authors further noted the lack of reporting about dosage for each component present. It was not always possible to determine program fidelity or mean dosage across participants, limiting the strength of their analyses.⁵⁷ In this review, few study authors reported intervention components in enough detail to allow for replication; even fewer provided any form of guidance as to how interventions could be scaled up. Finally, a further limitation to be considered is the risk of bias in the included studies. Although considered to be low across most categories, allocation concealment and random sequence generation were high or unclear for the majority of studies, whereas in some cases, the nature of feasible study designs for universally delivered interventions (such as in schools) precluded blinding of participants and outcome assessment. Furthermore, the quality of the body of evidence was not assessed by using the Grading of Recommendations, Assessment, Development and Evaluation tool.

These are novel results that will be used to design a universally delivered intervention as a part of the “Helping Adolescents Thrive” initiative. Further work should be undertaken to develop and test interventions that use these core components, especially in underresourced settings in which multiple risk factors for poor adolescent health are present.

HIC

high-income country

LMIC

low- and middle-income country

RCT

randomized controlled trial