Video Abstract
Most youths who die by suicide have interfaced with a medical system in the year preceding their death, placing outpatient medical settings on the front lines for identification, assessment, and intervention.
Review and consolidate the available literature on suicide risk screening and brief intervention with youths in outpatient medical settings and examine common outcomes.
The literature search looked at PubMed, OVID, CINAHL, ERIC, and PsychInfo databases.
Interventions delivered in outpatient medical settings assessing and mitigating suicide risk for youths (ages 10–24). Designs included randomized controlled trials, prospective and retrospective cohort studies, and case studies.
Authors extracted data on rates of referral to behavioral health services, initiation/adjustment of medication, follow-up in setting of assessment, suicidal ideation at follow-up, and suicide attempts and/or crisis services visited within 1 year of initial assessment.
There was no significant difference in subsequent suicide attempts between intervention and control groups. Analysis on subsequent crisis service could not be performed due to lack of qualifying data. Key secondary findings were decreased immediate psychiatric hospitalizations and increased mental health service use, along with mild improvement in subsequent depressive symptoms.
The review was limited by the small number of studies meeting inclusion criteria, as well as a heterogeneity of study designs and risk of bias across studies.
Brief suicide interventions for youth in outpatient medical settings can increase identification of risk, increase access to behavioral health services, and for crisis interventions, can limit psychiatric hospitalizations.
Suicide is a leading cause of death in youth across the United States. A recent survey revealed that 18.8% of students nationwide had seriously considered attempting suicide, 15.7% of students had made a plan about how they would attempt suicide, and 8.9% of students had attempted suicide more than once.1
Approximately 80% of youths who die by suicide interfaced with the medical system in the year preceding their death,2 but only 20% had contact with a mental health professional.3 Therefore, outpatient medical settings are well-positioned to identify, assess, and intervene with at-risk youth. Due in part to the increase in suicide rates, the American Academy of Pediatrics, the American Academy of Child and Adolescent Psychiatry, and the Children’s Hospital Association declared a national emergency in child and adolescent mental health.4
Assessing interventions for suicide prevention, screening, and brief intervention in medical settings is challenging. Suicide is a relatively rare occurrence, and studies require large sample sizes to reveal the impact. In addition, the duration of prevention studies is often too brief to collect sufficient data on the low-frequency end points of death by suicide or suicide attempts. Consequently, the authors of many studies have focused on proximal outcomes. Moreover, testing interventions for people at risk for suicide presents unique ethical dilemmas, particularly in randomized controlled trials (RCTs). Alternative designs, such as case-control and risk-based allocation studies, have been employed to avoid such issues.5
In this article, we cover the literature on interventions in outpatient medical settings that include either suicide screening or suicide risk assessment followed by a brief intervention with youths aged 10 to 24. We examine the outcomes of these interventions with the goal of providing recommendations for future research and clinical practice guidelines.
Methods
This systematic review is based on the updated Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines.6 The protocol is registered in the International Prospective Register of Systematic Reviews under protocol ID CRD42021277451.
Inclusion Criteria
The following designs were included: RCTs, prospective and retrospective cohort studies, and case studies reported in English. We excluded cross-sectional studies. We included studies in which most participants fell within the age range of 10 to 24 years. Eligible interventions included suicide risk assessment and at least 1 intervention step to reduce suicide risk by the clinical or research team occurring within 2 weeks of the point of care. Settings included outpatient care, such as general pediatrics, family practice, and school-based health centers. Many patients access urgent or emergency care for routine medical needs; thus, interventions in these settings were included. We excluded interventions delivered during hospital admission, in nonmedical secondary school settings, or studies that were set solely in the community.
Outcomes of Interest
The primary outcomes of interest included the number and rate of subsequent suicide attempts and of emergency department (ED)/crisis center visits for suicidal ideation (SI) within 1 year after the initial intervention. The secondary outcomes of interest included rates of referral to behavioral health services, behavioral health service use within 1 year after initial intervention, follow-up of suicide risk identification, and persistent SI within 18 months of identification and intervention.
Information Sources
The literature search, completed in November 2020, covered the PubMed, OVID, Cumulative Index of Nursing and Allied Health Literature, Education Resources Information Center, and PsychInfo databases, not limited to any range of years. Citations and PDFs of the articles were organized in EndNote and deduplicated. Article authors were contacted for missing or incomplete data.
Search Strategies
The search strategy for PubMed was as follows (it was minimally modified in the other databases to account for their specific search functions): ((primary care” OR “primary health care” OR “primary care nursing” OR “Primary Health Care”[Mesh] OR “Primary Care Nursing”[Mesh] OR “Physicians, Primary Care”[Mesh])) OR (“physician assistant” OR “nurse practitioner” OR “school nurse” OR “Physician Assistants”[Mesh] OR “Nurse Practitioners”[Mesh] OR “Pediatric Nurse Practitioners”[Mesh] OR “social worker” OR “Social Workers”[Mesh] OR “school nurse” OR “school health services “OR “School Nursing”[Mesh] OR “School Health Services”[Mesh] OR “School Mental Health Services”[Mesh] OR “emergency department” OR “emergency room” OR “emergency service” OR “emergency nursing” OR “urgent care OR “retail care” OR “emergency department” OR “emergency room” OR “emergency service” OR “emergency nursing” OR “urgent care” OR “retail care” OR “Emergency Service, Hospital”[Mesh] OR “Emergency Nursing”[Mesh] OR “retail clinic”) AND (“prevention and control” [Subheading] OR “Primary Prevention”[Mesh] OR prevention OR screening OR assessment OR intervention)) AND (suicide OR “attempted suicide” OR suicidality OR “suicidal ideation” OR “suicidal ideation[Mesh] OR “suicide, attempted”[Mesh]) Filters: Child: 6–12 years, Adolescent: 13–18 years, Young Adult: 19–24 years.
Article Selection
Each article was assigned 2 reviewers who examined the title and abstract to determine if it should be excluded on the basis of setting, population, or absence of an intervention. The reviewers then reconciled their determination and categorized each article for exclusion or closer review.
Each article included for further review was assigned a team of 2 reviewers who read the article and recorded information on the authors, title, year, setting, population, study design, description of intervention, primary outcomes, conclusion, limitations or gaps, and notes. On the basis of this information, each reviewer suggested that an article be included or excluded. Information recorded during this step was not shared until the completion of all reviews by that team. The reviewer pairs then met to calibrate their information and impressions. If a consensus was not reached by the pair, the article was brought to the larger author group for further review.
Data Collection Process
Data extraction forms were created and organized in a Microsoft Word template. All reviewers trained on and piloted the extraction forms before assignment to reduce errors and refine the form. Each article was assigned 2 reviewers for data extraction. After the data were extracted, reviewer pairs met to calibrate the data and discuss discrepancies. Discrepancies that could not be resolved were brought to the full author group for further discussion.
Data Items
Data extracted from the point of intervention included the identification rates of patients at risk for suicide. Outcome data included the rates of referral to behavioral health, rates of initiation or adjustment of psychotropic medication, rates of behavioral health service use after identification, follow-up rates in the setting of initial risk assessment, SI at follow-up, change in SI at follow-up, suicide attempt within 1 year after risk assessment, and ED or crisis services visited for SI within 18 months after risk assessment.
Risk of Bias in Individual Studies
The risk of bias for RCT studies was assessed by 2 reviewers by using the Cochrane Collaboration tool for assessing risk of bias for RCTs (RoB 2). After assessment, the 5 domains were categorized as “low risk,” “some concerns,” or “high risk” for each outcome. Nonrandomized studies were evaluated by using the Risk of Bias in Nonrandomized Studies of Interventions Cochrane Collaboration tool and categorized as having a “low,” “moderate,” “serious,” or “critical” risk of bias along the 7 domains. An overall risk of bias for each outcome was then determined using the RoB 2 and Risk of Bias in Nonrandomized Studies of Interventions tool criteria. Disagreements in any of the above were resolved through consultation with the author group.
Summary Measures
We determined that it was feasible to conduct meta-analyses on 5 outcomes, and they are presented using forest plots and a brief narrative summary for each outcome of interest. Given the low number of qualifying studies, all studies were included, regardless of the risk of bias. The outcomes that did not qualify for a meta-analysis because the missing data were summarized by using a narrative synthesis.
Statistical Model
Computations were conducted by using Comprehensive Meta-Analysis Version 4.7 We employed pre–post analysis whenever the relevant data were available, and only post scores when the authors did not report baseline or change scores.8 When several follow-up periods were reported (eg, both 2 and 6 months), we used the longest period.
The random effects model was employed for all the analyses because the diversity in population and interventions in the studies included in the analysis precludes the assumption of a common effect, which underlies the fixed effect model.7,9,–11 As a measure of effect size, we used the odds ratio (OR) when study authors reported proportions (eg, of immediate psychiatric hospitalization), and Hedges’ g when study authors reported continuous scores (eg, of depression scales). τ-squared was used to estimate the variance of the true effect sizes in each analysis, and the Q-statistic was used to test the null hypothesis that all studies in each analysis shared a common effect size. We should note, however, that this test is underpowered when the number of studies in each analysis is small.12
Results
Articles
A total of 575 articles were screened and 14 were included (Fig 1): 6 RCTs, 1 quasi-experimental, 4 retrospective, and 3 single-arm studies (Table 1). One RCT was analyzed as a prospective cohort trial for the outcome of interest.13 Results from the 3 single-arm studies are displayed in Table 2.
Preferred Reporting Items for Systematic Reviews and Meta-Analysis flow diagram.
Preferred Reporting Items for Systematic Reviews and Meta-Analysis flow diagram.
Study Characteristics
| Author, y | Sample Size | Age Range | Sex | Race/Ethnicity | Clinical Setting | Intervention | Intervention Category | Study Design |
|---|---|---|---|---|---|---|---|---|
| Asarnow et al, 2011 | 181 | 10–18 | Female: 66% | White, NH: 35% | ED | Youth and family crisis therapy session, follow-up phone contact | CIS | RCT |
| African American: 14% | ||||||||
| Hispanic: 42% | ||||||||
| Other: 10% | ||||||||
| Chisolm et al, 2009 | 3960 | 11–20 | Female: 53.8% | White 36.6% | Primary Care | Screening, provider feedback report | SAR | RCT for entire study; prospective cohort for outcome of interest |
| Male: 46.2% | Black 55.9% | |||||||
| Hispanic 3.8% | ||||||||
| Other 3.6% | ||||||||
| Etter et al, 2018 | 2134 | 12–20 | Female: 51.1% | Black: 60.3% | Primary Care | Screening, automated provider worksheet, enhanced EHR | SAR | Single-arm |
| Male: 48.9% | Hispanic: 14.3% | |||||||
| Other: 17.5% | ||||||||
| White: 7.9% | ||||||||
| Gardner et al, 2010 | 1547 | 11–20 | Female: 58% | Black: 57% | Primary Care | Screening, feedback, risk triage, referral to behavioral health | SAR | Single-arm |
| Male: 42% | Other: 9% | |||||||
| White: 34% | ||||||||
| Grupp-Phelan et al, 2019 | 159 | 12–17 | Female: 79.2% | White: 50.3% | ED | MI-based family therapy session, behavioral health referral, case management | CIS | RCT with parallel design |
| Grupp-Phelan et al, 2012 | 24 | 12–17 | Female: 79.2% | White: 29.2% | ED | Screening, brief evaluation, assessment, discussion, and referral for behavioral health appointment | CIS | RCT |
| Male: 20.8% | African American: 50.0% | |||||||
| Other: 20.8% | ||||||||
| King et al, 2015 | 46 | 14–19 | Female: 80% | African American: 57% | ED | Personalized feedback, MI-based session, phone check-in | CIS | RCT with parallel design |
| Caucasian: 39% | ||||||||
| American Indian or Alaska Native: 4% | ||||||||
| Native Hawaiian/Pacific Islander: 2% | ||||||||
| Hispanic: 2% | ||||||||
| Other: 2%a | ||||||||
| Parker et al, 2003 | 682 | <19 | Female: 51% | Not listed | ED/Community | Physician training, behavioral health referral | CIS | Retrospective |
| Male: 49% | ||||||||
| Patel et al, 2018 | 4868 | 12–19 | Female: 55.6% | White: 68.5% | Urgent Care | SAR to behavioral health | SAR | Single-arm |
| Male: 44.4% | Black:12.8% | |||||||
| Hispanic: 8.9% | ||||||||
| Multiracial: 4.2% | ||||||||
| Asian: 1.8% | ||||||||
| Unknown: 1% | ||||||||
| Other: 1.9% | ||||||||
| Rotheram- Borus et al, 2000 | 140 | 12–18 | Female: 100% | Latino: 89.2% (experimental) | ED | Staff or provider training, family-based crisis intervention, referral for behavioral health follow-up | CIS | Quasi-experimental |
| Latino: 85.3% (control) | ||||||||
| Wharff et al, 2012 | 250 | 13–18 | Female: 76% | White: 65% | ED | Family-based, cognitive behavioral crisis intervention session | CIS | Retrospective |
| Male: 24% | Black: 17% | |||||||
| Hispanic/Latino: 10% | ||||||||
| Biracial: 2% | ||||||||
| Asian: 3% | ||||||||
| Another race: 4% | ||||||||
| Wharff et al, 2019 | 139 | 13–18 | Female: 72% | Black: 6% | ED | Family-based, cognitive behavioral crisis intervention session | CIS | RCT with parallel design |
| Male: 28% | White: 66% | |||||||
| Latino: 9% | ||||||||
| Asian: 3% | ||||||||
| Mixed: 18% | ||||||||
| Wintersteen et al, 2010 | 2976 | 12–17.9 | Female: 58%, 52%, 48% | Black: 77%, 77%, 99% | Adolescent medicine and primary care | Staff or provider training, SAR to behavioral health | SAR | Retrospective |
| White: 16%, 19%, 0% | ||||||||
| Hispanic: 2%, 1%, 0% | ||||||||
| Other: 5%, 3%, 0% | ||||||||
| Wintersteen et al, 2013 | 56 352 | 12–17.9 | Female: 51.5% | African American: 71.6% | ED | Screening, provider and staff training, evaluation | SAR | Retrospective |
| Male: 48.5% | Caucasian: 22.8% | |||||||
| Other: 5.6% |
| Author, y | Sample Size | Age Range | Sex | Race/Ethnicity | Clinical Setting | Intervention | Intervention Category | Study Design |
|---|---|---|---|---|---|---|---|---|
| Asarnow et al, 2011 | 181 | 10–18 | Female: 66% | White, NH: 35% | ED | Youth and family crisis therapy session, follow-up phone contact | CIS | RCT |
| African American: 14% | ||||||||
| Hispanic: 42% | ||||||||
| Other: 10% | ||||||||
| Chisolm et al, 2009 | 3960 | 11–20 | Female: 53.8% | White 36.6% | Primary Care | Screening, provider feedback report | SAR | RCT for entire study; prospective cohort for outcome of interest |
| Male: 46.2% | Black 55.9% | |||||||
| Hispanic 3.8% | ||||||||
| Other 3.6% | ||||||||
| Etter et al, 2018 | 2134 | 12–20 | Female: 51.1% | Black: 60.3% | Primary Care | Screening, automated provider worksheet, enhanced EHR | SAR | Single-arm |
| Male: 48.9% | Hispanic: 14.3% | |||||||
| Other: 17.5% | ||||||||
| White: 7.9% | ||||||||
| Gardner et al, 2010 | 1547 | 11–20 | Female: 58% | Black: 57% | Primary Care | Screening, feedback, risk triage, referral to behavioral health | SAR | Single-arm |
| Male: 42% | Other: 9% | |||||||
| White: 34% | ||||||||
| Grupp-Phelan et al, 2019 | 159 | 12–17 | Female: 79.2% | White: 50.3% | ED | MI-based family therapy session, behavioral health referral, case management | CIS | RCT with parallel design |
| Grupp-Phelan et al, 2012 | 24 | 12–17 | Female: 79.2% | White: 29.2% | ED | Screening, brief evaluation, assessment, discussion, and referral for behavioral health appointment | CIS | RCT |
| Male: 20.8% | African American: 50.0% | |||||||
| Other: 20.8% | ||||||||
| King et al, 2015 | 46 | 14–19 | Female: 80% | African American: 57% | ED | Personalized feedback, MI-based session, phone check-in | CIS | RCT with parallel design |
| Caucasian: 39% | ||||||||
| American Indian or Alaska Native: 4% | ||||||||
| Native Hawaiian/Pacific Islander: 2% | ||||||||
| Hispanic: 2% | ||||||||
| Other: 2%a | ||||||||
| Parker et al, 2003 | 682 | <19 | Female: 51% | Not listed | ED/Community | Physician training, behavioral health referral | CIS | Retrospective |
| Male: 49% | ||||||||
| Patel et al, 2018 | 4868 | 12–19 | Female: 55.6% | White: 68.5% | Urgent Care | SAR to behavioral health | SAR | Single-arm |
| Male: 44.4% | Black:12.8% | |||||||
| Hispanic: 8.9% | ||||||||
| Multiracial: 4.2% | ||||||||
| Asian: 1.8% | ||||||||
| Unknown: 1% | ||||||||
| Other: 1.9% | ||||||||
| Rotheram- Borus et al, 2000 | 140 | 12–18 | Female: 100% | Latino: 89.2% (experimental) | ED | Staff or provider training, family-based crisis intervention, referral for behavioral health follow-up | CIS | Quasi-experimental |
| Latino: 85.3% (control) | ||||||||
| Wharff et al, 2012 | 250 | 13–18 | Female: 76% | White: 65% | ED | Family-based, cognitive behavioral crisis intervention session | CIS | Retrospective |
| Male: 24% | Black: 17% | |||||||
| Hispanic/Latino: 10% | ||||||||
| Biracial: 2% | ||||||||
| Asian: 3% | ||||||||
| Another race: 4% | ||||||||
| Wharff et al, 2019 | 139 | 13–18 | Female: 72% | Black: 6% | ED | Family-based, cognitive behavioral crisis intervention session | CIS | RCT with parallel design |
| Male: 28% | White: 66% | |||||||
| Latino: 9% | ||||||||
| Asian: 3% | ||||||||
| Mixed: 18% | ||||||||
| Wintersteen et al, 2010 | 2976 | 12–17.9 | Female: 58%, 52%, 48% | Black: 77%, 77%, 99% | Adolescent medicine and primary care | Staff or provider training, SAR to behavioral health | SAR | Retrospective |
| White: 16%, 19%, 0% | ||||||||
| Hispanic: 2%, 1%, 0% | ||||||||
| Other: 5%, 3%, 0% | ||||||||
| Wintersteen et al, 2013 | 56 352 | 12–17.9 | Female: 51.5% | African American: 71.6% | ED | Screening, provider and staff training, evaluation | SAR | Retrospective |
| Male: 48.5% | Caucasian: 22.8% | |||||||
| Other: 5.6% |
EHR, electronic health record; MI, motivational interviewing; NH, non-Hispanic.
Some identified as biracial.
Single-Arm Study Findings
| Outcome | Study | Findings |
|---|---|---|
| Primary outcomes | ||
| Suicidal ideation at follow-up | Gardner et al, 2010 | 3.4% of patients seen for suicidality after receiving intervention |
| Secondary outcomes | ||
| Suicide risk identification | Etter et al, 2018 | 6% (131) of screened patients |
| Suicide risk identification | Patel et al, 2018 | 2.4% (119) of screened patients |
| Immediate psychiatric hospitalization | Etter et al, 2018 | 7.5% of patients screening positive |
| Immediate psychiatric hospitalization | Patel et al, 2018 | 15% of patients with a positive risk assessment on C-SSRS |
| Behavioral health referrals | Gardner et al, 2010 | 74% of patients screening positive |
| Behavioral health referrals | Patel et al, 2018 | 82% of patients with positive screens |
| Behavioral/mental health care service use | Gardner et al, 2010 | 65% of patients referred internally received a service |
| Outcome | Study | Findings |
|---|---|---|
| Primary outcomes | ||
| Suicidal ideation at follow-up | Gardner et al, 2010 | 3.4% of patients seen for suicidality after receiving intervention |
| Secondary outcomes | ||
| Suicide risk identification | Etter et al, 2018 | 6% (131) of screened patients |
| Suicide risk identification | Patel et al, 2018 | 2.4% (119) of screened patients |
| Immediate psychiatric hospitalization | Etter et al, 2018 | 7.5% of patients screening positive |
| Immediate psychiatric hospitalization | Patel et al, 2018 | 15% of patients with a positive risk assessment on C-SSRS |
| Behavioral health referrals | Gardner et al, 2010 | 74% of patients screening positive |
| Behavioral health referrals | Patel et al, 2018 | 82% of patients with positive screens |
| Behavioral/mental health care service use | Gardner et al, 2010 | 65% of patients referred internally received a service |
C-SSRS, Columbia-Suicide Severity Rating Scale.
Participants
There was a total of 73 458 participants. Sample sizes ranged from 24 to 56 352. Ages ranged from 10 to 20 years. Five studies included participants who presented with suicide attempts or ideation,8,14,–17 4 studies included participants who screened positive for SI,18,–21 and 5 studies did not have suicide-related inclusion criteria.13,22,–25
Settings
Interventions
Six studies were classified as screening, assessment, and referral (SAR) interventions if they only included suicide screening, patient assessment, or referral to behavioral health. Eight were classified as crisis intervention session (CIS) interventions if the intervention included additional components, such as family-based interventions,8,16,17,20 motivational interviewing,20,21 follow-up,16,18,21 staff or provider training,8 enhanced behavioral health interventions (behavioral health service beyond usual care),8,15 and consultation.24
Risk of Bias
Outcomes in RCTs had varying characteristics contributing to the risk of bias.20 Three studies had outcomes with an overall low risk of bias, 3 had some concerns about the risk of bias, and 1 had an overall high risk of bias because of how the outcome of interest was measured (Fig 2, Supplemental Table 3).
The overall risk of bias for outcomes in nonrandomized controlled (NRC) studies was typically determined to be serious, most commonly as a result of the outcome measurement methods (Fig 3, Supplemental Tables 3–7). One study also had serious bias due to deviations from the intended interventions. The outcomes for 1 NRC study were deemed to be at an overall moderate risk of bias and only 1 had outcomes deemed to be at an overall low risk of bias. Three single-arm studies could not be assessed by using the ROBINS or RoB 2 tools.
Primary Outcome: Subsequent Suicide Attempts
This meta-analysis is based on 3 studies.8,15,20 The effect size index is the odds ratio, which was 0.850 with a 95% confidence interval (CI) of 0.398 to 1.815. The Z-value was −0.42 with P = .674, indicating that the null hypothesis cannot be rejected (Fig 4). The Q-statistic was 1.518 with 2 degrees of freedom, and because this value was smaller than the expected value under the null hypothesis (ie, 2), the variance of the true effects was estimated as 0. Two studies had some concerns for risk of bias because of missing outcome data,8,20 potential outcome assessor bias,8 and the post hoc inclusion of this analysis.20 It is unclear if these potential biases favored intervention or control.
Forest plot of subsequent suicide attempts.10,22,27 Odds ratios <1 indicate that the interventions resulted in fewer suicide attempts at follow-up. Rates imputed from Rotheram-Borus10 assume equal rates of attrition between groups, per the author’s advice. See Supplemental Table 4 for details on study-specific outcome data.
Forest plot of subsequent suicide attempts.10,22,27 Odds ratios <1 indicate that the interventions resulted in fewer suicide attempts at follow-up. Rates imputed from Rotheram-Borus10 assume equal rates of attrition between groups, per the author’s advice. See Supplemental Table 4 for details on study-specific outcome data.
Primary Outcome: Subsequent ED, Crisis Center Visits, and Hospitalizations
Only 1 study had comparative results for this outcome.16 The authors of this study assessed rebound ED visits within 1 month, finding that intervention patients had a non-statistically significant increased rate of returning for crisis intervention (13% vs 4%; Fisher’s exact test; P = .07).16 The same study found a non-statistically significant increased number of intervention patients required subsequent psychiatric hospitalization within 1 month of intervention (7% vs 3%; Fisher’s exact test; P = .27).16 In another study, 3.6% of patients in the intervention were hospitalized because of SI in the 3 months after intervention, but there was no comparison with the control group.17
Secondary Outcomes
Screening and Identification Rates
The authors of only 1 study compared the rates of suicide risk screening (82.1% vs 36.5%; OR 2.49; 95% CI 2.02–2.97; P < .05) and identification of patients at risk of suicide (3.6% vs 0.8%; OR 4.33; 95% CI 3.73–4.94; P < .05) between intervention and control groups, with both outcomes favoring the intervention group.25
Immediate Hospitalizations
This meta-analysis is based on 5 studies from 3 articles.16,17,24 The mean effect size was 0.336 with a 95% CI of 0.149 to 0.755. The Z-value is −2.639 with P = .008, indicating a statistically significant reduction in immediate psychiatric hospitalizations in the intervention versus the control conditions (Fig 5). The Q-value was 35.544 with 4 degrees of freedom, P < .001. τ-squared, the variance of true effect sizes, was 0.733 in log units. All studies had a serious or high risk of bias for this outcome because of outcome assessors also delivering the interventions,16,17,24 and 1 had a high risk of bias for deviation from the intended intervention.24
Forest plot of immediate psychiatric hospitalizations.25,26 Odds ratios <1 indicate that the interventions resulted in fewer hospitalizations at follow-up as compared with controls. Twelve-month Ns were imputed from reported monthly participant rates for Parker30 for inclusion in this meta-analysis. Parker et al, 2003 measured immediate hospitalizations at Hotel Dieu Hospital in after introduction of an intervention program in 1998 (Parker 2003a), in 2000 after termination of the program in 1999 (Parker 2003b), and at the Hospital for Sick Children in 1999 (Parker 2003c). See Supplemental Table 5 for details on study-specific outcome data.
Forest plot of immediate psychiatric hospitalizations.25,26 Odds ratios <1 indicate that the interventions resulted in fewer hospitalizations at follow-up as compared with controls. Twelve-month Ns were imputed from reported monthly participant rates for Parker30 for inclusion in this meta-analysis. Parker et al, 2003 measured immediate hospitalizations at Hotel Dieu Hospital in after introduction of an intervention program in 1998 (Parker 2003a), in 2000 after termination of the program in 1999 (Parker 2003b), and at the Hospital for Sick Children in 1999 (Parker 2003c). See Supplemental Table 5 for details on study-specific outcome data.
One study with an SAR intervention was not included in the meta-analysis because of missing critical data.14 The study authors reported a significant decrease in emergency room referrals from the intervention clinic (87% decrease, OR = 0.13, 95% CI 0.03–0.36, P < .001).14 However, the authors reported no statistical difference in psychiatric hospitalization rates from the intervention clinic, likely because of being underpowered (10.7 admissions per year, 4 admissions per year after intervention).
Referrals to Behavioral Health
Interventions were positively associated with referrals to behavioral health in 2 studies.17,25 One revealed a significant increase in behavioral health referrals for SI made by primary care providers after introducing a suicide risk screening program (3.6% vs 0.8% of patients screened for suicide risk; OR 4.33; 95% CI 3.73–4.94; P < .05).25 The other study revealed a shift from inpatient admission to referral to intensive outpatient (21% vs 5.3%; χ2 value not reported; P < .001) and regular outpatient therapy (43% vs 37%; χ2 value not reported; nonsignificant) in intervention patients.17 Both studies had an overall serious risk of bias for this outcome.
Behavioral Health Service Use
Both CIS and SAR interventions revealed an association between intervention and patient behavioral health service use and engagement.8,13,15,18,20 The mean effect size was 3.843 with a 95% CI of 2.12 to 6.967. The Z-value is 4.44 with P < .001, indicating an overall statistically significant effect of the interventions (Fig 6). The Q-value was 13.041 with 4 degrees of freedom, P = .011, and τ-squared was 0.284 in log units. These studies all had a low or moderate risk of bias for this outcome. This moderate risk of bias was largely mediated by missing outcome data8,20 with unclear implications for the directionality of bias.
Forest plot of behavioral and mental health care service use.10,19,20,22,27 Odds ratios >1 indicate that the interventions resulted in higher rates of using behavioral and mental health care services compared with controls. See Supplemental Table 6 for details on study-specific outcome data.
Subsequent Depression Symptoms
This meta-analysis is based on 5 studies.8,15,18,20,21 The mean effect size was 0.405 with a 95% CI of 0.143 to 0.666. The Z-value was 3.035 with P = .002, indicating an overall statistically significant effect of the interventions (Fig 7). The Q-value was 7.133 with 4 degrees of freedom, P = .129, and τ-squared, the variance of true effect sizes, was 0.039 in d units. Two of these studies had a moderate risk of bias for missing outcome data,8,20 and 1 had a moderate risk of bias due to participants being unblinded about their intervention status.18
Forest plot of the effects of interventions on subsequent depression symptoms.10,20,22,27,28 Hedges’ g >0 indicates that interventions reduced scores on measures of depression at follow-up (controlling for baseline scores) compared with controls. All studies had a low or moderate risk of bias for these outcomes. See Supplemental Table 7 for details on study-specific outcome data.
Forest plot of the effects of interventions on subsequent depression symptoms.10,20,22,27,28 Hedges’ g >0 indicates that interventions reduced scores on measures of depression at follow-up (controlling for baseline scores) compared with controls. All studies had a low or moderate risk of bias for these outcomes. See Supplemental Table 7 for details on study-specific outcome data.
Subsequent SI
This meta-analysis is based on 4 studies.15,16,20,21 The mean effect size for SI measures was 0.053 with a 95% CI of −0.160 to 0.255. The Z-value was −0.485 with P = .628, which implies that there is no statistically significant effect of the interventions on SI scores (Fig 8). The Q-value was 1.656 with 3 degrees of freedom, and because this value is smaller than the expected value under the null hypothesis (ie, 3), the variance of the true effects was estimated as 0. There was, however, a significant time effect revealing decreased SI for both intervention and control groups across 3 studies over time.16,20,21 One study had a moderate risk of bias with an unclear impact on results due to missing participant data.20 A fifth study that revealed no significant change in SI (β = −0.316; P value not reported) was not included in this meta-analysis because it did not include SI symptom scores, and they were not available from the author.8 This study had a moderate risk of bias due to missing data and concerns that outcome assessors (therapists) were not blind to participant intervention status.
Forest plot of the effects of interventions on subsequent suicidal ideation.2,22,25,27 Hedges’ g >0 means that interventions reduced scores on measures of suicidal ideation at follow-up (controlling for baseline scores) as compared with controls. See Supplemental Table 7 for details on study-specific outcome data.
Forest plot of the effects of interventions on subsequent suicidal ideation.2,22,25,27 Hedges’ g >0 means that interventions reduced scores on measures of suicidal ideation at follow-up (controlling for baseline scores) as compared with controls. See Supplemental Table 7 for details on study-specific outcome data.
Discussion
Overall, the results revealed a lack of intervention effect on subsequent suicide attempts and subsequent SI. The results did, however, suggest an impact on secondary outcomes of reducing immediate psychiatric hospitalizations and improving patient access to mental health care, with a milder effect on patients’ subsequent depressive symptoms. These findings provide evidence that brief suicide interventions in medical settings can improve the delivery of mental health care to suicidal youth. However, they also make clear that these interventions alone are insufficient at decreasing future suicide attempts and SI in youths.
Subsequent Suicide Attempts and Crisis Interventions
A previous meta-analysis of 14 brief interventions that combined adult and adolescent studies (4 overlapped with the current study) revealed a trend toward fewer subsequent suicide attempts.26 However, of the 7 studies examining this outcome, only 2 (included in this review) involved adolescents, and the results were the weakest for those. Because a reduction in suicide attempts could be considered the most critical outcome of prevention efforts, our finding of a lack of such effects is disappointing. This may reflect the challenge facing suicide prevention studies related to low base rates of suicide and small sample sizes.
A large percentage of patients seen in the ED for suicide or self-harm return to the ED for psychiatric reasons within 1 year.27 The authors of the small number of studies reviewed in the current paper did not sufficiently evaluate the effect of intervention on this outcome.
Subsequent Symptomatology
Five studies assessed subsequent depressive symptoms. Overall, intervention groups had significantly lower depression scores at follow-up compared with control groups. The impact of interventions on subsequent SI was not significant based on 4 studies. However, there was significantly decreased SI over time for intervention and control groups in 3 of the 4 studies included in this review. A previous systematic review of suicide prevention interventions identified 13 studies in health care settings (ED, clinics, inpatient) that measured SI at follow-up.28 Of those, the authors of only 4 reported significant effects of the intervention condition, although many of the interventions were more intensive than those included in the current review.28
Immediate Psychiatric Hospitalization
Our results suggested a drastic reduction in immediate psychiatric hospitalizations in the intervention conditions. Although the degree of this effect was mediated by a serious risk of bias among outcome assessors in all qualifying studies, the directionality of the effect was likely real. Coupling suicide risk assessment and safety and disposition planning is a core feature of modern brief suicide prevention strategies.29,30
Over the past 15 years, pediatric hospitalizations for mental health disorders have increased by >50%, with total expenditures of $11.6 billion.31 Studies have revealed that significant percentages of youth are readmitted for inpatient psychiatric care within 1 year of discharge.32,–36 Such repeat hospitalizations can disrupt social support and school performance and result in greater stigmatization for youth and families. Thus, brief interventions such as the Family-Based Crisis Intervention or the Rapid Response Model may provide an alternative to psychiatric hospitalization, providing social benefits to the patient and cost benefits to the system.16,17,24
Access to Mental Health Care
CIS interventions had a strong association with increased access to outpatient behavioral health care with little indication that this finding was subject to a significant risk of bias. These findings are consistent with the Doupnick meta-analyses, which revealed that the pooled effect of the interventions was to increase linkage to follow-up care in both adults and adolescents.26 They are particularly important because attendance at mental health follow-up after an ED visit or hospitalization for suicide risk is associated with as much as a 75% lower risk of subsequent suicide death.37 However, overall rates of mental health follow-up after the acute treatment of suicide attempts are low, particularly for youths who were not engaged in care before the ED visit.38,39
Limitations
This review has several limitations. First, our literature search was limited to published articles and reports in English. Second, only 14 studies met the inclusion criteria, and only the subset with relevant outcomes was included in each meta-analysis. Additionally, the studies were heterogeneous, with a mix of RCTs, non-RCTs with 2 or more arms, and single-arm trials. Because of this, definitive effect sizes for specific settings or intervention models could not be estimated. Risk of bias was also a limitation; we could only conduct a risk of bias assessment on 11 studies, 8 of which had at least 1 outcome with a moderate risk of bias. Finally, we were not able to examine whether brief suicide interventions reduced suicide deaths because most studies did not include death as an outcome.
Implications for Future Research and Practice
This review and meta-analysis synthesized the literature on suicide screening and brief interventions with youths in outpatient medical settings. In agreement with other reviews, such interventions may be effective at improving patients’ linkage to follow-up mental health care, as well as reducing immediate hospitalization and symptoms of depression.
The small number of studies likely results from several factors, including challenges in approaching acutely suicidal patients and families, clinical site environmental factors, and youth and parental concerns regarding the study. The authors of future studies should focus on established ways to enhance recruitment.40 Reliable state-level and health system-level estimates of suicides, suicide attempts, and rehospitalizations would help determine the true impact of suicide prevention programs.
Access to timely care after the identification and assessment of suicide risk continues to be a challenge. Although this review revealed that brief interventions in medical settings can be successful, further investigation into engaging families and high-risk adolescents in treatment is warranted, particularly with family-focused interventions.41,42 In addition, because positive results have been reported in interventions using technology with adults, further studies exploring technology to enhance engagement43 and increase access44,45 are encouraged.
The success of these efforts underlines the importance of integrating interventions into general medical care. The Joint Commission’s 2016 mandate to identify patients at risk for suicide4 has increased the number of health care settings screening for suicide risk. However, successful integration depends on several factors. First, health care settings need robust systems to identify patients with suicide risk. Clinical pathways for universal suicide risk screening for youths in EDs and similar medical settings have been developed, with attention to both assessment tools and implementation factors.45 Nonetheless, their success varies. Second, suicide prevention interventions need experienced clinicians who have the skills and knowledge to deliver them.41 Yet, many settings have limited access to this workforce and may rely on other professionals to deliver components of the interventions. Settings with limited access to mental health professionals may consider using telehealth to improve access to mental health. Also, to ensure appropriate resources, health care systems need to identify mechanisms to reimburse the time and resources required to deliver evidence-based suicide prevention interventions.
Finally, the utility of such clinical intervention lies primarily in its ability to link at-risk youths with appropriate care. Procedures that facilitate referral should be systematic and include a structured approach to engage high-risk patients in safety planning and ensure that patients have access to evidence-based services. Because most suicidal patients who leave the ED never attend a follow-up appointment, facilitating the transition of care after discharge is essential. Telehealth and other modes of contact should be considered to facilitate care coordination, which is a key component of many of the included interventions.46
Conclusions
Brief suicide prevention interventions in outpatient medical settings can increase access to behavioral health services while limiting unnecessary psychiatric hospitalizations. Improving patient symptoms and functioning in the long-term likely relies on many postintervention variables, including adherence to and quality of mental health care and community support systems.
Acknowledgment
The author group would like to acknowledge Erin Bishop, MPH, for her initial coordination of this project.
Drs Anthony and Wood and Ms Sharma contributed to reviewing articles for inclusion, extracting data, and drafting and reviewing the manuscript; Dr Dar contributed to meta-analyses and drafting related parts of the manuscript; Ms De Jesus Ayala contributed to extracting data and drafting and reviewing the manuscript; Ms Doyle contributed to extracting data, synthesizing data, reviewing study risk of bias, and drafting and reviewing the manuscript; Dr Pitts coordinated all review efforts and was lead in designing the protocol and contributed to reviewing articles for inclusion, extracting data, synthesizing data, reviewing study risk of bias, and drafting and reviewing the manuscript; Ms St. Pierre conducted an article search through databases and drafted sections related to article search strategies; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: No external funding.
CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no potential conflicts of interest to disclose.
- CI
confidence interval
- CIS
crisis intervention session
- ED
emergency department
- NRC
nonrandomized controlled
- OR
odds ratio
- RCT
randomized controlled trial
- RoB 2
Cochrane Collaboration tool for assessing risk of bias for randomized trials, version 2
- ROBINS-I
Cochrane Collaboration tool for assessing risk of bias in nonrandomized studies of interventions
- SAR
screening, assessment, and referral
- SI
suicidal ideation
Comments