Suicidal ideation and plans are increasing among US adolescents. Changing prevalence of online victimization is frequently hypothesized as an explanation for this increase. We tested trends in online and offline victimization and whether they contribute to recent trends in adolescent suicidal outcomes.
Youth Risk Behavior Survey data (2011–2019, N = 73 074) were collected biennially through national cross-sectional surveys of US school-attending adolescents. We examined trends in past-year victimization. We also examined whether the relationship between victimization and past-year suicidal ideation, plans, attempts, and injury changed over time using survey-weighted logistic regressions that adjusted for sex and race and ethnicity. We also sex-stratified results to examine sex differences.
Although suicidal ideation and plans increased among US adolescents (mainly girls), online and offline victimization prevalence did not increase over time (offline: 20.0% in 2011, 19.5% in 2019; online: 16.2% in 2011, 15.7% in 2019). Online and offline victimization were associated with suicidal outcomes, especially co-occurring online and offline victimization (eg, adjusted odds ratio [co-occurring online and offline victimization versus none, outcome: suicidal injury] = 8.37; 95% confidence interval: 7.06–9.91). The magnitude of the associations between victimization and suicidal outcomes largely remained stable over time.
Peer victimization prevalence has not sufficiently changed over time in concert with suicidal outcomes to explain increased suicidal outcomes. The prevalence of victimization has remained relatively invariant across time despite growing awareness and programming, making online and offline victimization consistent, socially-patterned risk factors that warrant further monitoring and interventions. Research must examine risk factors beyond victimization to explain increasing suicidal outcomes.
What’s Known on the Subject:
Bullying victimization is linked to worse adolescent mental health, which is an urgent priority given increases in US adolescent suicidality.
What This Study Adds:
Although the links between victimization and suicidal behavior are clear, victimization is not changing, either in prevalence or in the strength of the association with suicidality, over time to explain suicidal behavior trends.
Suicidal behavior in children and adolescents is an increasingly serious problem, with 3000 suicide deaths among youth ages 10 to 19 in 2018.1 The national number of emergency department visits by children and adolescents with suicide attempts and ideation nearly doubled between 2007 and 2015,2 and suicide deaths among youth ages 10 to 19 increased from 4.37 to 7.10 per 100 000 between 2009 and 2018.1 Data from the Youth Risk Behavior Survey (YRBS) indicate that suicidal ideation and plans have increased among US adolescents since 2009, with significant heterogeneity by sex, given that trends were primarily seen among female adolescents.3–5 Understanding the factors behind these trends is an important public health issue.
One factor hypothesized to contribute to recent trends in adolescent suicide is peer victimization, partially in person (ie, perceived offline victimization) but especially through the rise of social media and other Internet sites (ie, perceived online victimization).6 Offline7,8 and online bullying and victimization7,9 are well-documented as factors associated with adolescent mental health and suicidal outcomes.10 They are also correlated with each other,11,12 indicating that adolescents who are victimized online are more likely to be victimized offline as well.
Although researchers in previous studies have demonstrated a link between peer victimization and adverse psychiatric outcomes, much remains unknown. Specific questions include whether the prevalence of peer victimization has changed, whether such trends in peer victimization explain increases in suicidal outcomes, and whether these dynamics apply equally across sexes. Online and offline peer victimization could explain trends in suicidal ideation, plans, attempts, and injury among adolescents if their prevalence had increased in concert with the prevalence of these suicidal outcomes. Although opportunities for anonymous online interaction (and thus victimization) have increased with the proliferation of social media and other online engagement sources, it is not known whether these opportunities have translated to changes in peer victimization prevalence. Online and offline peer victimization may also explain trends in suicidal outcomes if the magnitude of the relationship between victimization and a given outcome has increased over time, yet no studies to date have assessed time trends in the magnitude of these risk factor relationships, overall or by sex.
In this study, we aimed to examine (1) whether online or offline peer victimization and suicidal ideation, plans, attempts, and injury have increased in prevalence over time, and (2) the extent to which victimization explains trends in suicidal outcomes within a US nationally representative sample of adolescents from 2011 to 2019, overall and by sex. Findings could be used to inform suicide prevention interventions, adding nuanced understanding of the specific risks posed by various forms of victimization and exploring the extent to which trends in suicidal outcomes are accounted for by victimization.
Methods
We used YRBS data collected biennially through national cross-sectional surveys, with ∼15 000 US school-attending adolescents in each survey (grades 9 to 12). Participation is voluntary and anonymous. As of 2019, the school response rate was 75.1%, whereas the student response rate was 80.3%.13 The Centers for Disease Control and Prevention Institutional Review Board approved the protocol. Full sample size was 73 074 from 2011 to 2019, starting with the year when online victimization was introduced to the survey.
Measures
The YRBS included 4 items related to suicide. Suicidal ideation was assessed with the question, “During the past 12 months, did you ever seriously consider attempting suicide?” Suicide plans were assessed with the question, “During the past 12 months, did you make a plan about how you would attempt suicide?” Suicide attempts were assessed with the question, “During the past 12 months, how many times did you actually attempt suicide?” Suicide injury was assessed with the question, “If you attempted suicide during the past 12 months, did any attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?”. Items that queried the number of times in which suicidal behavior occurred were dichotomized into any versus none. These items exhibit strong convergent and discriminant validity.14
Perceived peer victimization was assessed with 2 self-reported items with yes or no responses. Offline victimization was assessed with the question “During the past 12 months, have you ever been bullied on school property?” Online victimization was assessed with the question “During the past 12 months, have you ever been electronically bullied? (Count being bullied through texting, Instagram, Facebook, or other social media.)” These items were combined into a four-level exposure from 2011 onward: no victimization, online only, offline only, or both online and offline.
To account for potential demographic confounding, we examined sex and race and ethnicity as covariates. Sex was self-described as “male” or “female,” and race and ethnicity were based on 5 self-reported categories, with options to select as many as applicable, and reconfigured into 6 categories: American Indian or Alaskan Native, Asian American and Pacific Islander, non-Hispanic Black or African American, Hispanic and/or Latino, non-Hispanic white, and non-Hispanic multiracial.
Race and ethnicity are examined here as a social construct and important personal identity which, along with sex, can shape both peer victimization experiences and suicidal outcomes. For instance, adolescents of color can face uniquely harmful effects of bias-based peer victimization, or disproportionate levels of contextual-level risk factors linked to peer victimization (eg, adverse school environments).15 For suicidal outcomes, not only do the factors contributing to suicidal behavior vary across racial and ethnic categories,16 but the prevalence of these outcomes varied across racial and ethnic groups in YRBS data.17 These prevalence disparities also occurred by sex, with girls having higher levels of both suicidal outcomes and peer victimization.17,18 Therefore, controlling for this racial, ethnic, and sexual heterogeneity is an important step needed to understand the extent to which peer victimization explains and predicts suicidal outcomes at the population level.
Statistical analysis
We estimated prevalence and 95% confidence intervals (CIs) of offline only, online only, and co-occurring forms of victimization in each biennial YRBS survey. We also estimated the biyearly time trends in victimization using survey-weighted logistic regression models with each form of victimization as the outcome and the continuous year (per 2 years) as a predictor.
We characterized trends in suicidal outcomes with the biennial prevalence estimates and 95% CIs of dichotomous self-reported outcomes (ideation, plans, attempts, injury).
To examine the extent to which victimization predicts each outcome, we fit separate survey-weighted logistic regression models to assess the association between each form of victimization and each of the 4 binary suicide outcomes, unadjusted and adjusted for sex and race and ethnicity.
To further understand whether victimization explains trends in suicidal outcomes, we compared estimates for year predicting each outcome with and without adjustment for victimization. Additionally, we examined temporal heterogeneity in the association between victimization and suicidal outcomes by fitting weighted logistic regression models linking victimization and each outcome for each survey year and testing whether associations vary over time by including the interaction between year (treated categorically) and victimization in models using data from all survey years.
Finally, to examine heterogeneity by sex in these associations and patterns, we stratified all of the above analyses by sex. Sample sizes for many racial and ethnic groups were too small to have sufficient power for stratification.
We used the SURVEYFREQ procedure (SAS 9.4; SAS Institute, Inc, Cary, NC) to estimate prevalence and the SURVEYLOGISTIC procedure (SAS 9.4) to fit logistic regression models. All statistical analyses accounted for the complex sampling design with YRBS.
Results
Distributions of demographic variables, experiences of victimization, and suicidal outcomes by year can be seen in Supplemental Table 6.
Trends in Offline and Online Victimization
In 2019, 24.78% of participants reported any victimization, whereas the overall prevalence of experiencing victimization over the study period was consistently close to the overall mean of 25.35%. The prevalence of offline and online victimization stayed relatively flat over the study period (Fig 1), with slight decreases in the experience of offline only or online only victimization over time (odds ratio [OR] for each 2-year increase in time = 0.98 [95% CI: 0.96–0.99] and 0.97 [95% CI: 0.95–0.99], respectively). Co-occurring online and offline victimization increased from 9.4% in 2011% to 10.3% in 2019, but there was limited evidence for trends (OR for each 2-year increase in time = 1.01 [95% CI: 1.00–1.03; P > .05]). Victimization experiences were correlated; students who experienced offline victimization had 12.95 times the odds of experiencing online victimization compared with students who did not (95% CI: 12.03–13.95).
Trends in peer victimization among high school students, 2011–2019 YRBS. Offline victimization was assessed with the question: “During the past 12 months, have you ever been bullied on school property?”; Online victimization was assessed with the question: “During the past 12 months, have you ever been electronically bullied? (Count being bullied through texting, Instagram, Facebook, or other social media.).”
Trends in peer victimization among high school students, 2011–2019 YRBS. Offline victimization was assessed with the question: “During the past 12 months, have you ever been bullied on school property?”; Online victimization was assessed with the question: “During the past 12 months, have you ever been electronically bullied? (Count being bullied through texting, Instagram, Facebook, or other social media.).”
By sex, patterns were relatively consistent, although disparities existed (Fig 2). Offline only victimization decreased for male adolescents and was static for female adolescents (OR for each 2-year increase in time = 0.95 [95% CI–0.93, 0.97] and 1.00 [95% CI: 0.98–1.02], respectively). Conversely, online only victimization was static for male adolescents and decreased for female adolescents (OR for each 2-year increase in time = 0.98 [95% CI: 0.95–1.01] and 0.96 [95% CI: 0.94–0.98]). For male and female adolescents, co-occurring online and offline victimization prevalence has remained stable (OR for each 2-year increase in time = 1.02 [95% CI: 0.99–1.06] and 1.01 [95% CI: 0.99–1.03] respectively).
Trends in peer victimization among high school students, split by sex, 2011–2019 YRBS.
Trends in peer victimization among high school students, split by sex, 2011–2019 YRBS.
Trends in Suicidal Outcomes
Suicidal ideation prevalence gradually increased from 2011 (15.8%) to 2019 (18.8%). Similar trends emerged for making suicide plans, but trends for reported suicide attempts or injury by attempt were less marked (Fig 3). Suicidal ideation and plans were not increasing for male adolescents (Fig 4), but they were for female adolescents (Fig 5), and all outcomes were higher for female adolescents.
Trends in suicide outcomes, 2011–2019 YRBS. Plan: During the past 12 months, did you make a plan about how you would attempt suicide? Attempt: During the past 12 months, how many times did you actually attempt suicide? Injury: If you attempted suicide during the past 12 months, did any attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?
Trends in suicide outcomes, 2011–2019 YRBS. Plan: During the past 12 months, did you make a plan about how you would attempt suicide? Attempt: During the past 12 months, how many times did you actually attempt suicide? Injury: If you attempted suicide during the past 12 months, did any attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?
Trends in suicide outcomes, male, 2011–2019 YRBS. Plan: During the past 12 months, did you make a plan about how you would attempt suicide? Attempt: During the past 12 months, how many times did you actually attempt suicide? Injury: If you attempted suicide during the past 12 months, did any attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?
Trends in suicide outcomes, male, 2011–2019 YRBS. Plan: During the past 12 months, did you make a plan about how you would attempt suicide? Attempt: During the past 12 months, how many times did you actually attempt suicide? Injury: If you attempted suicide during the past 12 months, did any attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?
Trends in suicide outcomes, female, 2011–2019 YRBS. Plan: During the past 12 months, did you make a plan about how you would attempt suicide? Attempt: During the past 12 months, how many times did you actually attempt suicide? Injury: If you attempted suicide during the past 12 months, did any attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?
Trends in suicide outcomes, female, 2011–2019 YRBS. Plan: During the past 12 months, did you make a plan about how you would attempt suicide? Attempt: During the past 12 months, how many times did you actually attempt suicide? Injury: If you attempted suicide during the past 12 months, did any attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?
Associations Between Offline and Online Victimization and Suicidal Outcomes
The relationships between each form of victimization and suicidal outcomes are displayed in Table 1, which includes the unadjusted associations and the associations adjusting for sex and race and ethnicity. Experiences of online only, offline only, and co-occurring victimization were all associated with each of the 4 suicidal outcomes. After adjustment, the associations between victimization and suicidal outcomes ranged in magnitude from the association between online victimization and having suicidal plans (adjusted odds ratio [aOR] = 2.69, 95% CI: 2.40–3.02) to the association between co-occurring online and offline victimization and suicidal injury (aOR = 8.37, 95% CI: 7.06–9.91).
Unadjusted ORs and aORs for Suicide Outcomes Based on Experiences of Victimization (Reference = No Victimization), 2011–2019 YRBS
Suicidal Behavior Outcome . | Experience of Victimization . | ||
---|---|---|---|
Offline Victimization Only . | Online Victimization Only . | Both . | |
Suicidal ideation, OR (95% CI) | 2.83 (2.61–3.07) | 3.11 (2.81–3.45) | 6.40 (5.92–6.92) |
Suicidal ideation,a aOR (95% CI) | 2.84 (2.62–3.07) | 2.83 (2.55–3.14) | 5.83 (5.37–6.32) |
Suicidal plans, OR (95% CI) | 2.71 (2.48–2.96) | 2.88 (2.58–3.21) | 5.78 (5.29–6.33) |
Suicidal plans,a aOR (95% CI) | 2.76 (2.53–3.01) | 2.69 (2.40–3.02) | 5.40 (4.92–5.92) |
Suicide attempts, OR (95% CI) | 2.74 (2.42–3.09) | 3.58 (3.13–4.08) | 6.84 (6.14–7.63) |
Suicide attempts,a aOR (95% CI) | 2.83 (2.51–3.21) | 3.42 (2.98–3.92) | 6.81 (6.08–7.62) |
Suicide injury, OR (95% CI) | 2.67 (2.19–3.25) | 4.39 (3.53–5.46) | 8.10 (6.89–9.53) |
Suicide injury,a aOR (95% CI) | 2.79 (2.28–3.41) | 4.32 (3.40–5.48) | 8.37 (7.06–9.91) |
Suicidal Behavior Outcome . | Experience of Victimization . | ||
---|---|---|---|
Offline Victimization Only . | Online Victimization Only . | Both . | |
Suicidal ideation, OR (95% CI) | 2.83 (2.61–3.07) | 3.11 (2.81–3.45) | 6.40 (5.92–6.92) |
Suicidal ideation,a aOR (95% CI) | 2.84 (2.62–3.07) | 2.83 (2.55–3.14) | 5.83 (5.37–6.32) |
Suicidal plans, OR (95% CI) | 2.71 (2.48–2.96) | 2.88 (2.58–3.21) | 5.78 (5.29–6.33) |
Suicidal plans,a aOR (95% CI) | 2.76 (2.53–3.01) | 2.69 (2.40–3.02) | 5.40 (4.92–5.92) |
Suicide attempts, OR (95% CI) | 2.74 (2.42–3.09) | 3.58 (3.13–4.08) | 6.84 (6.14–7.63) |
Suicide attempts,a aOR (95% CI) | 2.83 (2.51–3.21) | 3.42 (2.98–3.92) | 6.81 (6.08–7.62) |
Suicide injury, OR (95% CI) | 2.67 (2.19–3.25) | 4.39 (3.53–5.46) | 8.10 (6.89–9.53) |
Suicide injury,a aOR (95% CI) | 2.79 (2.28–3.41) | 4.32 (3.40–5.48) | 8.37 (7.06–9.91) |
Adjusted for race and ethnicity and sex.
There was some heterogeneity in the links between victimization and suicidal outcomes by sex, particularly for suicidal plans, attempts, and injury (Table 2). Whereas links between offline victimization and these outcomes were relatively consistent between sexes, links between online or co-occurring patterns of victimization and these outcomes were stronger for male adolescents.
Unadjusted ORs and aORs for Suicide Outcomes Based on Experiences of Victimization (Reference = No Victimization), by Sex, 2011–2019 YRBS
Suicidal Behavior Outcome . | Female . | Male . | Interaction P . | ||||
---|---|---|---|---|---|---|---|
Offline Victimization Only . | Online Victimization Only . | Both . | Offline Victimization Only . | Online Victimization Only . | Both . | ||
Suicidal ideation, OR (95% CI) | 2.81 (2.55–3.09) | 2.68 (2.38–3.02) | 5.44 (4.93–6.02) | 2.84 (2.50–3.23) | 3.12 (2.56–3.81) | 6.50 (5.71–7.39) | .1378 |
Suicidal ideation,a aOR (95% CI) | 2.85 (2.58–3.15) | 2.73 (2.42–3.08) | 5.68 (5.11–6.30) | 2.80 (2.46–3.19) | 3.08 (2.52–3.75) | 6.30 (5.54–7.16) | |
Suicidal plans, OR (95% CI) | 2.72 (2.45–3.03) | 2.40 (2.12–2.73) | 4.86 (4.37–5.40) | 2.65 (2.31–3.04) | 3.21 (2.61–3.94) | 6.23 (5.34–7.27) | .0070 |
Suicidal plans,a aOR (95% CI) | 2.80 (2.52–3.12) | 2.49 (2.18–2.83) | 5.14 (4.60–5.74) | 2.68 (2.33–3.08) | 3.22 (2.62–3.95) | 6.11 (5.23–7.14) | |
Suicide attempts, OR (95% CI) | 2.74 (2.38–3.17) | 2.83 (2.40–3.33) | 5.70 (5.02–6.46) | 2.64 (2.15–3.23) | 4.53 (3.50–5.84) | 7.35 (6.13–8.81) | .0015 |
Suicide attempts,a aOR (95% CI) | 2.86 (2.48–3.30) | 2.98 (2.52–3.51) | 6.43 (5.66–7.31) | 2.78 (2.26–3.42) | 4.64 (3.59–6.00) | 7.70 (6.33–9.36) | |
Suicide injury, OR (95% CI) | 2.59 (1.99–3.38) | 3.33 (2.57–4.32) | 6.78 (5.60–8.22) | 2.72 (1.96–3.80) | 6.20 (4.13–9.29) | 8.81 (6.67–11.64) | .0459 |
Suicide injury,a aOR (95% CI) | 2.70 (2.08–3.51) | 3.52 (2.69–4.62) | 7.53 (6.21–9.13) | 2.92 (2.09–4.07) | 6.40 (4.21–9.71) | 9.85 (7.36–13.19) | — |
Suicidal Behavior Outcome . | Female . | Male . | Interaction P . | ||||
---|---|---|---|---|---|---|---|
Offline Victimization Only . | Online Victimization Only . | Both . | Offline Victimization Only . | Online Victimization Only . | Both . | ||
Suicidal ideation, OR (95% CI) | 2.81 (2.55–3.09) | 2.68 (2.38–3.02) | 5.44 (4.93–6.02) | 2.84 (2.50–3.23) | 3.12 (2.56–3.81) | 6.50 (5.71–7.39) | .1378 |
Suicidal ideation,a aOR (95% CI) | 2.85 (2.58–3.15) | 2.73 (2.42–3.08) | 5.68 (5.11–6.30) | 2.80 (2.46–3.19) | 3.08 (2.52–3.75) | 6.30 (5.54–7.16) | |
Suicidal plans, OR (95% CI) | 2.72 (2.45–3.03) | 2.40 (2.12–2.73) | 4.86 (4.37–5.40) | 2.65 (2.31–3.04) | 3.21 (2.61–3.94) | 6.23 (5.34–7.27) | .0070 |
Suicidal plans,a aOR (95% CI) | 2.80 (2.52–3.12) | 2.49 (2.18–2.83) | 5.14 (4.60–5.74) | 2.68 (2.33–3.08) | 3.22 (2.62–3.95) | 6.11 (5.23–7.14) | |
Suicide attempts, OR (95% CI) | 2.74 (2.38–3.17) | 2.83 (2.40–3.33) | 5.70 (5.02–6.46) | 2.64 (2.15–3.23) | 4.53 (3.50–5.84) | 7.35 (6.13–8.81) | .0015 |
Suicide attempts,a aOR (95% CI) | 2.86 (2.48–3.30) | 2.98 (2.52–3.51) | 6.43 (5.66–7.31) | 2.78 (2.26–3.42) | 4.64 (3.59–6.00) | 7.70 (6.33–9.36) | |
Suicide injury, OR (95% CI) | 2.59 (1.99–3.38) | 3.33 (2.57–4.32) | 6.78 (5.60–8.22) | 2.72 (1.96–3.80) | 6.20 (4.13–9.29) | 8.81 (6.67–11.64) | .0459 |
Suicide injury,a aOR (95% CI) | 2.70 (2.08–3.51) | 3.52 (2.69–4.62) | 7.53 (6.21–9.13) | 2.92 (2.09–4.07) | 6.40 (4.21–9.71) | 9.85 (7.36–13.19) | — |
—, not applicable.
Adjusted for race and ethnicity.
Offline and Online Victimization Explaining Trends in Suicidal Outcomes
Adjusting for victimization increased, rather than decreased, the model estimate for year predicting the outcome of suicidal ideation (unadjusted β: 0.0199 versus adjusted β: 0.0236), suicide plans (unadjusted β: 0.0231 versus adjusted β: 0.0264), suicide attempts (unadjusted β: 0.0126 versus adjusted β: 0.0143), and suicidal injury (unadjusted β: 0.00466 versus adjusted β: 0.00567). The regression coefficients, β, can be interpreted as the increase in log odds of each suicide outcome given a 2-year increase. For both suicidal ideation and plans, the significantly increasing trends remained significant after adjustment for victimization (aOR for each 2-year increase in time [suicidal ideation] = 1.02, 95% CI: 1.01–1.04; [suicidal plans] = 1.03, 95% CI: 1.01–1.04). By sex, patterns were similar, as adjusting for victimization did not decrease the model estimate for year predicting each outcome for either male or female adolescents.
Offline and Online Victimization Predicting Suicidal Outcomes Over Time
In Table 3 we characterize the association between each form of victimization and each suicidal outcome in each time point adjusting for sex and race and ethnicity, as well as the P values testing the interactions between victimization and year predicting each outcome. The interactions between year and online only victimization indicated no substantial changes in the magnitude of associations across time. The same was found for offline only victimization. However, the association between co-occurring online and offline victimization and suicidal injury has generally changed over time (eg, 2011 aOR = 4.33 [95% CI: 2.77–6.76]; 2019 aOR = 7.66 [95% CI: 5.08–11.55]). Similar patterns were observed in the unadjusted models (Supplemental Table 7).
aORs for Suicide Outcomes Based on Experiences of Victimization by Year, 2011–2019 YRBS
Victimization Experience Predictor, Suicidal Behavior Outcome . | 2011, aOR (95% CI) . | 2013, aOR (95% CI) . | 2015, aOR (95% CI) . | 2017, aOR (95% CI) . | 2019, aOR (95% CI) . | P For Interaction Between Year and Victimization . |
---|---|---|---|---|---|---|
Offline Victimization (reference: none) | ||||||
Suicidal ideation | 2.65 (2.24–3.13) | 2.85 (2.36–3.45) | 2.92 (2.57–3.32) | 3.12 (2.62–3.72) | 2.66 (2.15–3.29) | .7502 |
Suicidal plans | 2.70 (2.26–3.23) | 2.72 (2.35–3.17) | 2.37 (1.96–2.86) | 3.26 (2.68–3.97) | 2.84 (2.26–3.57) | .2337 |
Suicide attempts | 2.33 (1.82–2.99) | 2.82 (2.29–3.47) | 2.83 (2.09–3.84) | 3.64 (2.84–4.66) | 2.66 (1.98–3.58) | .2361 |
Suicide injury | 1.67 (1.01–2.76) | 2.88 (2.05–4.06) | 3.19 (2.10–4.86) | 4.15 (2.80–6.15) | 2.31 (1.30–4.11) | .0868 |
Online victimization (reference: none) | ||||||
Suicidal ideation | 3.26 (2.65–4.01) | 2.86 (2.27–3.59) | 2.58 (2.08–3.20) | 3.40 (2.69–4.30) | 2.21 (1.73–2.81) | .0855 |
Suicidal plans | 3.05 (2.46–3.79) | 2.71 (2.19–3.36) | 2.24 (1.76–2.85) | 3.15 (2.33–4.25) | 2.53 (1.93–3.33) | .3745 |
Suicide attempts | 3.46 (2.55–4.68) | 4.03 (2.90–5.60) | 3.12 (2.33–4.17) | 3.98 (2.84–5.56) | 2.66 (1.97–3.61) | .3401 |
Suicide injury | 3.73 (2.15–6.47) | 4.43 (2.99–6.55) | 5.35 (3.23–8.87) | 3.21 (1.88–5.50) | 4.75 (2.50–9.06) | .8007 |
Both online and offline victimization (reference: none) | ||||||
Suicidal ideation | 5.30 (4.34–6.46) | 6.35 (5.48–7.35) | 6.78 (5.63–8.15) | 5.84 (4.92–6.94) | 5.00 (4.23–5.91) | .0845 |
Suicidal plans | 5.24 (4.18–6.57) | 5.58 (4.68–6.65) | 6.25 (4.95–7.89) | 5.11 (4.17–6.27) | 4.88 (4.11–5.79) | .5222 |
Suicide attempts | 5.66 (4.55–7.03) | 7.31 (5.82–9.17) | 7.92 (6.01–10.43) | 7.28 (5.67–9.36) | 5.91 (4.64–7.52) | .2915 |
Suicide injury | 4.33 (2.77–6.76) | 10.49 (7.22–15.25) | 11.33 (8.16–15.72) | 8.90 (6.34–12.50) | 7.66 (5.08–11.55) | .0149 |
Victimization Experience Predictor, Suicidal Behavior Outcome . | 2011, aOR (95% CI) . | 2013, aOR (95% CI) . | 2015, aOR (95% CI) . | 2017, aOR (95% CI) . | 2019, aOR (95% CI) . | P For Interaction Between Year and Victimization . |
---|---|---|---|---|---|---|
Offline Victimization (reference: none) | ||||||
Suicidal ideation | 2.65 (2.24–3.13) | 2.85 (2.36–3.45) | 2.92 (2.57–3.32) | 3.12 (2.62–3.72) | 2.66 (2.15–3.29) | .7502 |
Suicidal plans | 2.70 (2.26–3.23) | 2.72 (2.35–3.17) | 2.37 (1.96–2.86) | 3.26 (2.68–3.97) | 2.84 (2.26–3.57) | .2337 |
Suicide attempts | 2.33 (1.82–2.99) | 2.82 (2.29–3.47) | 2.83 (2.09–3.84) | 3.64 (2.84–4.66) | 2.66 (1.98–3.58) | .2361 |
Suicide injury | 1.67 (1.01–2.76) | 2.88 (2.05–4.06) | 3.19 (2.10–4.86) | 4.15 (2.80–6.15) | 2.31 (1.30–4.11) | .0868 |
Online victimization (reference: none) | ||||||
Suicidal ideation | 3.26 (2.65–4.01) | 2.86 (2.27–3.59) | 2.58 (2.08–3.20) | 3.40 (2.69–4.30) | 2.21 (1.73–2.81) | .0855 |
Suicidal plans | 3.05 (2.46–3.79) | 2.71 (2.19–3.36) | 2.24 (1.76–2.85) | 3.15 (2.33–4.25) | 2.53 (1.93–3.33) | .3745 |
Suicide attempts | 3.46 (2.55–4.68) | 4.03 (2.90–5.60) | 3.12 (2.33–4.17) | 3.98 (2.84–5.56) | 2.66 (1.97–3.61) | .3401 |
Suicide injury | 3.73 (2.15–6.47) | 4.43 (2.99–6.55) | 5.35 (3.23–8.87) | 3.21 (1.88–5.50) | 4.75 (2.50–9.06) | .8007 |
Both online and offline victimization (reference: none) | ||||||
Suicidal ideation | 5.30 (4.34–6.46) | 6.35 (5.48–7.35) | 6.78 (5.63–8.15) | 5.84 (4.92–6.94) | 5.00 (4.23–5.91) | .0845 |
Suicidal plans | 5.24 (4.18–6.57) | 5.58 (4.68–6.65) | 6.25 (4.95–7.89) | 5.11 (4.17–6.27) | 4.88 (4.11–5.79) | .5222 |
Suicide attempts | 5.66 (4.55–7.03) | 7.31 (5.82–9.17) | 7.92 (6.01–10.43) | 7.28 (5.67–9.36) | 5.91 (4.64–7.52) | .2915 |
Suicide injury | 4.33 (2.77–6.76) | 10.49 (7.22–15.25) | 11.33 (8.16–15.72) | 8.90 (6.34–12.50) | 7.66 (5.08–11.55) | .0149 |
Adjusted for race and ethnicity and sex.
In Tables 4 and 5 we characterize these associations over time by sex. For female adolescents (Table 4), the interactions between year and each form of victimization indicated no substantial changes in the magnitude of associations across time (all P values > 0.05). For male adolescents (Table 5), although there were combinations of victimization pattern and suicidal outcome that exhibited heterogeneity in the strength of associations over time, these changes were not monotonic, and male suicidal outcomes exhibited no significant increases since 2011.
aORs for Suicide Outcomes Based on Experiences of Victimization by Year, Female, 2011–2019 YRBS
Victimization Experience Predictor, Suicidal Behavior Outcome . | 2011, aOR (95% CI) . | 2013, aOR (95% CI) . | 2015, aOR (95% CI) . | 2017, aOR (95% CI) . | 2019, aOR (95% CI) . | P for Interaction Between Year and Victimization . |
---|---|---|---|---|---|---|
Offline victimization (reference: none) | ||||||
Suicidal ideation | 2.76 (2.13–3.58) | 2.98 (2.38–3.72) | 3.02 (2.54–3.59) | 2.95 (2.35–3.72) | 2.54 (2.07–3.11) | .7923 |
Suicidal plans | 2.81 (2.25–3.52) | 2.82 (2.23–3.57) | 2.69 (2.12–3.43) | 3.00 (2.30–3.92) | 2.70 (2.18–3.35) | .9824 |
Suicide attempts | 2.17 (1.63–2.91) | 3.09 (2.34–4.09) | 2.63 (1.86–3.72) | 3.88 (2.90–5.19) | 2.62 (1.96–3.49) | .1160 |
Suicide injury | 1.82 (0.77–4.33) | 3.00 (2.01–4.48) | 2.80 (1.66–4.73) | 3.96 (2.28–6.88) | 1.80 (0.93–3.51) | .4202 |
Online victimization (reference: none) | ||||||
Suicidal ideation | 3.34 (2.58–4.32) | 2.84 (2.25–3.59) | 2.79 (2.23–3.49) | 2.79 (2.07–3.77) | 2.07 (1.55–2.76) | .2742 |
Suicidal plans | 2.78 (2.15–3.61) | 2.63 (2.08–3.33) | 2.04 (1.63–2.54) | 2.79 (1.90–4.12) | 2.45 (1.81–3.32) | .3972 |
Suicide attempts | 2.88 (1.99–4.18) | 3.45 (2.48–4.81) | 2.54 (1.88–3.42) | 3.83 (2.48–5.90) | 2.37 (1.64–3.41) | .3755 |
Suicide injury | 3.21 (1.60–6.42) | 3.67 (2.29–5.90) | 3.76 (2.30–6.14) | 3.36 (1.70–6.61) | 3.44 (1.71–6.93) | .9939 |
Both online and offline victimization (reference: none) | ||||||
Suicidal ideation | 5.39 (4.27–6.80) | 5.97 (4.78–7.45) | 7.03 (5.54–8.91) | 5.55 (4.49–6.86) | 4.64 (3.79–5.69) | .1408 |
Suicidal plans | 5.20 (4.12–6.55) | 5.27 (4.18–6.64) | 5.70 (4.28–7.59) | 5.07 (4.03–6.38) | 4.54 (3.72–5.54) | .7655 |
Suicide attempts | 5.15 (4.02–6.60) | 6.95 (5.23–9.24) | 7.16 (5.18–9.90) | 7.40 (5.68–9.63) | 5.69 (4.36–7.43) | .3226 |
Suicide injury | 4.91 (3.01–8.01) | 8.62 (5.71–13.00) | 9.39 (6.09–14.47) | 7.77 (5.36–11.28) | 7.03 (4.18–11.83) | .3723 |
Victimization Experience Predictor, Suicidal Behavior Outcome . | 2011, aOR (95% CI) . | 2013, aOR (95% CI) . | 2015, aOR (95% CI) . | 2017, aOR (95% CI) . | 2019, aOR (95% CI) . | P for Interaction Between Year and Victimization . |
---|---|---|---|---|---|---|
Offline victimization (reference: none) | ||||||
Suicidal ideation | 2.76 (2.13–3.58) | 2.98 (2.38–3.72) | 3.02 (2.54–3.59) | 2.95 (2.35–3.72) | 2.54 (2.07–3.11) | .7923 |
Suicidal plans | 2.81 (2.25–3.52) | 2.82 (2.23–3.57) | 2.69 (2.12–3.43) | 3.00 (2.30–3.92) | 2.70 (2.18–3.35) | .9824 |
Suicide attempts | 2.17 (1.63–2.91) | 3.09 (2.34–4.09) | 2.63 (1.86–3.72) | 3.88 (2.90–5.19) | 2.62 (1.96–3.49) | .1160 |
Suicide injury | 1.82 (0.77–4.33) | 3.00 (2.01–4.48) | 2.80 (1.66–4.73) | 3.96 (2.28–6.88) | 1.80 (0.93–3.51) | .4202 |
Online victimization (reference: none) | ||||||
Suicidal ideation | 3.34 (2.58–4.32) | 2.84 (2.25–3.59) | 2.79 (2.23–3.49) | 2.79 (2.07–3.77) | 2.07 (1.55–2.76) | .2742 |
Suicidal plans | 2.78 (2.15–3.61) | 2.63 (2.08–3.33) | 2.04 (1.63–2.54) | 2.79 (1.90–4.12) | 2.45 (1.81–3.32) | .3972 |
Suicide attempts | 2.88 (1.99–4.18) | 3.45 (2.48–4.81) | 2.54 (1.88–3.42) | 3.83 (2.48–5.90) | 2.37 (1.64–3.41) | .3755 |
Suicide injury | 3.21 (1.60–6.42) | 3.67 (2.29–5.90) | 3.76 (2.30–6.14) | 3.36 (1.70–6.61) | 3.44 (1.71–6.93) | .9939 |
Both online and offline victimization (reference: none) | ||||||
Suicidal ideation | 5.39 (4.27–6.80) | 5.97 (4.78–7.45) | 7.03 (5.54–8.91) | 5.55 (4.49–6.86) | 4.64 (3.79–5.69) | .1408 |
Suicidal plans | 5.20 (4.12–6.55) | 5.27 (4.18–6.64) | 5.70 (4.28–7.59) | 5.07 (4.03–6.38) | 4.54 (3.72–5.54) | .7655 |
Suicide attempts | 5.15 (4.02–6.60) | 6.95 (5.23–9.24) | 7.16 (5.18–9.90) | 7.40 (5.68–9.63) | 5.69 (4.36–7.43) | .3226 |
Suicide injury | 4.91 (3.01–8.01) | 8.62 (5.71–13.00) | 9.39 (6.09–14.47) | 7.77 (5.36–11.28) | 7.03 (4.18–11.83) | .3723 |
Adjusted for race and ethnicity.
aORs for Suicide Outcomes Based on Experiences of Victimization by Year, Male, 2011–2019 YRBS
Victimization Experience Predictor, Suicidal Behavior Outcome . | 2011, aOR (95% CI) . | 2013, aOR (95% CI) . | 2015, aOR (95% CI) . | 2017, aOR (95% CI) . | 2019, aOR (95% CI) . | P for Interaction Between Year and Victimization . |
---|---|---|---|---|---|---|
Offline victimization (reference: none) | ||||||
Suicidal ideation | 2.52 (1.92–3.31) | 2.69 (2.00–3.63) | 2.81 (2.17–3.64) | 3.36 (2.58–4.37) | 2.82 (2.01–3.94) | .5730 |
Suicidal plans | 2.58 (1.83–3.65) | 2.58 (1.93–3.43) | 1.92 (1.47–2.50) | 3.62 (2.77–4.73) | 3.01 (2.11–4.29) | .0225 |
Suicide attempts | 2.55 (1.73–3.77) | 2.39 (1.53–3.74) | 3.13 (1.98–4.96) | 3.21 (2.01–5,14) | 2.62 (1.61–4.27) | .8827 |
Suicide injury | 1.52 (0.78–2.89) | 2.55 (1.27–5.11) | 3.67 (1.84–7.31) | 4.47 (2.18–9.18) | 3.34 (1.38–8.09) | .2516 |
Online victimization (reference: none) | ||||||
Suicidal ideation | 3.20 (2.24–4.58) | 2.89 (1.69–4.97) | 2.15 (1.33–3.45) | 5.17 (3.46–7.71) | 2.49 (1.52–4.08) | .0431 |
Suicidal plans | 3.69 (2.58–5.27) | 2.92 (1.90–4.48) | 2.82 (1.65–4.82) | 4.03 (2.72–5.96) | 2.70 (1.47–4.93) | .2677 |
Suicide attempts | 5.04 (3.16–8.02) | 5.76 (3.14–10.59) | 4.84 (2.53–9.26) | 4.50 (2.42–8.35) | 3.45 (1.81–6.56) | .8024 |
Suicide injury | 5.17 (2.64–10.13) | 6.04 (3.04–12.01) | 9.87 (3.58–27.24) | 2.26 (0.79–6.44) | 8.67 (3.21–23.45) | .4360 |
Both online and offline victimization (reference: none) | ||||||
Suicidal ideation | 5.35 (3.95–7.25) | 7.87 (5.80–10.66) | 6.44 (5.20–7.97) | 6.69 (4.94–9.05) | 5.84 (4.34–7.86) | .5412 |
Suicidal plans | 5.50 (3.45–8.76) | 6.69 (4.80–9.32) | 7.79 (5.67–10.71) | 5.37 (3.83–7.53) | 5.77 (4.33–7.69) | .6393 |
Suicide attempts | 6.68 (4.78–9.32) | 8.31 (5.89–11.71) | 9.82 (6.22–15.52) | 7.50 (4.54–12.40) | 6.54 (4.25–10.08) | .7819 |
Suicide injury | 3.10 (1.54–6.24) | 15.14 (9.11–25.16) | 15.23 (8.58–27.03) | 12.82 (6.61–24.89) | 8.36 (4.22–16.55) | .0040 |
Victimization Experience Predictor, Suicidal Behavior Outcome . | 2011, aOR (95% CI) . | 2013, aOR (95% CI) . | 2015, aOR (95% CI) . | 2017, aOR (95% CI) . | 2019, aOR (95% CI) . | P for Interaction Between Year and Victimization . |
---|---|---|---|---|---|---|
Offline victimization (reference: none) | ||||||
Suicidal ideation | 2.52 (1.92–3.31) | 2.69 (2.00–3.63) | 2.81 (2.17–3.64) | 3.36 (2.58–4.37) | 2.82 (2.01–3.94) | .5730 |
Suicidal plans | 2.58 (1.83–3.65) | 2.58 (1.93–3.43) | 1.92 (1.47–2.50) | 3.62 (2.77–4.73) | 3.01 (2.11–4.29) | .0225 |
Suicide attempts | 2.55 (1.73–3.77) | 2.39 (1.53–3.74) | 3.13 (1.98–4.96) | 3.21 (2.01–5,14) | 2.62 (1.61–4.27) | .8827 |
Suicide injury | 1.52 (0.78–2.89) | 2.55 (1.27–5.11) | 3.67 (1.84–7.31) | 4.47 (2.18–9.18) | 3.34 (1.38–8.09) | .2516 |
Online victimization (reference: none) | ||||||
Suicidal ideation | 3.20 (2.24–4.58) | 2.89 (1.69–4.97) | 2.15 (1.33–3.45) | 5.17 (3.46–7.71) | 2.49 (1.52–4.08) | .0431 |
Suicidal plans | 3.69 (2.58–5.27) | 2.92 (1.90–4.48) | 2.82 (1.65–4.82) | 4.03 (2.72–5.96) | 2.70 (1.47–4.93) | .2677 |
Suicide attempts | 5.04 (3.16–8.02) | 5.76 (3.14–10.59) | 4.84 (2.53–9.26) | 4.50 (2.42–8.35) | 3.45 (1.81–6.56) | .8024 |
Suicide injury | 5.17 (2.64–10.13) | 6.04 (3.04–12.01) | 9.87 (3.58–27.24) | 2.26 (0.79–6.44) | 8.67 (3.21–23.45) | .4360 |
Both online and offline victimization (reference: none) | ||||||
Suicidal ideation | 5.35 (3.95–7.25) | 7.87 (5.80–10.66) | 6.44 (5.20–7.97) | 6.69 (4.94–9.05) | 5.84 (4.34–7.86) | .5412 |
Suicidal plans | 5.50 (3.45–8.76) | 6.69 (4.80–9.32) | 7.79 (5.67–10.71) | 5.37 (3.83–7.53) | 5.77 (4.33–7.69) | .6393 |
Suicide attempts | 6.68 (4.78–9.32) | 8.31 (5.89–11.71) | 9.82 (6.22–15.52) | 7.50 (4.54–12.40) | 6.54 (4.25–10.08) | .7819 |
Suicide injury | 3.10 (1.54–6.24) | 15.14 (9.11–25.16) | 15.23 (8.58–27.03) | 12.82 (6.61–24.89) | 8.36 (4.22–16.55) | .0040 |
Adjusted for race and ethnicity.
Discussion
This study aimed to estimate the extent to which victimization explained trends in adolescent suicidal outcomes, overall and by sex. For victimization to explain increases in a given outcome, victimization would need to be increasing over time or the strength of the association between victimization and a given suicidal outcome would need to increase over time, but neither situation occurred for the general population, nor did either dynamic apply to male or female adolescents. Victimization has remained relatively stable in prevalence over time, whereas suicidal outcomes, particularly ideation and plans, have increased from 2011 onward (specifically among female adolescents). Although there is a strong relationship between peer victimization and suicidal outcomes, the only heterogeneity in the strength of the association between victimization and a specific outcome in the overall population was observed for co-occurring online and offline victimization and suicidal injury, which is not increasing over time (OR for suicidal injury per 2-year time point: 1.01, 95% CI: 0.98–1.03). Additionally, if peer victimization were contributing to increasing suicidal outcome trends, then the model estimate for year predicting a given outcome should reduce after adjustment for victimization. However, this was not the case for any of the suicidal outcomes.
Still, the strong risk for suicidal ideation, plans, attempts and injury that accompanies both online and offline peer victimization remains a powerful determinant of adolescent health. This is especially true for students being bullied both online and offline who not only faced substantially higher odds of suicidal outcomes compared with those who were not bullied but typically had approximately twice the odds of any given outcome compared with those who were bullied either online only or offline only. Although victimization did not explain the trends in suicidal outcomes, the harmful psychosocial impact of peer victimization cannot be overstated. Our results align with similar findings linking offline and online victimization to suicidal outcomes,19–22 including studies that use smaller subsets of YRBS data,23–25 and suicidal symptom outcomes are just some of the adverse mental health consequences of being victimized, along with depression,26,27 anxiety,21 and low self-esteem.28,29 These psychological factors, together with loneliness and hopelessness, have been hypothesized as possible mediators linking victimization to suicidal outcomes, although the causal structure of these connections is still being explored.30–33
Reducing peer victimization remains a challenging and important goal, particularly as prevalence is remaining flat despite substantial national investment in efforts to reduce peer victimization and bullying. Literature supporting these efforts suggests that programs that are sustained longer are more effective,34 as opposed to single assemblies for bullying awareness.35 Empirically supported components, such as parental engagement and education,36 consistent administrative monitoring and enforcement of programs,36 and bystander training for students37 may be underutilized.
However, there is growing evidence that these programs may not be effective for older adolescents,38 given the shift to more relational (eg, rumors) rather than direct (eg, hitting) forms of victimization and the dynamic nature of social interaction among older adolescents. Intervention effectiveness studies should isolate samples of older adolescents and tailor programs to this population. Additionally, the long-term impact of certain interventions may have unintended consequences of harming certain adolescents, even if the overall environment is healthier.39 This is known as the healthy context paradox and may stem from added self-blame, isolation, or targeting of specific adolescents by bullies as fewer peers face victimization. Reinforcing messaging and norms that reduce victim blame for violence and feelings of isolation (ie, sharing that others have faced similar hardships) while empowering victims to seek help is crucial. Current violence reduction programs are likely insufficient, as evidenced by the stagnant trends in victimization.
Fundamentally, our findings indicate that the patterns of victimization seem to be changing, with many young people being victimized both online and offline, and so interventions must be designed and evaluated to address this co-occurring victimization. With offline victimization, the threat of physical violence exists and so a programmatic emphasis on teaching and enforcing nonviolence is warranted. Progress achieved for more traditional forms of victimization may be offset by new forms of victimization, especially as new digital social platforms provide novel spaces and methods for peer victimization. With online victimization, harm may be more direct without any possibility of intervention from bystanders. Perpetrators may find it easier to bully given the distance and potential anonymity of online harassment, leading to more enduring or persistent victimization. School administrations and parents maintaining open communication about safety online and monitoring issues as they arise are just some potential ways to recognize and handle this problem.
Although connections between victimization and suicidal outcomes are etiologically important for prevention, they do not explain the recent increases in suicidal ideation or plans given that victimization has not increased in the United States. Explanation for suicidal trends, therefore, should focus on other areas. Risk factors such as sleep quality and duration40–43 as well as mood symptoms and affective disorders44 are promising explanatory mechanisms for the increase in suicidal outcomes given trends that coincide with the increase in suicidal ideation and plans.
Limitations
For individual adolescents, the cross-sectional nature of the YRBS does not permit temporal ordering of victimization and suicidal outcomes, preventing causal attribution. Although the hypothesized association is that peer victimization leads to worsened mental health and increased suicidal outcomes, the reverse is plausible, wherein adolescents who are struggling emotionally may face more victimization or may perceive peer interactions as more hostile. These represent different pathways that longitudinal data can disentangle, although both may operate simultaneously.
Suicide items were limited. For suicidal ideation, students only report whether these thoughts are present, not their frequency or intensity. Similarly, victimization in both forms was self-reported as having occurred or not, rather than the frequency or content of the victimization. Offline victimization only dealt with instances on school property, so victimization not on school grounds could not be captured. Lastly, other potentially important covariates, like mental health diagnosis and socioeconomic status, were not available in these data, and so their exclusion from models, while unavoidable, is a limitation of these models.
Certain YRBS years lack data from individual states, limiting the extent to which data are nationally representative. For instance, in 2019, Oregon, Washington, Wyoming, and Minnesota did not participate.13 These data also do not represent students outside of the school systems where surveying occurs and so do not capture homeschooled students or homeless youth not attending school.
Conclusions
Although experiences of victimization predict suicidal ideation, plans, attempts and injury, victimization trends do not explain the worrying increases seen in suicidal ideation and plans. Victimization prevention efforts should be strengthened with the acknowledgment that victimization, both online and offline, is a major risk factor for adolescent suicidal outcomes, especially when these forms of victimization co-occur. To explain the recent increases in suicidal ideation and plans, researchers must look beyond victimization.
Mx. Kreski conducted the analyses, drafted the initial text of the manuscript, and contributed all tables and figures, as well as incorporated coauthor feedback; Dr Keyes conceptualized the study and analyses, as well as critically reviewed and revised the manuscript; Drs Chen, Olfson, Cerdá, Hasin, Martins, and Mauro critically reviewed the manuscript for important intellectual content, interpreted data, and provided direct edits and feedback to strengthen the text; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: Funded by grant R01DA048853 (PI: Keyes) and with support from the Columbia Center for Injury Science and Prevention (R49-CE003094). Additionally, Dr Martins reports funding from grant R01DA037866, Dr Hasin reports funding from grant R01DA048860, and Dr Mauro reports funding from grant K01DA045224. The funder had no role in the design and conduct of the study. Funded by the National Institutes of Health (NIH).
References
Competing Interests
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Comments