BACKGROUND

The coronavirus disease 2019 (COVID-19) pandemic created high levels of psychological distress and may have increased suicide risk.

METHODS

We used the 4-item Ask Suicide-Screening Questions (ASQ) to assess suicide risk among all patients 12 to 24 years of age at a children’s hospital. We compared demographics, encounter type (telehealth or face-to-face [F2F]), and screening results from April to June 2020 (T2) to those from April to June 2019 (T1).

RESULTS

Fewer patients were seen at T2 than T1 (17 986 vs 24 863). A greater proportion of visits at T2 were by telehealth (0% vs 43%). The rate of positive suicide screens was higher in T2 than in T1 (12.2% vs 11.1%, adjusted odds ration [aOR], 1.24; 95% confidence interval [CI], 1.15–1.35). The odds of a positive screen were greater for older patients (aOR of 1.12 for age in years; 95% CI, 1.10–1.14), female patients (aOR, 2.23; 95% CI, 2.00–2.48), patients with public versus private insurance (aOR, 1.88; 95% CI, 1.72–2.07), and lower for Black versus White patients (aOR, 0.85; 95% CI, 0.77–0.95). Rates of positive screens were highest among inpatients (20.0%), intermediate for emergency department patients (14.4%), and lowest in outpatient clinics (9.9%) (P < .05).

CONCLUSIONS

Rates of positive suicide risk screens among adolescents rose in the pandemic’s early months with differences related to sociodemographics and visit type. Changes in health care delivery highlight the complexities of assessing and responding to mental health needs of adolescents. Additional research might determine the effects of screening methods and patient populations on screening results.

What’s Known on This Subject

Previous studies yielded contradictory results regarding suicidal ideation among teenagers during the coronavirus disease 2019 (COVID-19) pandemic. Rates were higher in some studies and the same or lower in others. Most studies were in emergency departments or primary care clinics.

What This Study Adds

This study, which included patients from all clinical settings, shows that during the pandemic’s early months, there were fewer patient visits, higher rates of suicidal ideation, and differences in rates of positive screens in different clinical settings and encounter types.

Suicide is the second leading cause of death for Americans between the ages of 10 and 24. Rates of suicide in this age group climbed 56% between 2007 and 2017.1  The 2019 Youth Risk Behavior Survey reported that 18.8% of high school students in the United States had seriously considered suicide, and 8.9% had made a suicide attempt in the past year.2 

According to the National Institute of Mental Health, the best way to prevent suicide is to identify people who are at elevated risk and link them to care.3  Health care providers are in a unique position to do this. Most individuals who die by suicide have recently seen a health care provider.4  However, their presuicide health encounter was usually not motivated by mental health concerns. Furthermore, most clinicians do not routinely screen teenagers for suicidal thoughts.5 

To identify youth who might be at risk for suicide, our children’s hospital implemented a universal suicide risk screening program in which we screen all patients ≥12 years old who are hospitalized or have health care encounters in ambulatory clinics, urgent care, or the emergency department. The screening program began in 2018.6  Data from the screening program allow us to compare rates of positive screens before and after the onset of the coronavirus disease 2019 (COVID-19) pandemic among patients seen in different care settings.

The COVID-19 pandemic led to increases in anxiety, depression, irritability, anger, and sleep disturbances among adolescents.79  The impact on suicide risk is not as clear. Some early reports suggested a rise in suicide risk. The Centers for Disease Control and Prevention reported higher rates of suicidal ideation in the spring of 2020 compared to 2018 and 2019.10  Hill et al reported that, among teenagers seen in their emergency department (ED), the odds of recent suicide ideation were 1.45 to 1.60 times higher in the spring of 2020 compared to 2019.11  They cautioned, however, that the overall number of ED visits had decreased in 2020 and so patient self-selection may have been a factor. Mayne and colleagues gathered data from a large pediatric primary care network and also observed increases in the number of adolescents screening positive for depressive symptoms and suicide risk during the pandemic.12  In contrast, Kemper et al studied the prevalence of depression and suicidal ideation among patients seen in 12 pediatric primary care practices before the COVID-19 pandemic and during the early months of the pandemic and found no increase.13 

We wanted to understand the effect of the COVID-19 pandemic on suicide risk among adolescents who received care in our health system. The COVID-19 pandemic caused increased psychological stress for many people.10,14  The COVID-19 pandemic also led to dramatic changes in hospital services and in health-seeking behavior. For example, fewer patients came to clinics or to EDs, and more visits were conducted by telemedicine. With these factors in mind, in our study we had 2 aims: First, we sought to compare rates of adolescent suicide risk in the early months of the pandemic to those in the previous year. Second, we evaluated the association between encounter types and settings and rates of positive suicide screens.

Children’s Mercy Kansas City initiated universal suicide screening in 2018. We conducted over 100 000 screens per year in 2019 and 2020. Patients are eligible for screening if they are 12 years or older and do not have a medical contraindication (eg, urgent medical problems, severe pain, nonverbal, etc) to completing a screen.

Data on suicide risk were based on the “Ask Suicide-Screening Questions” (ASQ).2,15  The ASQ is a freely available 4-item screening tool originally developed and validated for use in assessing suicide risk in pediatric emergency departments. The tool has since been further validated for use in pediatric inpatient and outpatient settings for ages 10 to 21, with sensitivity values ≥95% and specificity values ≥87%.3,4  The ASQ is administered independently with the patient (ie, not with parents present), unless the patient’s parent or guardian is opposed to the patient being screened without their presence. The ASQ items inquire about wishes to be dead, feelings of oneself or others being better off if dead, thoughts of killing oneself, and previous attempts at killing oneself. If a patient’s response is “no” to all 4 items, this is considered a negative screen and no further questions are asked unless clinical judgement overrides the screening result. If a patient responds “yes” to any item or refuses to answer, the screen is considered “positive” and a fifth question is asked to determine acuity (“Are you having thoughts of killing yourself right now?”). A “yes” answer or refusal to answer this fifth question is defined as an “acute positive screen.” The ASQ has been validated for administration by a variety of clinicians in a wide range of clinical settings. We are unaware of a validation study specifically focused on telemedicine.

For any positive screen, further safety assessment and safety precautions matched to risk level were implemented by social workers according to a hospital protocol (eg, safety planning including means restriction education, referral to outpatient care or transfer to inpatient care). This occurred for both face-to-face (F2F) visits and for telehealth visits.

For patients who had a documented past attempt from a previous ASQ screen, the ASQ was modified to ask whether there had been a suicide attempt since their last screening. If not, then the screen results were classified as negative.

Suicide risk screening is administered by nurses or care assistants who ask the screening questions verbally. Screening results, along with key demographic and clinical variables in the electronic health record, are deidentified by an “honest broker”12  and stored in a data repository that is available for research inquires. Data from this repository were used for the current study. Data were analyzed from individuals ages 12 to 24 years old (inclusive) having encounters occurring between April to June of 2019 (T1) and April to June of 2020 (T2).

We excluded patients who were seen in mental health clinics because the suicide risk assessment in those clinics follows a different protocol. We also excluded patients who had encounters for which suicide screening was omitted due to medical contradictions.

Within the specified study windows, some patients had multiple encounters and, in some encounters (eg, inpatient visits), some patients had multiple suicide screens. For patients with multiple encounters and/or screens in T1 or T2, we selected the first suicide screening of the first encounter to keep the unit of analysis at the patient level. Fields that were extracted from the repository for study analyses included patient demographic variables (age at the encounter, self-reported sex and race), encounter characteristics including the type of visit (F2F or telemedicine), the patient’s insurance type (public, private, or self-pay), the month in which screening took place at the encounter, and the patient’s answers to the suicide risk assessment questions. The Children’s Mercy institutional review board has determined research involving these deidentified data to be “not human subjects research.”

Frequency counts and percentages were used to summarize data on prevalence of suicide risk across medical settings. Considering some patients were seen in both periods (T1 and T2), patient-specific random intercepts were applied in generalized linear mixed models (GLMM) to evaluate the association of suicide risk and the COVID-19 pandemic. These analyses were carried out adjusting for the following covariates: encounter month, encounter type, patient age in years, patient sex, race, and type of insurance the family had at the time of the given encounter (the main effects model hereinafter). Our intention to evaluate the pandemic-by-visit-type interaction was not feasible because of virtually no telehealth visits before the pandemic. Instead, we fitted a separate GLMM to compare suicide risk among the year-by-visit type combinations (2019 FTF, 2020 FTF, and 2020 telehealth), adjusting covariates of month, age, sex, race, and insurance type (the pandemic-by-visit combination model hereinafter). GLMM parameters were estimated by adaptive Gaussian Quadrature by using 11 quadrature points. Because GLMM provided patient-specific estimates, the fixed-effect parameters were marginalized to give population-averaged interpretation16  and presented in adjusted odds ratio (aOR) and the corresponding 95% confidence interval (CI). The analysis was conducted in R version 4.0.2 by using the GLMMadaptive package version 0.8-0.17 

Wilcoxon rank test and χ2 tests were used as applicable to evaluate the association of screening status (screened or not) with patient demographics with standardized mean differences (SMD) of at least 0.2 used as a threshold for identifying group differences.

The inclusion and exclusion criteria led to analyzing records of 22 896 patients at T1 and 14 635 patients at T2. Of this total number of patients across time points, 32 728 were unique and 4803 had encounters during both the T1 and T2 time periods. Of the patients analyzed for T1 and T2, 9 (0.0%) and 6359 (43.5%) were seen via telemedicine visits.

Suicide screening was conducted in 19 265 of 22 896 (84.1%) and 9984 of 14 635 (68.2%) patients in T1 and T2. The proportion of eligible patients who were screened in the two years was similar among F2F visits (84.1% vs 85.7%). For telehealth visits in T2, 2891 of 6359 (45.5%) were screened (Fig 1).

FIGURE 1

Study population and key variables. Percentage based on the N from 1 level above (eg, among the 2020 visits, telemedicine was 6359 of 14 635 [43.5%] and the screen rate was 19 261 of 22 887 [84.2%] among the 2019 face-to-face visits).

FIGURE 1

Study population and key variables. Percentage based on the N from 1 level above (eg, among the 2020 visits, telemedicine was 6359 of 14 635 [43.5%] and the screen rate was 19 261 of 22 887 [84.2%] among the 2019 face-to-face visits).

Close modal

Demographic characteristics of screened patients were similar across the 2 years (Table 1), but White patients and patients with private insurance were overrepresented among patients screened by telehealth at T2 (Table 2).

TABLE 1

Characteristics of Screened Patients Between Years

Year 2019 (N = 19 265)Year 2020 (N = 9984)Pa
Age, median (IQR) 15 (13–16) 15 (13–16) <.01 
Sex, n (%)   <.01 
 Female 10 355 (53.8) 5540 (55.5)  
 Male 8906 (46.2) 4443 (44.5)  
 Missing 4 (0.0) 1 (0.0)  
Race, n (%)   .33 
 White 11 649 (60.5) 6021 (60.3)  
 Black 3395 (17.6) 1806 (18.1)  
 Other 3980 (20.7) 1995 (20.0)  
 Missing 241 (1.3) 162 (1.6)  
Primary insurance type, n (%)   <.01 
 Private 10 553 (54.8) 5216 (52.2)  
 Public 7447 (38.7) 4142 (41.5)  
 Self-pay 1263 (6.6) 626 (6.3)  
 Missing 2 (0.0)  
Screen result, n (%)   <.01 
 Negative 17 118 (88.9) 8761 (87.8)  
 Positive 2147 (11.1) 1223 (12.2)  
Year 2019 (N = 19 265)Year 2020 (N = 9984)Pa
Age, median (IQR) 15 (13–16) 15 (13–16) <.01 
Sex, n (%)   <.01 
 Female 10 355 (53.8) 5540 (55.5)  
 Male 8906 (46.2) 4443 (44.5)  
 Missing 4 (0.0) 1 (0.0)  
Race, n (%)   .33 
 White 11 649 (60.5) 6021 (60.3)  
 Black 3395 (17.6) 1806 (18.1)  
 Other 3980 (20.7) 1995 (20.0)  
 Missing 241 (1.3) 162 (1.6)  
Primary insurance type, n (%)   <.01 
 Private 10 553 (54.8) 5216 (52.2)  
 Public 7447 (38.7) 4142 (41.5)  
 Self-pay 1263 (6.6) 626 (6.3)  
 Missing 2 (0.0)  
Screen result, n (%)   <.01 
 Negative 17 118 (88.9) 8761 (87.8)  
 Positive 2147 (11.1) 1223 (12.2)  

IQR, interquartile range.

a

 Excluded missing values and ignored within-patient dependency; Wilcoxon rank test for age and χ2 test for the other variables.

TABLE 2

Characteristics of Screened Patients Between Year-by-Visit Combinations

2019 F2F (N =19 261)2019 Telehealth (N = 4)2020 F2F (N = 7093)2020 Telehealth (N = 2891)Pa
Age, median (IQR) 15 (13–16) 18 (17–19) 14 (13–16) 15 (13–16) <.01 
Sex, n (%)     <.01 
 Female 10 351 (53.7) 4 (100) 3849 (54.3) 1691 (58.5)  
 Male 8906 (46.2) 3243 (45.7) 1200 (41.5)  
 Missing 4 (0.0) 1 (0.0)  
Race, n (%)     <.01 
 White 11 645 (60.5) 4 (100) 3944 (55.6) 2077 (71.8)  
 Black 3395 (17.6) 1492 (21.0) 314 (10.9)  
 Other 3980 (20.7) 1523 (21.5) 472 (16.3)  
 Missing 241 (1.3) 134 (1.9) 28 (1.0)  
Primary insurance type, n (%)     <.01 
 Private 10 550 (54.8) 3 (75.0) 3462 (48.8) 1754 (60.7)  
 Public 7446 (38.7) 1 (25.0) 3114 (43.9) 1028 (35.6)  
 Self-pay 1263 (6.6) 517 (7.3) 109 (3.8)  
 Missing 2 (0.0)  
Screen result, n (%)     <.01 
 Negative 17 114 (88.9) 4 (100) 2583 (87.1) 2583 (89.3)  
 Positive 2147 (11.1) 915 (12.9) 308 (10.7)  
2019 F2F (N =19 261)2019 Telehealth (N = 4)2020 F2F (N = 7093)2020 Telehealth (N = 2891)Pa
Age, median (IQR) 15 (13–16) 18 (17–19) 14 (13–16) 15 (13–16) <.01 
Sex, n (%)     <.01 
 Female 10 351 (53.7) 4 (100) 3849 (54.3) 1691 (58.5)  
 Male 8906 (46.2) 3243 (45.7) 1200 (41.5)  
 Missing 4 (0.0) 1 (0.0)  
Race, n (%)     <.01 
 White 11 645 (60.5) 4 (100) 3944 (55.6) 2077 (71.8)  
 Black 3395 (17.6) 1492 (21.0) 314 (10.9)  
 Other 3980 (20.7) 1523 (21.5) 472 (16.3)  
 Missing 241 (1.3) 134 (1.9) 28 (1.0)  
Primary insurance type, n (%)     <.01 
 Private 10 550 (54.8) 3 (75.0) 3462 (48.8) 1754 (60.7)  
 Public 7446 (38.7) 1 (25.0) 3114 (43.9) 1028 (35.6)  
 Self-pay 1263 (6.6) 517 (7.3) 109 (3.8)  
 Missing 2 (0.0)  
Screen result, n (%)     <.01 
 Negative 17 114 (88.9) 4 (100) 2583 (87.1) 2583 (89.3)  
 Positive 2147 (11.1) 915 (12.9) 308 (10.7)  

IQR, interquartile range.

a

Excluded 2019 telehealth and missing values and ignored within-patient dependency; Kruskal-Wallis test for age and χ2 test for the other variables.

The overall rate of positive screens was higher in T2 than in T1 (12.2% vs 11.1%, aOR, 1.24; 95% CI, 1.15–1.35; Fig 2, year: 2020 vs 2019). For patients seen F2F, the proportion of positive screens was similarly higher in T2 than in T1 (12.9% vs 11.1%, aOR, 1.24; 95% CI, 1.14–1.35; Fig 2, 2020 F2F vs 2019 F2F). There was a lower rate of positive screens among patients seen in telehealth visits than among patients seen in F2F visits (10.7% vs 12.9%, aOR, 0.75; 95% CI, 0.66–0.87; Fig 2, 2020 telehealth vs 2020 F2F) in T2. When we compared positivity on telehealth visits to those in outpatient visits during the previous year, the positivity rate was higher for 2020 telehealth visits than for 2019 outpatient visits (10.7% vs 9.7%, aOR, 1.11; 95% CI, 0.97–1.26).

FIGURE 2

Effects of clinical, temporal, and sociodemographic variables on suicide risk. The adjusted odds ratios for the effect of different factors (encounter type, month, patient age, race, sex, and insurance type) on positive screen results. Odds ratios are along the x-axis. Adjusted odds ratios are estimated with contrasts of interest in year (red) and in year-by-encounter-type (black). Solid and hollow circles indicate P < .05 and P > .05, respectively. 2020.F2F, positive screen for suicide risk in face-to-face visits during 2020; 2019.F2F, positive screen for suicide risk in face-to-face visits during 2019; 2020.TELE, positive screen for suicide risk in telemedicine visits during 2020.

FIGURE 2

Effects of clinical, temporal, and sociodemographic variables on suicide risk. The adjusted odds ratios for the effect of different factors (encounter type, month, patient age, race, sex, and insurance type) on positive screen results. Odds ratios are along the x-axis. Adjusted odds ratios are estimated with contrasts of interest in year (red) and in year-by-encounter-type (black). Solid and hollow circles indicate P < .05 and P > .05, respectively. 2020.F2F, positive screen for suicide risk in face-to-face visits during 2020; 2019.F2F, positive screen for suicide risk in face-to-face visits during 2019; 2020.TELE, positive screen for suicide risk in telemedicine visits during 2020.

Close modal

The odds of a positive screen increased by 12% with each additional year increment in age (Fig 2, aOR, 1.12; 95% CI, 1.10–1.14). When dichotomized into <16 or ≥16 age groups, a significant difference in positive screen rates among older adolescents as compared to younger ones (15.3% vs 10.6% in 2020, Table 3) was observed. The odds of a positive screen were higher for female patients relative to male patients (aOR, 2.23; 95% CI, 2.00–2.48), lower in Black patients (aOR, 0.85; 95% CI, 0.77–0.95) and patients of other races (aOR, 0.82; 95% CI, 0.74–0.91) relative to White patients, and higher among patients with public insurance (aOR, 1.88; 95% CI, 1.72–2.07) relative to those with private insurance. These covariate effects were similar between the models that contrasted year and year-by-visit (Fig 2).

TABLE 3

Positive Screening Across Demographic and Encounter Types

2019 n/N (%)2020 n/N (%)Pa
White 1264/11 649 (10.9) 693/6021 (11.5) .19 
Black 420/3395 (12.4) 245/1806 (13.6) .24 
Other races 430/3980 (10.8) 250/1995 (12.5) .05 
Female 1525/10 355 (14.7) 877/5540 (15.8) .07 
Male 622/8906 (7.0) 346/4443 (7.8) .10 
Age <16 1141/12 059 (9.5) 680/6438 (10.6) .02 
Age ≥16 1006/7206 (14.0) 543/3546 (15.3) .07 
Private 926/10 553 (8.8) 488/5216 (9.4) .24 
Public 1072/7447 (14.4) 638/4142 (15.4) .15 
Self-pay 149/1263 (11.8) 87/626 (15.5) .03 
OUTP (all) 1244/12 788 (9.7) 657/6472 (10.2) .37 
OUTP (F2F) 1244/12 784 (9.7) 349/3581 (9.7)a .9999 
OUTP (TELE) 0/4 308/2891 (10.7)a .9999 
ED 832/6082 (13.7) 498/3164 (15.7) <.01 
INP 66/338 (19.5) 65/316 (20.6) .81 
2019 n/N (%)2020 n/N (%)Pa
White 1264/11 649 (10.9) 693/6021 (11.5) .19 
Black 420/3395 (12.4) 245/1806 (13.6) .24 
Other races 430/3980 (10.8) 250/1995 (12.5) .05 
Female 1525/10 355 (14.7) 877/5540 (15.8) .07 
Male 622/8906 (7.0) 346/4443 (7.8) .10 
Age <16 1141/12 059 (9.5) 680/6438 (10.6) .02 
Age ≥16 1006/7206 (14.0) 543/3546 (15.3) .07 
Private 926/10 553 (8.8) 488/5216 (9.4) .24 
Public 1072/7447 (14.4) 638/4142 (15.4) .15 
Self-pay 149/1263 (11.8) 87/626 (15.5) .03 
OUTP (all) 1244/12 788 (9.7) 657/6472 (10.2) .37 
OUTP (F2F) 1244/12 784 (9.7) 349/3581 (9.7)a .9999 
OUTP (TELE) 0/4 308/2891 (10.7)a .9999 
ED 832/6082 (13.7) 498/3164 (15.7) <.01 
INP 66/338 (19.5) 65/316 (20.6) .81 
a

P value obtained from the χ2 test.

Rates of F2F positive screens in T1 and T2, also differed by clinical settings. For both T1 and T2, respectively, patients who were screened in the inpatient setting had the highest rate of positive suicide screens (18.6% vs 20.0%), followed by patients screened in the ED (13.7% vs 15.7%) and patients screened in outpatient clinics (9.7% vs 9.9%). These differences between clinical settings were statistically significant in the combined 2-year data and in each year (P < .01).

The early months of the COVID-19 pandemic were psychologically stressful for children and adolescents. Children and adolescents reportedly had higher rates of anxiety and depression during the pandemic than in the previous year.7,8  Studies produced conflicting results regarding suicide risk level during the pandemic. Nevertheless, the clear evidence that more adolescent patients are experiencing increased anxiety and depression gives reason for concern about the possibility of more suicide attempts.8,9,11,18  Results of these studies are difficult to interpret because so many things changed during the early months of the pandemic. Fewer patients were seen in emergency departments and outpatient clinics, more visits were done by telemedicine, and some patients had better access to broadband internet (and thus both telemedicine and remote schooling) than others.

Patients seen F2F had higher rates of positive screens for suicide risk in 2020 compared to those seen in 2019. Patients seen by telemedicine, however, had lower rates. Furthermore, screening was completed in a lower percentage of patients seen by telemedicine than in those seen F2F. These results could be interpreted in a number of ways. Perhaps, in the early months of the pandemic, teenagers with higher levels of psychological stress were more likely to seek a F2F health visit. It could also reflect the recommendation of a health care provider for a F2F visit when a patient is already known to be high-risk. Such patient selection could have affected the rates of positive screening if sicker patients were more likely to be seen F2F. It is also possible that patients with access to broadband and thus telemedicine were less socially isolated and thus less psychologically stressed than patients who sought F2F visits. Higher rates of positive screens during F2F visits could also reflect better communication and trust during F2F visits leading to an increased willingness to disclose suicidal thoughts during the screening process. Future quality improvement efforts should focus on assessing and comparing quality in F2F and telehealth visits and ensuring that quality is comparable for the two types of clinical encounters. We speculate but cannot prove from this data set that some of the difference in rates of positive screens between F2F and telehealth visits could reflect the fact that F2F visits were more likely in clinical encounters that took place in higher-risk clinical situations such as the ED or inpatient units.

There were more “missed” screening opportunities during 2020 for telehealth encounters. Transition to telehealth was somewhat abrupt, and many clinicians were unsure in those early months how to best incorporate screening into this new clinic flow or no longer had access to nursing support in the same way during these visits. We cannot determine if our results would have remained the same if similar proportions of patients had been screened via telehealth visits in 2020.

Previous studies have shown lower rates of suicidal ideation in Black patients than in White patients.1,2,9,17  This is changing. Increased risk among Black youth of all ages has been reported, with not only increased thoughts and behaviors, but higher rates of suicide deaths for Black children and adolescents compared to their White peers.19  The COVID-19 pandemic has had a disproportionate negative effect on minority communities; therefore, we thought it important to examine the effect of race on suicide screening results. Our results showed a relatively lower rate of positive suicide screens for Black patients relative to White patients. Further research is needed to understand what factors may lead to this finding, examining factors such as trust, racial discordance between patient and provider, and stigma associated with mental health problems.

Our study has several limitations. As noted above, we do not know how patients chose or were directed to F2F versus telehealth visits so we can only speculate about the possibility of selection bias. In addition, this was a study from the first 3 months of the pandemic. Our hospital, along with the rest of the country, was in crisis mode. Thus, we may not have been as skilled at telemedicine as we should have been. This may have led to lower rates of screening in patients who sought care by telemedicine. Further study should examine whether the patterns seen in this study persisted in later months of the pandemic. Finally, as noted, the ASQ has not been validated for telemedicine encounters. The study also has strengths. It differs from most previous reports in that it includes data from inpatient, outpatient, and ED visits. Ours is also the first to compare results on F2F visits to those from telehealth visits. Another strength is that we used screening data rather than relying on discharge diagnoses or previous reports of behavioral problems. Thus, we were more likely to identify children without known risk factors or previous reports of suicidal thoughts. Our results suggest important factors to be considered in both implementing and evaluating suicide screening programs.

Further research should examine trends for suicide risk during the later months of the pandemic. Research should also determine whether overall rates of suicide among teenagers increased during these months. Such data should help assess the efficacy of screening programs and clarify the effect of the COVID-19 pandemic on suicide risk and suicide among teenagers. These data could guide efforts to address the mental health needs of adolescents in the aftermath of the pandemic or during future similar public health crises.

We thank the Suicide Risk Assessment Task Force at Children’s Mercy Kansas City and the following collaborating stakeholders at Children’s Mercy Kansas City for their contributions to the design and implementation of the original clinical data collection and/or data extraction procedures from which our study data were derived: Children’s Mercy Research Institute Population Health; Clinical Informatics; Department of Accreditation and Regulatory Readiness; Department of Social Work; Division of Developmental and Behavioral Sciences; Medical Informatics; Nursing and the Patient Care Services Department.

FUNDING: This work was supported by Children’s Mercy Kansas City and in part by philanthropic support from Laura’s Fund.

Dr Lantos, Mr Raza, Dr Connelly, Dr Goggin, and Dr Sullivant helped to conceptualize the project, analyze the data, and write and revise the manuscript; Dr Yeh helped to conceptualize the project, analyze the data, and revise the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

aOR

adjusted odds ratio

ASQ

Ask Suicide-Screening Questions

CI

confidence interval

COVID-19

coronavirus disease 2019

ED

emergency department

F2F

face-to-face

GLMM

generalized linear mixed models

SMD

standardized mean differences

1
Miron
O
,
Yu
KH
,
Wilf-Miron
R
,
Kohane
IS
.
Suicide rates among adolescents and young adults in the United States, 2000–2017
.
JAMA
.
2019
;
321
(
23
):
2362
2364
2
Centers for Disease Control
.
Youth risk behavior surveillance, United States, 2019
.
MMWR, Aug 21, 2020. Available at: https://www.cdc.gov/mmwr/volumes/69/su/pdfs/su6901-H.pdf. Accessed April 22, 2021
3
National Institute of Mental Health
.
ASQ Toolkit
.
4
Ahmedani
BK
,
Simon
GE
,
Stewart
C
, et al
.
Health care contacts in the year before suicide death
.
J Gen Intern Med
.
2014
;
29
(
6
):
870
877
5
Frankenfield
DL
,
Keyl
PM
,
Gielen
A
,
Wissow
LS
,
Werthamer
L
,
Baker
SP
.
Adolescent patients--healthy or hurting? Missed opportunities to screen for suicide risk in the primary care setting
.
Arch Pediatr Adolesc Med
.
2000
;
154
(
2
):
162
168
6
Sullivant
SA
,
Brookstein
D
,
Camerer
M
, et al
.
Implementing universal suicide risk screening in a pediatric hospital
.
Jt Comm J Qual Patient Saf
.
2021
;
47
(
8
):
496
502
7
Panchal
U
,
Salazar de Pablo
G
,
Franco
M
, et al
.
The impact of COVID-19 lockdown on child and adolescent mental health: systematic review
.
Eur Child Adolesc Psychiatry
.
2021
;
1
27
8
Panda
PK
,
Gupta
J
,
Chowdhury
SR
, et al
.
Psychological and behavioral impact of lockdown and quarantine measures for COVID-19 pandemic on children, adolescents and caregivers: a systematic review and meta-analysis
.
J Trop Pediatr
.
2021
;
67
(
1
):
fmaa122
9
Racine
N
,
McArthur
BA
,
Cooke
JE
,
Eirich
R
,
Zhu
J
,
Madigan
S
.
Global prevalence of depressive and anxiety symptoms in children and adolescents during COVID-19: a meta-analysis
.
JAMA Pediatr
.
2021
;
175
(
11
):
1142
1150
10
Czeisler
ME
,
Lane
RI
,
Petrosky
E
, et al
.
Mental health, substance use, and suicidal ideation during the COVID-19 pandemic. United States, June 24-30, 2020
.
MMWR Morb Mortal Wkly Rep
.
2020
;
69
(
32
):
1049
1057
11
Hill
RM
,
Rufino
K
,
Kurian
S
,
Saxena
J
,
Saxena
K
,
Williams
L
.
Suicide ideation and attempts in a pediatric emergency department before and during COVID-19
.
Pediatrics
.
2021
;
147
(
3
):
e2020029280
12
Mayne
SL
,
Hannan
C
,
Davis
M
, et al
.
COVID-19 and adolescent depression and suicide risk screening Outcomes
.
Pediatrics
.
2021
;
148
(
3
):
e2021051507
13
Kemper
AR
,
Hostutler
CA
,
Beck
K
,
Fontanella
CA
,
Bridge
JA
.
Depression and suicide-risk screening results in pediatric primary care
.
Pediatrics
.
2021
;
148
(
1
):
e2021049999
14
Centers for Disease Control
.
COVID-19: support for teens and young adults
.
Available at: https://www.cdc.gov/coronavirus/2019-ncov/daily-life-coping/stress-coping/young-adults.html. Published online Dec 30, 2020. Accessed April 22, 2021
15
Horowitz
LM
,
Bridge
JA
,
Teach
SJ
, et al
.
Ask Suicide-Screening Questions (ASQ): a brief instrument for the pediatric emergency department
.
Arch Pediatr Adolesc Med
.
2012
;
166
(
12
):
1170
1176
16
Hedeker
D
,
du Toit
SHC
,
Demirtas
H
,
Gibbons
RD
.
A note on marginalization of regression parameters from mixed models of binary outcomes
.
Biometrics
.
2018
;
74
(
1
):
354
361
17
GLMMadaptive: generalized linear mixed models using Adaptive Gaussian Quadrature
.
R package version 0.8-0
.
18
Yard
E
,
Radhakrishnan
L
,
Ballesteros
MF
, et al
.
Emergency department visits for suspected suicide attempts among persons aged 12-25 years before and during the COVID-19 pandemic – United States, January 2019-May 2021
.
MMWR Morb Mortal Wkly Rep
.
2021
;
70
(
24
):
888
894
19
Xiao
Y
,
Cerel
J
,
Mann
JJ
.
Temporal trends in suicidal ideation and attempts among US adolescents by sex and race/ethnicity, 1991-2019
.
JAMA Netw Open
.
2021
;
4
(
6
):
e2113513

Competing Interests

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no conflicts of interest relevant to this article to disclose.