Improving Youth Suicide Risk Screening and Assessment in a Pediatric Hospital Setting by Using The Joint Commission Guidelines

A workgroup of key stakeholders was assembled in October 2018 in response to this review to develop a new suicide risk screening process. This group included representatives from the divisions of psychiatry, psychology, and emergency medicine; nursing informatics; and the department of social work and met weekly to review progress and assign tasks to each subgroup.

Relevant literature was reviewed to guide policy and procedure development. A projected time line was created (Fig 1) with the goal to initiate the new process in the ED and OPC before the anticipated TJC visit in July 2019 and allow a 30-day gap between phase 1 and phase 2 to address initial problems.

The proposal was initially met with several concerns from hospital leadership, including increased burden on mental health clinicians in the ED, potential negative impact on ED LOS, lack of support staff to provide safety interventions, acceptability of screening to patients, and the lack of evidence that screening prevents suicide. Despite these concerns, buy-in was obtained because of the TJC policy (NPSG 15.01.01)⁹  on mandatory screening. To address the concern for delays in care and shortage of staff in the ED, a simulation model for how patients would be screened and assessed was developed. Notional values for the duration of each step of medical and behavioral health evaluation were used, incorporating current resources (such as staff and secure ED beds) available to patients with behavioral health complaints. Outcome estimates included overflow in the ED to non–behavioral health beds, mean wait times for these patients, and their mean ED LOS. These estimates revealed that a phased approach model¹²  targeting a smaller population at a time would have a less drastic impact on overflow and ED LOS. Approval for 2 additional positions for ED social workers, 5 additional behavioral health technicians for safety interventions, and 1 nursing informatics staff member to help with electronic health record (EHR) building was obtained.

The model had 3 phases: phase 1, screening of patients presenting to the ED with behavioral health complaints; phase 2, screening of patients presenting to the behavioral health OPC; and phase 3, screening all non–behavioral health patients presenting to any hospital setting. In this study, we describe phases 1 and 2.

A review of existing TJC guidelines at the commencement of this project revealed no mandate related to age (an update published by TJC in November 2019 now recommends using a validated screening tool for ages ≥12 years). Presuming a higher burden of suicide risk among the behavioral health population,^16,17  a lower limit of 6 years was chosen for this subgroup (phases 1 and 2) compared to a lower limit of 10 years for future non–behavioral health screening (phase 3). Although suicide in children <12 years is rare, it has been reported as the 10th most common cause of death^1,2  and is more likely to occur in Black children in the elementary school age group.¹⁸  In 2019, the National Violent Death Reporting statistics revealed a suicide rate of 5 of 1 000 000 for 5- to 9-year-olds.³ 

The factors considered by the workgroup were (1) the length of the tool, (2) the strength of psychometric properties in the pediatric population, and (3) the ability to stratify risk to guide safety interventions. TJC R3 report on NPSG 15.01.01 for Suicide Prevention⁸  recommends the Suicide Behavior Questionnaire–Revised,¹⁹  the Patient Health Questionnaire–9 (PHQ-9), the Patient Safety Screener,²⁰  the ASQ tool,²¹  or the C-SSRS²²  as possible options for screening tools for hospitals to use. For risk assessment, TJC recommends the C-SSRS Risk Assessment version,²²  The Beck Scale for Suicide Ideation,²³  and the Scale for Suicide Ideation–Worst.²⁴ 

Final contenders for suicide risk screening tools were the PHQ-9, the ASQ, and the C-SSRS given that these are most commonly used in the pediatric population (Supplemental Table 3). The PHQ-9 was not chosen because it is not specific to suicide risk screening. Both the C-SSRS and ASQ are validated in children and in triage risk. The C-SSRS tool was chosen because it stratified risk to low, moderate, and high to guide safety interventions, and the TJC recommends the C-SSRS Risk Assessment version to guide assessment.

The C-SSRS screener is a structured 6-question tool that screens for suicide risk by asking questions about thoughts, intent, plan, and behaviors over the past month and about any attempts over the past 3 months or the lifetime. It scores risk as high, medium, or low depending on affirmative answers (Fig 2). For further assessment, the C-SSRS Full version asks about the intensity of ideation and the severity of behavior, including assessing actual or potential lethality, and the risk assessment page provides a checklist of risk and protective factors. Studies of internal validity, sensitivity, and specificity of the screener have been conducted in patients 11 years or older^20,22 ; however, in several studies, the C-SSRS has been used for patients as young as 5 years old.²⁵  Two versions of the C-SSRS screener, varying in time line for symptoms and both available online,²⁶  were incorporated into the EHR: (1) the C-SSRS Recent, which assesses risk over the past month, and (2) the C-SSRS Since Last Asked, which assesses risk since last assessed. The latter was used for rescreening in OPCs and daily rescreening of patients admitted to psychiatry or medical inpatient units. A full assessment, which combined the C-SSRS Full and Risk Assessment pages to meet TJC standard, was also built into the EHR.

The key stakeholders collaborated to draft a process guide, which included both a written component as well as a visual algorithm in line with consensus recommendations from the Pathways in Clinical Care Group.¹⁴  The Pathways in Clinical Care Group clinical pathway was modified, with permission, according to the tool and interventions used (Supplemental Fig 4).

The ED is a 90-bed level 4 trauma center with 4 beds in a dedicated, locked behavioral health section and 5 behavioral health overflow beds in an unlocked area. Approximately 2000 patients a year present with a behavioral health complaint to the ED. The behavioral health section was staffed 24/7 by 1 emergency physician, 2 psychiatry social workers during peak hours, and 1 psychiatry social worker during nonpeak hours. One child and adolescent psychiatrist was on-site from 8 am to 5 pm. In addition, the ED had limited coverage by 1 medical social worker who traditionally saw families of patients coming in for medical complaints. The psychiatry social workers had past training and experience in treating patients with behavioral health concerns, whereas the medical social workers did not. Patients presenting with behavioral health complaints received a face-to-face evaluation by the psychiatry social worker, supervised in person or over the phone by a child and adolescent psychiatrist. Constant observation for safety, as needed, was provided by behavioral health technicians who had a high school diploma and training in crisis de-escalation.²⁷ 

Phase 1 went live on May 2, 2019. The C-SSRS was administered in the EHR by the triage nurse, preferably with the patient alone. Patients who refused to answer were triaged as moderate risk for further assessment. Each risk level automatically triggered specific safety orders in the EHR. Patients at low or moderate risk received environmental safety interventions (ie, change to hospital gown and safe meal trays), whereas all patients at high risk also received 1:1 constant observation. General environmental safety interventions included addition of lockers for belongings and metal detectors for visitors to the ED. See Supplemental Fig 5 for safety guidelines.

Additional support was garnered from medical social workers for completion of the C-SSRS Full Assessment, which reduced the burden on psychiatry social workers. Patients at risk who were discharged from the hospital received a safety plan, which included lethal means reduction counseling by the medical or psychiatry social worker. Discharged patients received a follow-up phone call from social work 24 hours after discharge.

Each month, a report for problems, including missed screens, was generated from the EHR, and reasons were determined; 24/7 EHR support was made available to the ED behavioral health staff for the first 72 hours after implementation. Early glitches included the screener appearing for children <6 years old and “refused to answer” failing to populate at moderate risk. These errors were immediately identified by triage nurses and corrected in the EHR. Another issue highlighted was that patients younger than 8 years were screening false-positive on the screener. Therefore, the C-SSRS screener was modified to include the simplified language in the C-SSRS Very Young Children²⁸  version.

The OPC houses clinics for psychiatry and psychology and cares for ∼16 000 patients a year. The clinical staff included 4.2 full-time equivalent (FTE) psychiatrists, 2.4 FTE child and adolescent psychiatry fellows, 2.0 FTE psychologists, and 1.0 FTE nurse. One FTE patient care technician position was added before phase 2. Patient care technicians differ from behavioral health technicians in that they do not have crisis de-escalation training. This position allowed the normal clinic process to flow if the clinic nurse had to provide 1:1 observation.

Phase 2 went live on June 27, 2019. Patients were screened by the patient care technician using the C-SSRS screen in the EHR at intake. The clinician was informed of the risk level by using a laminated color-coded card in addition to the risk level order in EHR. All patients screening at high risk were placed on 1:1 observation with the clinic nurse or patient care technician until assessed by the primary clinical provider (psychologist or psychiatrist). If the clinician assessment determined a need for inpatient admission, the existing procedure for admission to the hospital inpatient psychiatry unit was followed. One unique challenge in this setting was determining the frequency with which a C-SSRS screener should be completed for patients with weekly follow-up appointments. Because TJC does not provide specific guidelines for frequency, the following 3 criteria for rescreening were used:

1. a concern for worsening clinical status (as assessed by a clinician);

2. first appointment after discharge from inpatient psychiatry unit; and

3. yearly.

Training videos from the Columbia Lighthouse Project Web site²⁶  were assigned to staff involved in screening and assessment (social workers, psychiatrists, and psychologists) to be completed before implementation of phase 1. In addition, all ED social workers were required to complete an online course^10,29  on counseling for reducing access to lethal means (CALM). All hospital nursing and medical providers were required to complete a module³⁰  to increase awareness around suicide. Before implementation of each phase, educational flyers that outlined workflow and roles using screenshots from the EHR were widely distributed. Ongoing education was conducted via lectures and seminars.

The data for the study was obtained from the EHR. All ED chief complaints during the study period were queried. These were further categorized as behavioral health versus other by 4 members of the workgroup.

A retrospective cross-sectional analysis, by using SAS version 9.4 (SAS Institute, Inc, Cary, NC), of all patients 6 to 17 years presenting to the ED (n = 1053) for a behavioral health complaint or to the OPC (n = 571) for an initial behavioral health appointment over 5 months was conducted. Descriptive statistics were used to define the characteristics of our population, and a multivariable logistic regression was used to model factors associated with (1) high risk on the C-SSRS and (2) discharge from the ED. The average ED LOS and the proportion of boarders per behavioral health assessment were compared before and after implementation by using statistical process control charts. Boarders were defined as patients needing admission who waited in the ED for >24 hours because of the lack of bed availability.

The majority of patients were girls (n = 581; 55.2%). The mean age was 13.4 years (SD: 2.8). Almost two-thirds were of non-Hispanic Black ethnicity (n = 667; 63.4%) and were publicly insured (n = 691; 65.6%); 64.9% (n = 683) were discharged from the ED (Table 1).

Of all behavioral health patients aged 6 to 17 years presenting to ED, 83.5% (n = 879) were screened. Of all patient screened, 71.8% (n = 631) had a positive suicide risk. Suicide risk level distribution among all screened patients was as follows: 18.9% (n = 166) were low risk, 13.3% (n = 117) were moderate risk, and 39.6% (n = 348) were high risk (Table 1). Only 0.46% of all patients screened (n = 4) left against medical advice.

After controlling for race and/or ethnicity, age group, sex, insurance status, and disposition, the adjusted odds of screening at high risk for suicide were twice as high for girls compared with boys (adjusted odds ratio [aOR] 1.93; 95% confidence interval [CI] 1.44–2.59), 32% lower for public insurance compared with private insurance (aOR 0.68; 95% CI 0.47–0.98), and twice as high in patients who were admitted compared with those who were discharged (aOR 2.04; 95% CI 1.53–2.73) (Table 2).

After controlling for race and/or ethnicity, age group, sex, insurance status, and level of suicide risk, the adjusted odds of being discharged were lower for moderate suicide risk (aOR 0.56; 95% CI 0.34–0.90) and high suicide risk (aOR 0.41; 95% CI 0.28–0.59). Adjusted odds of discharge were higher for patients who were self-insured (aOR 4.58; 95% 1.83–11.40) and non-Hispanic Black compared with those who were non-Hispanic white (aOR 1.72; 95% CI 1.08–2.76) (Supplemental Table 4).

There was no increase in average ED LOS for behavioral health patients or in the proportion of boarders, suggesting that our intervention did not prolong ED evaluation times (Fig 3).

Overall compliance with completion of the C-SSRS screener was 83.5% (n = 879) (Supplemental Table 5). Compliance with full assessment for patients screening positive, which was previously not being done, was 88.0% (n = 773) The most common reason for missed assessment (n = 53) was “discharged by ED physician without social work consult.”

The majority was male (n = 316; 55.3%). The mean age was 11.2 years (SD 3.30). Unlike the ED sample, approximately one-third (33.10%; n = 189) were of non-Hispanic Black ethnicity, and the majority had private insurance (n = 344; 60.25%) (Table 1).

Of all behavioral health patients aged 6 to 17 years presenting to the OPC for initial assessment, 65.5% (n = 374) received a C-SSRS screener. Of all screened patients, 18.2% (n = 68) had a positive suicide risk screen result. Suicide risk level distribution among all screened patients was as follows: 6.1% (n = 23) were low risk, 6.1% (n = 23) were moderate risk, and 5.9% (n = 22) were high risk (Table 1).

After controlling for age, race and/or ethnicity, and insurance status, the adjusted odds of screening at high risk for suicide were higher for girls compared with boys (aOR 3.15; 95% CI 1.15–8.56). The odds for screening high risk were not significantly different on the basis of age, insurance, or ethnicity in this sample (Table 2).

Overall compliance with completion of the C-SSRS screener was 65%, with an initial rising trend up to 78% during the first 4 months, followed by a drop to 47% in month 5 (Supplemental Table 5).

Implementation of a standardized method of suicide risk screening and assessment in behavioral health patients by using the C-SSRS brought hospital practice in compliance with TJC standards without increase in ED LOS or boarding. Seventy-two percent of behavioral health patients presenting to the ED were identified as having a positive suicide risk, justifying the effort to improve the care of these patients. The frequency of high-risk screens was lower in the OPC (5.9%) compared with the ED (39.6%) likely because of overall lower acuity in the OPC.

Compliance with risk screening in the ED remained stable between 76% and 88%, averaging at 83.5% over a 5-month period. However, in the OPC, screening compliance dropped precipitously in month 5 after the patient care technician position became vacant, highlighting the importance of appropriate personnel.

Integration of the C-SSRS Full Assessment with risk factors into the EHR made it part of the social work workflow and standardized suicide risk assessment in the ED. The phased approach allowed for problem solving, for example, inclusion of the C-SSRS Very Young Children version in the EHR before going live in the OPC. Furthermore, aligning safety interventions and psychiatry social work evaluations to the risk level reduced the burden on mental health personnel. Anecdotally, the medical social workers reported growing comfort with use of the C-SSRS Full Assessment to guide their decision-making, and use of automated safety orders allayed provider concerns for missing safety interventions.

Delays in care and increased boarding in the ED were commonly cited concerns regarding wider screening for suicide risk.¹¹  An important finding from this study was that the ED LOS did not increase and that the majority of patients screening positive for suicide risk were discharged from the hospital. Boarding rates also decreased after implementation, and the downtrend was sustained over time. This may indicate that conducting a comprehensive suicide risk assessment and safety planning in the ED improves provider comfort in discharging patients.³¹ 

In both samples, female sex was associated with screening at high risk for suicide, consistent with reports of higher rates of suicidal thinking in girls.^7,32,33 

There was no significant difference in suicide risk rates between non-Hispanic Black and non-Hispanic white youth in both samples; however, non-Hispanic Black youth had much higher odds of discharge from the hospital after controlling for sociodemographic risk factors and suicide risk. Further disparity was noted in patient demographics (ie, a majority of patients accessing outpatient care were white and had private insurance, whereas ED services were accessed by Black and publicly insured patients). Modifying public insurance policies by widening eligibility to slightly higher income brackets, increasing public insurance coverage of outpatient behavioral health care, and reducing co-pays might ensure that appropriate care is provided to this vulnerable population.^34,35 

We did not report some key metrics to assess implementation. Measuring compliance with safety planning in phase 1 and compliance with standardized assessment in phase 2 required a chart review, which was out of the scope of this study. The presenting behavioral health chief complaint was entered as free text in the EHR; therefore, descriptive statistics could not be reliably reported. Additionally, analysis of full assessment items (ie, severity of ideation and behavior) was not conducted. Provider feedback regarding acceptability was not formally obtained.

The study has implications for hospital administrators and providers given the TJC mandate, and we find that a systematic approach can be implemented without overburdening the treatment system. The variability in practice systems and level of administration buy-in are the most likely hinderances to implementation of this process in other hospitals.

Phase 3 will expand screening and assessment to non–behavioral health patients presenting to the hospital. This will require further expansion of behavioral health services. Further research is also necessary to determine if standardized methods of risk assessment after screening will achieve the goal of reduction in national suicide rates, which was the driver for the revised NPSG 15.01.01. Given overall low rates of completed suicides, pooling data from other hospitals that have implemented similar processes may help answer this question.

We thank the members of the Children’s National Suicide Screening workgroup for their hard work on this project.