Improving Documentation of Firearm Access During Pediatric Emergency Visits for Suicidal Ideation Free

This QI initiative was conducted at an urban, academic children’s hospital with an annual volume of ∼56 000 ED encounters and 1500 annual behavioral health ED encounters. Patients presenting with chief complaints related to suicidal ideation or self-injury are cared for by a pediatric emergency medicine (PEM) attending, fellow physicians, resident physicians (including pediatric, family medicine, and emergency medicine specialties), and nurse practitioners. Psychiatric social workers are available in person 24 hours per day and 7 days per week to assist with behavioral health evaluations. The provision of lethal means counseling is a standard of care in our ED for patients with suicidal ideation, regardless of endorsed access to firearms or other lethal means. Our hospital uses Epic for our electronic health record (EHR). The ED confidential note, introduced in March 2021 in response to the 21st Century Cures Act,²³  is used at the discretion of clinicians to document confidential conversations and includes a structured phrase related to firearm access. Additionally, as part of a hospital-wide effort to reduce access to lethal means among our patients, starting in October 2021, free cable gun locks were made available to any family in the ED who endorses firearm access. No patient identifiers are collected with device distribution to comply with privacy standards, but distribution for reasons of “mental health concerns or suicidal ideation” (ie, to decrease access to lethal means in a patient at-risk for intentional self-harm) versus “unintentional injury prevention” are tracked.

We convened a multidisciplinary QI team that included ED physicians, ED nurses, an ED nurse practitioner, psychiatric social workers, a hospital data analyst, injury prevention experts, and a QI expert. We used the Model for Improvement framework²⁴  to identify barriers to the documentation of firearm access and storage. Identified barriers included inadequate staff knowledge and training, concerns about acceptability to patients and families, and poorly defined roles and workflows. A key driver diagram was used to develop specific aims and potential solutions to identified barriers (Fig 1). We implemented 2 interventions to sequentially target barriers to the documentation of firearm access and storage. Team meetings occurred quarterly to review progress and direct further interventions.

Our first intervention in October 2021 involved the addition of a templated phrase to psychiatric social work consult notes: “Access to guns: yes/no.” The wording was chosen to parallel a phrase already used in the ED confidential note templates. In May 2022, we conducted a second intervention, revising the templated section of both the psychiatric social work consult note and the ED confidential note to incorporate more precise language. The revised note template stated: “Guns present in or around the home: yes/no.” If “yes” was chosen, this prompted: “Are guns locked: yes/no/family unsure,” “Are guns unloaded: yes/no/family unsure,” and “Are guns stored separate from ammunition: yes/no/family unsure.” Providers were not required to complete these prompts, and templated phrases could be manually deleted from notes. Education for ED providers, nurses, and psychiatric social workers accompanied each template change. Education occurred during staff meetings and through e-mail and included reviewing note template changes and screening processes, brief scripting examples (Supplemental Fig 5), and electronic resources on firearm safety and suicide prevention to offer to patients and families.

We developed a keyword-based search strategy to detect and measure firearm access documentation among children <18 years old seen in the ED for suicidal ideation or self-injury, which were identified through a previously validated list of diagnosis codes.²⁵  To identify firearm-related keywords, we manually reviewed 2 months of baseline data, including ED provider notes, ED confidential notes, and psychiatric social work consult notes. Candidate keywords were “gun,” “firearm,” “handgun,” “shotgun,” “shot,” “pistol,” and “lethal.” Snippets of text around these keywords, including 50 to 200 text characters before the keyword and 300 text characters after the keyword, were extracted to provide context to the narrative surrounding the keyword. Of the candidate keywords, gun or firearm were identified in all snippets of text that documented firearm access. Candidate keywords handgun, shotgun, and pistol were not identified in any note text. The candidate keyword lethal was found in standardized discharge instructions and safety planning templates, but no text snippets with this word involved the documentation of firearm access. The candidate keyword shot was found in reference to immunizations only.

We then applied the selected keywords, gun and firearm, to 4 additional months of baseline data. We evaluated abstracted snippets of text around the keywords gun and firearm for notes flagged with these keywords. We also manually reviewed all complete notes for ED encounters with youth with suicidal ideation or self-injury without these keywords to ensure that no other keywords regarding the documentation of firearm access could be identified. During this step, no additional keywords were identified that documented firearm access in notes. At this juncture, “shoot” was proposed as an additional keyword and was added to the search strategy. For a review of the remaining notes during the baseline and intervention periods, we proceeded with targeted reviews of text snippets surrounding the keywords gun, firearm, and shoot. Notably, the keyword shoot did not appear in any note text during this review.

For each encounter, we abstracted patient demographics (age, sex, race and ethnicity) and disposition (discharged from the hospital, admitted to the hospital, transferred to another institution). Race and ethnicity were defined on the basis of EHR categorization, which is variably assigned by patients, parents or caregivers, or third parties. We acknowledge race and ethnicity as a social construct, and we included this variable in the analysis to understand the diversity of the patient population at our study site.

After the completion of our QI initiative, we administered a survey to our most involved clinicians (ie, PEM attending and fellow physicians and psychiatry social workers) to ascertain their perspectives on screening for firearm access among patients with suicidal ideation and self-injury. Survey items were adapted from an intervention-specific theoretical framework of acceptability questionnaire²⁶  and assessed various factors related to screening on a Likert scale including attitude, coherence, perceived effectiveness, self-efficacy, burden, opportunity cost, and general acceptability (Supplemental Information).

Our primary study measure was the process measure of the documentation of firearm access among ED encounters with children with suicidal ideation or self-injury. We defined the documentation of firearm access as any ED note indicating the presence or absence of a firearm in or around the child’s home. Our secondary study measure was the process measure of the documentation of firearm storage among children identified to have firearms present in or around the home. We defined the documentation of firearm storage as any ED note indicating any information documenting how a firearm was stored. We further categorized the documentation of firearm storage as complete if it included documentation of all safe firearm storage elements, including whether the firearm was stored (1) locked, (2) unloaded, and (3) separate from ammunition. Our final process measure was the distribution of cable gun locks among children with “mental health concerns or suicidal ideation.”

For clinician surveys, to determine general acceptability, and for each dimension of acceptability, we calculated the percentage of clinicians indicating any positive response on the corresponding Likert scale (eg, for the dimension of self-efficacy, responses of “confident” or “very confident” in performing screening were considered positive responses).

Data were abstracted and reviewed quarterly to assess our interventions’ impact. We used descriptive statistics to describe encounter characteristics, including patient demographics and disposition, and clinician survey results. We compared encounter characteristics during the baseline and intervention periods using χ² tests for categorical variables and 2-sided t tests for continuous variables. We used statistical process control (SPC) charts and run charts to analyze changes in process measures over time. Upper and lower control limits were calculated by using standard 3 sigma calculations. Special cause variation was identified by standard rules, including 6 or more consecutive data points above or below the centerline for run charts and 8 or more consecutive data points above or below the centerline for SPC charts.²⁷ 

As a QI initiative, this project was determined by the hospital’s institutional review board not to constitute human subjects research. We followed the Standards for Quality Improvement Reporting Excellence Guidelines for reporting QI work.

During the baseline and intervention periods, 2158 encounters occurred with children <18 years old for suicide or self-injury. This included 1157 encounters during the 20-month baseline period (February 2020–September 2021) and 1001 encounters during the 13-month intervention period (October 2021–October 2022). Among the included encounters, the mean age was 13.7 years, and most patients were female (69.3%). Race and ethnicity were documented as Hispanic 34.2%, non-Hispanic Black 16.8%, non-Hispanic white 39.1%, and other 8.8%. Patient demographics did not differ significantly during the baseline and intervention periods (Table 1). Patients were more likely to be discharged from the hospital during the intervention period compared with the baseline period (47.6% vs 34.7%), whereas patients were more likely to be admitted to the hospital (51.0% vs 46.0%) or transferred to another facility (14.1% vs 8.5%) during the baseline period compared with the intervention period (P < .001).

The initial mean rate of the documentation of firearm access was 8.6%. After the launch of the ED confidential note, before the onset of this QI initiative, the mean increased to 37.8% with a centerline shift. After the implementation of the templated phrase assessing firearm access in the psychiatric social work consult note (Intervention 1), a second centerline shift occurred, and the mean rate of documentation of firearm access increased to 81.6% (Fig 2). This rate was maintained after the second intervention and through the remainder of the intervention period. During the intervention period, 97.2% of encounters with documentation of firearm access used the templated phrase. Among encounters using the templated phrase, 74.5% were documented in psychiatric social work consult notes only, 5.1% were documented in ED confidential notes only, and 20.6% were documented in both psychiatric social work consult and ED confidential notes.

Among those who endorsed firearm access, the initial mean rate of documentation of any firearm storage practices was 50.0%. After the launch of the ED confidential note and persisting throughout the intervention period, the mean rate of documentation of any firearm storage practices increased to 78.0%, resulting in a centerline shift (Fig 3). After the addition of individual safe firearm storage elements to the note templates (Intervention 2), the documentation of firearm storage practices changed from always incomplete (ie, always missing 1 or more firearm storage element) to always complete (ie, always documented whether a firearm was stored locked, unloaded, and separately from ammunition).

During the intervention period, 10 cable gun locks were distributed. Of these, 5 were distributed for “mental health concerns or suicidal ideation” and 5 were distributed for “unintentional injury prevention.” We cannot correlate the distribution of cable gun locks directly to specific encounters given the lack of associated patient identifiers.

The response rate for clinician surveys was 76.6%. Respondents included PEM attending physicians (N = 33, 67.4%), PEM fellows (N = 8, 16.3%), and psychiatry social workers (N = 8, 16.3%). Overall, 98.0% of clinicians reported screening for firearm access among patients with suicidal ideation and self-injury is generally acceptable (Fig 4). Most clinicians reported that it is clear how screening will help prevent suicide in this population (91.8%; coherence), agreed that screening for firearm access improves their ability to conduct lethal means counseling (65.3%; effectiveness), felt confident in their ability to screen (81.6%; self-efficacy), and felt comfortable with screening (93.9%; attitude). Most clinicians reported that screening takes little to no effort (93.9%; burden) and minimally interferes with other priorities (77.6%; opportunity cost).

A multidisciplinary QI initiative resulted in improved documentation of firearm access and storage among children presenting to the ED with suicidal ideation or self-injury. Before any formalized documentation process, the baseline rate of documentation of firearm access in this population was low (<9%). As a modification to comply with changes related to the 21st Century Cures Act, the ED confidential note was introduced before this QI initiative, which included a structured question related to firearm access. Although this improved documentation rates, we aimed to increase the documentation of firearm access to at least 80%, given the importance of recognizing access to lethal means among this high-risk population. We achieved this goal by modifying note templates used by psychiatric social workers and ED clinicians to include a templated phrase related to firearm access. After the modification of that phrase to include all safe firearm storage elements, subsequent documentation of firearm storage was always complete. Our clinicians found screening for firearm access among patients with suicidal ideation Apriland self-injury to be acceptable, were comfortable and confident with screening, and felt that although screening took some effort, it did not significantly interfere with other priorities.

Previous QI studies assessing the use of EHR templates to improve the documentation of firearm access among children have revealed variable results.^28 –30  Through interventions including an EHR prompt and educational materials for families, one resident outpatient continuity clinic site improved the documentation of firearm safety discussions during well-child visits from 3% to 84% of visits.²⁸  Other similar QI initiatives conducted in the ED and hospital settings for children with suicidal thoughts and behaviors have resulted in minimal to moderate improvements (up to 30% change) in the documentation of firearm access.^29,30  Our study targeted a similar population but resulted in a significant improvement in the documentation of firearm access and storage. Factors that influenced the success of our effort may have included the interdisciplinary nature of the QI team (ie, multiple ED providers, nurses, and psychiatric social workers), recurring provider education, ease of use of the template, embedding processes into existing workflows, and the availability of free cable gun locks for families, which may have increased clinician motivation to screen because a tangible next step was available. The sustainability of our effort is demonstrated by the maintenance of improved documentation rates for 6 months after the final intervention. In addition, we developed a keyword-based search strategy to identify notes with documentation of firearm access, which enhanced the feasibility and sustainability of measuring this documentation over time.

Despite the success of our QI initiative, we recognize several ongoing barriers to the documentation of firearm access and storage for children in the ED. Importantly, our interventions did not directly target clinician attitudes related to screening for firearm access. Studies have revealed that providers, including those within the ED setting, have skepticism about the effectiveness of screening, concerns about insufficient training and whether screening is within their professional scope, and apprehension regarding acceptability to patients and families.^19,31  This may contribute to why, in our study and previous studies,²⁹  most documentation of firearm access was performed by psychiatric social workers rather than ED or hospital clinicians despite the availability of the templated phrase in both types of provider notes. Our clinician survey results reveal that screening for firearm access is acceptable across various measures to a variety of clinicians. Previous work has also revealed that ED-based screening for firearm access is feasible, effective, and acceptable to patients and families.^16,17,32  To address these barriers, a future direction will include the development and dissemination of a digital provider toolkit that summarizes supporting evidence and delineates best practices for screening for firearm access and lethal means counseling, along with scripting examples. Finally, because our intervention focused on documentation, we did not measure changes to related processes such as the provision of lethal means counseling (because of a lack of standardized documentation) or screening associated with the distribution of gun locks (because distribution was not paired with patient identifiers), which may be more proximal measures to long-term outcomes of reducing firearm-related suicide among youth. The authors of future work should examine the impact of improved rates of screening for firearm access on the provision of lethal means counseling and firearm storage device distribution in the ED.

Understanding firearm access and storage practices is the first of several steps needed to prevent firearm-related suicide among youth. Asking about firearm access must also be coupled with lethal means restriction counseling that effectively motivates families to change storage practices. For example, one study, entitled the SAFETY trial, evaluated a brief lethal means counseling intervention for caregivers whose child presented to the ED for suicidal thoughts, suicidal actions, or other behavioral health concerns. Twice as many caregivers reported locking all previously unlocked firearms at a 2-week follow-up compared with caregivers who did not receive the intervention (69% vs 37%).¹⁷  One randomized controlled trial evaluated the provision of and training on safe storage devices to caregivers of children presenting to the ED with behavioral health concerns and revealed modest increases in triple-safe storage (ie, locked, unloaded, stored separately from ammunition) among caregivers who received the intervention (32% at baseline to 56% at 7- and 30-day follow-up) compared with caregivers who did not receive the intervention (21% at baseline to 31% at 7- and 30-day follow-up).³³  Overall, the combination of screening for firearm access, lethal means counseling, and the provision of a safe firearm storage device holds promise for preventing firearm-related suicide. Future QI efforts and implementation science studies should target increased adoption and fidelity to these evidence-based interventions for high-risk youth.

There were several limitations we must acknowledge in the interpretation of our results. First, we used documentation as a proxy for screening; there may have been instances in which screening occurred but was not documented or in which screening was documented but did not occur. In addition, given our keyword-based search strategy, some notes documenting firearm access may have been missed if no keywords were present. This is likely infrequent based on our baseline review and would only serve to increase documentation rates further if any missed notes were incorporated. Future prospective, observational studies are needed to assess the extent to which the documentation of screening aligns with actual clinician practice, as well as to understand how screening and counseling change firearm storage practices. Additionally, we used billing diagnosis codes to identify our study population of children with suicidal ideation or self-injury, which may also be prone to misclassification. Although we considered several balancing measures, we were unable to measure them. Specifically, we were unable to measure the time involved in asking and documenting firearm access screening questions, which must be balanced against competing priorities in the ED environment. We also did not assess the acceptability of screening to patients; however, acceptability has been demonstrated in previous studies.³²  Lastly, because our QI initiative was based at a single children’s hospital, some of our interventions may have limited generalizability to other hospitals that lack access to psychiatric social workers or have limited capabilities to modify standardized note templates.

Adding a standardized question to ED clinician and psychiatric social work consult note templates facilitated a substantial increase in the documentation of firearm access and storage practices among children presenting to the ED for suicidal ideation or self-injury. This approach may be replicated by other EDs seeking to improve screening for firearm access and storage practices among children. When coupled with counseling and the provision of gun locks, this simple intervention holds promise to reduce the burden of morbidity and mortality of pediatric firearm injuries. The authors of future studies should assess whether improved documentation is associated with improved storage practices and reductions in firearm injuries after ED encounters.

The authors would like to thank Kimberly Denicolo, Patricia Aquino, and Leslie Flament for serving on the interdisciplinary team that assisted with these QI efforts. The authors would like to thank our psychiatric social worker colleagues for their partnership, flexibility, and compassion in caring for children in our ED. The authors would also like to thank Erene Hanna and Kaitlin Keeley for their assistance with the review of note text to identify documentation of firearm access.

ED

emergency department

EHR

electronic health record

PEM

pediatric emergency medicine

QI

quality improvement

SPC

statistical process control