Assessing the Effect of Electronic Medical Record Note Template on Firearm Access Screening in High-Risk Children Free

This was a retrospective pre- and postintervention study. The study subjects are children <19 years old admitted to a tertiary care freestanding children’s hospital inpatient unit after a suicide attempt as identified by previously validated international classification of disease version 10 codes (Supplemental Table 3).¹³  Preintervention cohort was defined from January 1, 2016 to March 31, 2017. The postintervention cohort was defined from April 1, 2017 to March 1, 2020.

Our hospital’s electronic medical record (EMR) launched in 2012, this included “SmartPhrases” (text macros with selectable text aiding clinical documentation¹⁴ ) which were distributed to all emergency department (ED) and psychiatry clinicians. All other clinicians could access the SmartPhrase and copy them into their own toolboxes; however, they were not preloaded as they were for ED and psychiatry clinicians. This also included lethal means access screening questions (Table 1). SmartPhrase use was encouraged in ED and psychiatry divisional meetings, but no formal tracking mechanism or feature rollout was conducted.

In 2017, a new psychiatry note template was integrated into the EMR that included the lethal means screening SmartPhrase verbatim (Table 1), immediately after the review of systems section. The rationale for this change was to remove the burden on the clinician from needing to remember to use the SmartPhrase and instead include it in a tool ingrained in their workflow: a templated consultation note. The questions included a field that a provider was required to either fill out or delete before signing the note. The updated template was distributed via the EMR to all learners and clinicians who potentially could complete the consultation note: licensed clinical social workers, psychiatry or psychology clinicians, and rotating clinician learners. Use of the new note was again encouraged but not mandatory.

No formal methods to track outcomes of this intervention were created at the time of the rollout. This research team chose firearm access documentation in any inpatient note as our primary outcome. This was selected as the best proxy for our goal of increasing the frequency of firearm access screening. We also measured, as secondary analyses, which provider team recorded the firearm screening, if lethal means reduction counseling was documented, and the method of firearm screening: free text, SmartPhrase, or in the updated template. All pediatric and psychiatric clinician notes were read in their entirety for documentation of the above outcomes. If no documentation was found, a back-up process of searching the entire patient’s medical record for the phrases “firearm” or “gun” to check that no documentation episodes were missed was conducted.

To ensure data quality, 60 randomly selected charts were independently reviewed by an attending physician and the research assistant for the primary outcome (the presence or absence of firearm screening documentation). Interrater reliability was assessed by Cohen’s κ calculation.¹⁵  Study data were collected and managed by using Research Electronic Data Capture.^16,17  Research Electronic Data Capture is a secure, web-based software platform designed to support data capture for research studies.

To analyze the intervention effect over time, proportions of charts screened for firearm access in the study group were assessed using interrupted time series analysis. This analysis includes the analysis of covariance framework without bootstrapping with the “its.analysis” package in R Studio.^18–20  Data were aggregated quarterly to allow for a sufficient denominator.

Of 26 088 total admissions, 357 met inclusion criteria (preintervention n = 109, postintervention n = 248). The majority were teenagers (15 years old, interquartile range 14 to 16), female (77%), and White (47%) (Table 2). There were no meaningful demographic differences pre- and postintervention. Assessment of the primary outcome suggests a high interrater reliability by calculating Cohen’s κ coefficient (k = .721, P < .001).¹⁵  The new templated note was quickly adopted, with screening documentation transitioning away from SmartPhrase use (Fig 1).

Overall, 80% (n =286) of patients had firearm access screening documentation, 78% preintervention, and 82% post intervention. There were no significant demographic differences between those who were or were not screened. Despite the successful transition from SmartPhrase to new consult note template, the proportion of children at high risk screened for firearm access did not change either in the absolute proportion of patients screened (P = .74), or trend over time (P = .84) on interrupted time series analysis (Fig 2). Of those screened, 90.2% were screened by psychiatry clinicians, 6.6% were screened by both psychiatry and pediatric clinicians, and 3% were captured by the pediatric team. Overall, the pediatric team screened only 28 of 357 patients for firearm access (7.7%).

Twenty percent of patients were discharged without a firearm screen. Of the 20% not screened for firearm access (n = 71), 50 (70%) were transferred to psychiatric facilities, and 21 (30%) were discharged directly to home: 6% of the overall cohort.

During the study period, documentation of firearm access screening changed from a SmartPhrase to a templated consult note in the EMR; however, the proportion of children with documented firearm access screening did not change. Pediatricians rarely screened this high-risk cohort, whereas psychiatry did the majority of firearm access screening. Despite the high risk of firearm related suicides, 20% of the patients had no documented firearm screen.

Clinicians often do not screen for firearm access. Among pediatric patients evaluated by a psychiatry resident in a psychiatric ED, only 3% were screened for firearm access.²¹  Similarly, only 5% of adolescents presenting to a pediatric ED with suicidal or homicidal ideation were screened for firearm access by pediatric residents.²²  Inpatient physicians have similar rates to these previous publications: <3% of all pediatric inpatient admissions are screened for access to firearms.¹¹  Admissions for mood disorders were screened most frequently, with 37% of patients screened for firearm access; however, this is a concerningly low screening rate.

It is unclear which clinical specialty is the appropriate clinician to screen for firearm access. Most ED physicians (83%) believe that it is a psychiatrist’s responsibility to screen.²³  This reflects the current screening paradigm. In a chart review of 500 inpatients screened for firearm access, >95% were completed by child psychiatry clinicians.¹¹  These findings are similar to our study: 97% of patients with completed screens were documented by psychiatry clinicians. The cause for this large discrepancy in screening rates (7.7% by pediatricians vs. 97% by psychiatrists) is not clear; however, this presents an opportunity for hospitalists and their trainees to change the status quo.

Approaching this problem using a systems-based approach would likely be more successful than depending on a single clinician to correctly complete a screening tool whether in a templated note or a SmartPhrase. The rationale of the intervention appeared sound: incorporate a validated tool, with fields needing to be addressed, into a universally used note. However, the small gains, if any, from making this change did not overcome the realities of psychiatric medical stabilization and human error. If a patient had altered mental status at the time of consult note initiation, how could a screening take place? If the psychiatry clinician was rushed and skipped the question in their interview, who else could do the screen? One approach would be to make the fields truly mandatory. Another would be to improve nonpsychiatry team screening rates as a backstop for when psychiatry team failed. In this study, although the pediatric team rarely screened for firearm access, in 3% of cases it captured a case which otherwise would have gone unscreened by psychiatry. High screening rates are achievable. Gastineau et al successfully increased firearm screening in outpatient resident clinic, moving from 3% to 84%.²⁴  They accomplished this by educating team members about the issue using the BeSMART framework,²⁵  and modifying the default note template for well child visits. Although it is unclear if this improvement would have occurred with implementation of a SmartPhrase as opposed to a templated note, it is clear that the cultural shift coupled with education and a screening tool was critical.

One of the limitations of this study is that it was a single-site retrospective pre- and postanalysis; the change from a SmartPhrase to a templated note may achieve different results in a region with lower baseline screening, or an area of the country with less baseline focus on firearm injury prevention. The SmartPhrase is specific to the EMR at our hospital, although other EMRs likely have similar templates and functions. Our baseline time period did not compare free text and starting a SmartPhrase or a new template. Future work should investigate the effect of this strategy. The lack of a real-time intervention and rollout strategy is a major weakness of this endeavor. The rollout and update may have been more effective had the intervention been done universally. Finally, the primary outcome is screening documentation, not observation of screening or lethal means reduction counseling completion. Prospective work is needed to conduct these analyses.

In conclusion, reliance on a technologic fix, a templated note with input fields, did not improve our rates of screening inpatient suicidal adolescents for firearm access. In our institution a SmartPhrase and templated note generated similar rates of screening for firearm access; however, either alone is inadequate. The next steps are to create an interdisciplinary team to change from selected to universal firearm screening using a quality improvement approach.

We thank the Connecticut Children’s Injury Prevention Center for their support.