Safety event reporting systems facilitate identification of system-level targets to improve patient safety. Resident physicians report few safety events despite their role as frontline providers and the frequent occurrence of events. The objective of this study is to increase the number of pediatric resident safety event submissions from <1 to 4 submissions per 14-day period within 12 months.
We conducted an iterative quality improvement process with 39 pediatric residents at a children’s hospital. Interventions focused on 4 key drivers: user-friendly event submission process, resident buy-in, nonpunitive safety culture, and data transparency. The primary outcome measure of number of pediatric resident event submissions was analyzed by using statistical process control. Balancing measures included time from submission to feedback, duplicate submissions, and nonevent submissions. As a control, the primary outcome measure was monitored for nonpediatric residents during the same period.
The mean number of pediatric resident event submissions increased from 0.9 to 5.7 submissions per 14 days. Impactful interventions included a designated space in the resident workroom to list safety events to submit, monthly project updates, and an interresident competition. There were no duplicate submissions or nonevent submissions in the postintervention period. Time to feedback in the postintervention period had both upward and downward shifts, with >8 consecutive points above and below the baseline period’s centerline. The control group showed no sustained change in event submissions.
Our improvement process was associated with significant increase in pediatric resident safety event submissions without an increase in the number of submissions categorized as duplicates or nonevents.
Iatrogenic causes, especially medical errors, are the third leading cause of death in the United States.1 Up to 1.1% of patients, or >400 000 individuals, admitted to a hospital will die because of a medical error.1 Among hospitalized patients who live, ∼10% of them are affected by at least 1 adverse event.2 Children are particularly vulnerable to experiencing patient harm, with factors such as complex medical conditions, physical characteristics, cognitive and physiologic development, and legal status as minors exacerbating their risk for harm.3,4
Safety event reporting systems in teaching hospitals are a valuable source of documented information to both direct patient safety improvements as well as to guide education,5 but recognizing similarities and patterns in sources of risk can only occur if events are submitted and analyzed. Unfortunately, many current event reporting systems provide an unreliable and nonrepresentative measure of errors and adverse events.2,6 Physicians, including residents, are less likely than nurses to report safety events, with authors of previous studies identifying barriers to reporting such as time constraints, lack of feedback, cultural norms, beliefs about risks, resource constraints, unclear definitions of a reportable incident, and perceived lack of value in the process.6
Despite resident physicians being frontline providers who are well positioned to identify problems and solutions with the care delivery system, residents have traditionally been uninvolved in health care organization initiatives to improve quality and safety.7 A goal of the Accreditation Council for Graduate Medical Education’s Clinical Learning Environment Review program is to engage residents in patient safety and develop reporting behaviors to carry forward throughout their careers.8 Although most residents have been involved in an adverse or near-miss event, they infrequently report these incidents, and authors of previous studies have found low baseline rates of event reporting among housestaff.9–11
Pediatric patients hospitalized in teaching hospitals experience 26.2 adverse events per 1000 patient days.12 These data suggest that our hospital may have ∼40 pediatric adverse events every 14 days, with additional near-miss events occurring as well. However, our hospital was only reporting on average 2 pediatric safety events every 14 days, and of those, 0 to 1 was resident reported. The low rate of pediatric resident reporting was similar to other residency programs at our institution. Our goal was to use improvement methods to develop and implement a sustained process to increase safety event submissions by residents, with the global aim of improving patient safety at our hospital. Our specific aim was to increase the number of pediatric resident safety event submissions from <1 submission to 4 submissions per 14-day period within 12 months.
Methods
Context
Our children’s hospital, imbedded in an academic health system, is a tertiary and quaternary referral center for children. The pediatric residency program is composed of 39 categorial residents.
The health system quality and safety department supports standardized processes to collect, review, and respond to safety events submitted by staff and physicians. To promote safety event reporting, the quality and safety department introduced a safety event submission telephone hotline ∼2 years before this study. Targeted education regarding submission of safety events via the traditional reporting software and new hotline was added to Graduate Medical Education orientation sessions. The quality and safety staff present additional education at resident meetings throughout the hospital. Feedback regarding the outcome of an event review is provided via e-mail to the submitter. Anonymous reporting is not permitted to adequately gather information to review events and to provide feedback.
Planning the Interventions
A pediatric resident-led multidisciplinary improvement team was assembled. This team included a pediatric hospitalist, pediatric intensivist, pediatric residents, quality improvement (QI) experts, and members of the hospital quality and safety department. The team met monthly to discuss tests of change and review stakeholder feedback.
Initial pediatric resident perspectives on safety events and the reporting process were obtained via a focus group with 16 pediatric residents. This process identified barriers to submitting safety events, including workflow integration, punitive culture, and lack of knowledge about reporting. Identified facilitators included receiving feedback on outcomes from previous events and recognizing the importance of submitting events. The session also identified potential interventions to increase event submissions, including advertising the event submission phone line, educating residents on various types of safety events, and disseminating a resident-specific safety bulletin. Our improvement team used these data to develop a key driver diagram (Fig 1) and a possible, implement, challenge, and kill chart (Supplemental Fig 5) to prioritize interventions to be completed by using sequential plan do study act (PDSA) cycles.
Improvement Activities
Interventions were focused on 4 key drivers of increasing pediatric resident event submissions: (1) user-friendly event submission process, (2) resident buy-in, (3) nonpunitive safety culture, and (4) data transparency. Changes were tested through a series of PDSA cycles, and the initial key driver diagram was revised over time to highlight active efforts.
Throughout the PDSA cycles, feedback was gathered verbally in person from residents and attending physicians and was compiled by the improvement team and used to guide iterative improvements. This feedback included thoughts on the perceived impact of the active interventions. Pediatric residents participated in team meetings and were encouraged to share real-time feedback with the project lead. A description of the interventions is presented in Table 1.
Category of Intervention . | Specific Intervention . | Start Date . | Intervention Description . | Status . |
---|---|---|---|---|
User-friendly safety event submission process | Workroom signs | November 5, 2018 | Place signs in resident workrooms with event submission telephone hotline number | Adopt |
Whiteboard list in workroom | December 10, 2018 | Create designated space on whiteboards in hospital workrooms for residents to list safety events needing to be called in | Adopt | |
Badge sticker | January 24, 2019 | Pass out stickers with event submission hotline number to place on resident badges | Adopt | |
Patient safety presentation | June 21, 2019 | PowerPoint presentation at GME orientation on Event Reporting and Just Culture, including patient safety video with submission demonstration | Adopt | |
Resident buy-in | Safety newsletter | November 6, 2018 | Distribute monthly safety newsletter via e-mail to residents, with safety event submitting tips and examples | Adapt |
Hospitalist champion | December 11, 2018 | Hospitalist attending serve as primary driver to promote QI project reminding residents to be more aware of safety events and to submit those identified | Abandon | |
Resident safety champions | January 28, 2019 | Adapt champion role from attending to resident; assign a senior wards resident as safety champion for 2-wk period to promote QI project and win a $50 award if more events are submitted during their period in comparison to previous period (announce award via e-mail and at resident conference) | Abandon | |
Text message to hospitalist attending physicians | March 18, 2019 | Weekly text from quality and safety director to hospitalist attending physicians to remind about importance of encouraging residents to submit safety events | Abandon | |
All-resident competition | July 1, 2019 | Adapt champion role from one designated resident to an all-resident competition in which anyone who submits the most IRs in a 2-wk period wins a $50 award | Adopt | |
Nonpunitive safety culture | Name change | December 8, 2018 | Adopt “safety catch” nomenclature to replace “incident report” | Adopt |
Optimize feedback | June 13, 2019 | Add statement to written feedback provided to event submitter to reply with clarification if the interpretation of the event by the analyst is incorrect | Adopt | |
Data transparency | Publicize run charts | January 24, 2019 | Share updated run charts in safety newsletter and e-mail announcements | Adapt |
Category of Intervention . | Specific Intervention . | Start Date . | Intervention Description . | Status . |
---|---|---|---|---|
User-friendly safety event submission process | Workroom signs | November 5, 2018 | Place signs in resident workrooms with event submission telephone hotline number | Adopt |
Whiteboard list in workroom | December 10, 2018 | Create designated space on whiteboards in hospital workrooms for residents to list safety events needing to be called in | Adopt | |
Badge sticker | January 24, 2019 | Pass out stickers with event submission hotline number to place on resident badges | Adopt | |
Patient safety presentation | June 21, 2019 | PowerPoint presentation at GME orientation on Event Reporting and Just Culture, including patient safety video with submission demonstration | Adopt | |
Resident buy-in | Safety newsletter | November 6, 2018 | Distribute monthly safety newsletter via e-mail to residents, with safety event submitting tips and examples | Adapt |
Hospitalist champion | December 11, 2018 | Hospitalist attending serve as primary driver to promote QI project reminding residents to be more aware of safety events and to submit those identified | Abandon | |
Resident safety champions | January 28, 2019 | Adapt champion role from attending to resident; assign a senior wards resident as safety champion for 2-wk period to promote QI project and win a $50 award if more events are submitted during their period in comparison to previous period (announce award via e-mail and at resident conference) | Abandon | |
Text message to hospitalist attending physicians | March 18, 2019 | Weekly text from quality and safety director to hospitalist attending physicians to remind about importance of encouraging residents to submit safety events | Abandon | |
All-resident competition | July 1, 2019 | Adapt champion role from one designated resident to an all-resident competition in which anyone who submits the most IRs in a 2-wk period wins a $50 award | Adopt | |
Nonpunitive safety culture | Name change | December 8, 2018 | Adopt “safety catch” nomenclature to replace “incident report” | Adopt |
Optimize feedback | June 13, 2019 | Add statement to written feedback provided to event submitter to reply with clarification if the interpretation of the event by the analyst is incorrect | Adopt | |
Data transparency | Publicize run charts | January 24, 2019 | Share updated run charts in safety newsletter and e-mail announcements | Adapt |
Resident refers specifically to pediatric resident. GME, graduate medical education; IR, incident report.
User-friendly Event Submission Process
The existing safety event reporting telephone hotline service was underused because of lack of knowledge among residents of the hotline phone number. Early PDSA cycles were focused on spreading awareness and technical information for accessing the hotline via word of mouth and posting signs with the phone number in the resident workrooms. Small stickers with the hotline phone number were made and distributed to residents to place on the back of resident identification badges. Resident feedback revealed that safety events were often identified at times inconvenient to submit, so submissions were deferred for later and often forgotten. Thus, a designated space on the whiteboard in the workrooms was made for residents to list events needing submission by writing the medical record number, date, and key words about the event to allow for easy recall when submitting the event at a later time.
Resident Buy-in
Safety newsletters were already being created by quality and safety department QI analysts for hospital-wide intradepartmental meetings. We adapted these newsletters to be more relevant to residents. Newsletters were disseminated monthly via the pediatric chief resident e-mail updates, which was reported by residents to be well read. Resident and project team feedback contributed to improving these newsletters over time, including adding more examples of resident-submitted events and their outcomes, making them more reader friendly (diagrams, bullet points, etc) and adding the run chart.
The concept of a “safety champion” was developed with resident input. In this model, one of the senior residents on wards was assigned to be the champion for a designated 14-day block. This role included reminding residents to think about events on rounds and to submit events that were identified and/or written on the whiteboard. The goal for each champion was to beat the number of events during the previous champion’s 14-day block. A $10 gift card was awarded if they did not beat it and $50 if the champion had more events submitted during their block. This model was successful in getting more residents to be engaged with the event submission process, and feedback revealed that residents continued to promote event reporting even when not the designated champion.
Nonpunitive Safety Culture
Before this project, cultural barriers to submitting events included fear of retaliation, concerns about placing blame on others, and overall lack of discussion about the topic. One strategy was to change the nomenclature used. “Incident reports” was the long-standing term used across our hospital, and in our study, we renamed them “safety catches.” Our QI initiative was referred to as “Safety Catches 4Kids” to avoid the pejorative connotation of an incident report and to promote the ideal of capturing near misses and latent system defects.
Historically, feedback on the outcomes of an event submission was provided via e-mail to all reporters. This feedback allowed closed-loop communication and allowed reporters to see the effects of their submission. However, residents expressed that, at times, the feedback conveyed a sense of blame and discouragement. We thus focused on optimizing reporter feedback to avoid wording that suggested fault on certain individuals and to explicitly state that a reply from the submitter is encouraged if the interpretation of the event by the QI analyst was incorrect.
Data Transparency
To provide pediatric residents with real-time feedback on the interventions, updated run charts presenting the number of pediatric safety event submissions per 14-day period were shared monthly via e-mail.
Measures
Our primary outcome measure was safety event reports, defined as the number of events submitted by pediatric residents per 14-day period. Events submitted via the online system and the telephone hotline were both included. The QI analysts would process these data and report them every 14 days via e-mail to our team. As a balancing measure, we monitored the safety report turnaround time, defined as time (days) from the pediatric resident event submission to when the submitter received feedback on the outcome of the event. This balancing measure was chosen to monitor the potential overload on the quality and safety department in processing higher volumes of events. Balancing measures also included percentage of duplicated events among pediatric residents as well as percentage of events categorized as a “nonevent.” Duplicated events and nonevents were determined by the quality and safety department analysts. Nonevents were defined as events that were reported but on investigation did not rise up to the level of a potential safety concern. As a control, we similarly measured rates of nonpediatric resident safety event reporting during the same time period. Nonpediatric residents included the residents from the health system’s 19 other residency programs.
Analysis
The number of event submissions were analyzed by using statistical process control for Excel.13 Our primary outcome measure and control data were analyzed on a c-chart every 14 days. A c-chart was used because the number of residents stayed constant over time. Our balance measure was analyzed on an i-chart because the measures of performance were obtained infrequently because they were based on the number of submissions. Upper and lower control limits were defined as greater than or <3 σ. Special cause variation was identified by a single point outside the control limits or by 8 consecutive points above or below centerline. For the i-chart, special cause variation was also identified by 2 of 3 points above or below 2 σ or 4 of 5 points above or below 1 σ. The average safety report turnaround time, number of duplicate submissions, and nonevent submissions were obtained every 14 days from the quality and safety department analysts. To identify which interventions were most impactful, the improvement team reviewed the resident and attending feedback in conjunction with the primary outcome measure c-chart data at the monthly team meetings. This study was deemed exempt by our university’s institutional review board.
Results
During the 12-month study period, residents submitted 148 safety events. The mean number of events submitted increased from 0.9 during the baseline period to 5.7 per 14 days during the intervention period (Fig 2). During the study period, there was only one 14-day period with 0 safety events submitted; the remainder of the weeks had more events submitted than the baseline of 0.9.
Special cause variation is shown in Fig 2, with 2 points outside the upper control limit during the intervention period. Resident and attending feedback suggested these 2 significantly high data points were the result of highly motivated individual attending physicians and residents being on the wards during those blocks. Additionally, after initiating the project, there was a shift with >8 consecutive points above the baseline period’s centerline.
Regarding the control group, the mean number of events submitted among nonpediatric residents remained unchanged at 12.6 per 14 days (Fig 3). The control data revealed no shifts. There was 1 special cause variation; 1 point during the intervention period was outside the upper control limit.
Among the interventions implemented, the most impactful interventions included a designated whiteboard space in the resident workroom to list safety events to submit, biweekly emails to publicize QI data, and an interresident competition. Special cause variation occurred during the first study period with the hospitalist champion with 14 events submitted and the fourth study period with the all-resident competition with 18 events reported.
Among pediatric residents, the mean turnaround time from event submission to feedback was 23.2 days (Fig 4). The baseline period collected over 4 months revealed steady state. During the study period, special cause variation occurred; some submissions had a turnaround time above the upper control limit, whereas one was below the lower control limit. There were 1 and 2 series of ≥8 consecutive points above and below the centerline, respectively. Regarding the other balancing measures, there were 0 nonevents and 0 events that reported duplicated information among the pediatric resident submissions in the postintervention period.
Discussion
Our study reveals that residents can be effectively engaged in safety event reporting through a multifaceted approach. Specifically, our study revealed a sixfold increase in resident reporting. Residents continuously face competing priorities in providing safe patient care,14 and increasing self-reporting of events by residents represents an important step in systems-based improvement of patient safety. Among the interventions implemented, the most impactful interventions included a designated whiteboard space in the resident workroom to list safety events to submit, biweekly emails to publicize QI data, and an interresident competition. Feedback sessions with residents revealed that there was variability in which intervention each resident felt was most impactful for them; some described the more timely and structured feedback made them feel their efforts were more impactful, others felt the interresident competition was a motivating factor, and others described the safety newsletter as a good reminder of ongoing efforts and progress. For many, they felt a combination of factors ultimately contributed to their change in behavior and overall a change in our resident safety culture.
Many of the reports submitted by residents offered new insights into previously unnoticed or underemphasized systems issues. Some of the reported events led to systemic change; for example, a “Best Practice Alert” was developed, which pops up for both the ordering provider and the pharmacist when any medication is ordered for patients with a dextrose allergy. The alert warns the provider and pharmacist that the patient has a dextrose allergy and that any medication should be reviewed with the pharmacist before ordering.
Interventions that were abandoned included having hospitalists or preidentified residents serve as safety champions. The challenge with hospitalists as champions was that there was unequal buy-in from all providers and a short period during which each attending was on service. The all-resident competition proved more successful than a preidentified resident champion because it allowed for both interns and seniors to become champions; additionally, this structure created a competition as opposed to more of a predefined winner. Creating a competition between residents was a unique intervention that has not been commonly described in literature and could serve as a platform for others looking to more effectively engage residents.
It is imperative to demonstrate to residents that reporting is valuable and makes a difference as a tenet to establishing and maintaining a reporting culture. Creating a nonpunitive reporting culture includes education and empowerment of residents on what and why to report. This concept is similarly supported in previous publications. Katz et al15 demonstrated that if staff members reporting safety events know that their submission will be used as a QI tool and not as a punitive method, the level of voluntary reporting significantly increases.
In previous studies, authors have examined various methods to improve resident event reporting, but there is limited evidence supporting sustained effective interventions to increase resident engagement in submitting safety events.10,11 For example, Boike et al16 identified positive attitudes toward reporting among residents but were unable to achieve a long-term commitment from them. Despite incorporating patient safety and medical fallibility into the curriculum, Madigosky et al17 demonstrated an unsustained improvement in reporting patient safety events by medical students. On the other hand, Jericho et al18 increased the number of adverse event reports and improved attitudes toward reporting among anesthesiology resident physicians after their educational interventions.
Unfortunately, patient safety research is vastly limited by the fact that we still do not have sufficient means or tools to identify all medical errors. However, expert consensus agrees that event reporting systems are a primary avenue to addressing the immense health care safety problem.19 Mitchell et al20 concluded that if the health care industry wants to learn from its mistakes, both miss and near-miss events, it will need to take event reporting as seriously as the health budget.
Limitations to this study include the implementation at a single residency training program, so the generalizability of our findings may be limited. The safety culture at our hospital, or even within the pediatric residency program, may be uniquely different from other hospitals and programs. Additionally, this QI study leveraged our established safety event reporting system and telephone hotline, which may not exist at other hospitals, and involved multiple interventions, which may make it more difficult to ascertain the most impactful one. Furthermore, additional balance measures may exist that were not measured in our study. Nevertheless, the implemented interventions in our multidisciplinary QI process can be tested by teams working in other settings attempting to achieve increased safety event submissions.
Conclusions
We found that a multifaceted approach that focused on creating a user-friendly event submission process, increasing resident buy-in, establishing a nonpunitive safety culture, and promoting data transparency led to increased resident safety event reporting. Increased resident involvement gives hospital quality and safety leaders direct access to the unique perspective that only frontline providers can offer. Efforts to spread similar interventions to other residency programs at our hospital are underway. Overall, more work is needed to understand how to best engage residents in efforts to reengineer safer, more reliable health care systems.
Dr Mattes conceptualized and designed the study, led the multidisciplinary improvement team, led the implementation of interventions, and drafted the manuscript; Ms Sauers-Ford contributed to study design and data analysis and participated in the multidisciplinary improvement team; Ms Selleck and Ms Slee participated in the multidisciplinary improvement team and contributed to data collection; Dr Natale contributed to study design and participated in the multidisciplinary improvement team; Dr Rosenthal conceptualized and designed the study, participated in the multidisciplinary improvement team, and supervised data collection; and all authors reviewed and revised the manuscript and approved the final manuscript as submitted.
FUNDING: Supported by the University of California Davis Graduate Medical Education High Value Competition Award. Dr Rosenthal’s effort on this project was supported by the National Center for Advancing Translational Sciences, National Institutes of Health (through grant UL1TR001860 and linked award KL2 TR001859). The content is solely the responsibility of the authors and does not necessarily represent the official views of the University of California Davis or the National Institutes of Health. Funded by the National Institutes of Health (NIH).
References
Competing Interests
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Comments