Over the last decade, pediatric mental health emergency department (ED) visits and hospitalizations have risen substantially.1,2 Increasingly, children experience boarding in the ED and inpatient medical units while awaiting inpatient psychiatric beds.3,4 To address this evolving crisis, quality improvement (QI) can be leveraged to improve outcomes by effecting local change. This issue of Pediatrics reports 2 successful QI initiatives that improved acute pediatric mental health services using different interventions to address distinct targets. These reports offer valuable lessons regarding both strengths and limitations of QI, relative to alternative approaches.
Afzal et al5 implemented a boarding stabilization protocol in an ED observation unit, centered around brief interventions delivered by social workers and activities provided by child life specialists. After implementation, inpatient psychiatric admissions decreased from 66% to 49%. Because the boarding stabilization protocol relied on increased personnel support, it may be difficult to translate to lower resourced settings. This illustrates an inherent limitation of QI work—its focus on local improvement can, at times, constrain generalizability. As these authors did, QI reports should carefully describe local context to allow readers to assess if the interventions used are applicable to their settings. Although the interventions in this report led to important decreases in admission rates, additional studies are needed to test feasible approaches, such as self-guided activity kits,6 for children experiencing mental health boarding in less resourced settings.
Winner et al7 employed 4 consecutive strategies to reduce readmissions to a behavioral inpatient unit: individualized programming, postdischarge follow-up calls, a conflict management curriculum, and a discharge preparation checklist for readmitted patients to reduce the likelihood of a second readmission. Readmissions within 7 and 30 days decreased by 49% and 36%, respectively. Control chart analysis of their performance measures, a commonly used method for examining QI data, showed decreased readmissions immediately after implementation of the conflict management curriculum and several months after initiation of follow-up calls. This illustrates another limitation of QI—it is often difficult to ascertain which interventions are the primary drivers of change. In contrast to research, QI is not designed to systematically compare interventions or prove causality. Instead, QI efforts should be based on a theory of improvement that links drivers and potential interventions to desired outcomes. Interventions are then tested through Plan-Do-Study-Act cycles to confirm desired impact. However, it is important to recognize that, beyond the specific interventions under study, unmeasured confounders and temporal trends may also contribute to changes in performance seen over time.
How can these potential limitations of QI be mitigated? With regards to generalizability of QI interventions, several strategies can help spread learnings across centers. As described above, clear descriptions of context are necessary in QI publications to enable initial determination of applicability for other settings. In addition, although interventions to drive change may be particularly aligned with local context, other elements of QI frameworks are likely more easily spread across settings, including the composition of multidisciplinary teams, specific aims, measure definitions, identification of drivers, and approaches to data monitoring and analysis. Increasingly, QI collaboratives are bringing together centers with differing local contexts to share ideas for improvement and lessons learned while using a shared QI framework to address common improvement targets.
Building evidence to support the causality of QI interventions is more challenging.8 From a methodologic standpoint, traditional research methods such as randomized controlled trials are more optimally suited to evaluate intervention effectiveness and contribute to generalizable knowledge. Ideally, research to establish the effectiveness of interventions would precede QI and generate evidence to inform QI work. For instance, if Afzal et al5 had chosen a research design to study the boarding stabilization protocol, more rigorous approaches could have been used to mitigate the influence of bias and confounders. However, research methods may not always be feasible, particularly given the time involved in obtaining research funding and regulatory approvals, which conflicts with the real needs of health systems to implement immediate solutions. Research approaches also present particular challenges when partial implementation of an evidence-based intervention has already occurred within a given setting, such that proposing to withhold the intervention from certain patients (eg, in a control group) is unethical or not palatable to frontline clinicians and health system leaders. As an alternative to randomization at the patient level, stepped wedge deigns, in which an intervention is rolled out sequentially in different units or health systems, can add rigor to QI approaches by allowing areas that have not yet implemented the intervention to serve as controls. Additionally, the integration of analytic methods such as interrupted time series into QI work can highlight the specific contributions of interventions relative to temporal trends.
Given the limitations of both QI and traditional research approaches, what can the work by Afzal et al5 and Winner et al7 teach us about using QI to address challenges in care? First, Afzal et al5 applied the fundamental QI tenet of testing a small, manageable change prior to scaling. During the study period, the boarding stabilization protocol was piloted among approximately one-quarter of patients seen in the ED observation unit, while subsequently the protocol was adopted as the standard approach for all patients presenting with suicidal thoughts. Second, both studies employed balancing measures (eg, ED revisits within 30 days, behavioral health unit length of stay) to monitor for unintended consequences of the interventions. Finally, both studies deployed interventions that were rooted in prior evidence and likely to be beneficial to patients and their families with little risk of causing harm. In contrast, the testing of novel interventions that pose greater than minimal risk to patients should be reserved for research studies that seek informed consent and assent, rather than QI.
One additional common feature of QI work, applicable to both studies in this issue, is worth highlighting—QI interventions are often designed by health care professionals with limited input from patients and families. Moreover, many QI efforts focus on safety, timeliness, or effectiveness, while neglecting patient-centeredness.9 To develop patient-centered health systems, the perspectives of health care consumers must be integrated into the design and deployment of QI interventions. As a complementary methodology to accomplish this goal, human-centered design involves cultivating a deep understanding of user needs and experiences, followed by rapid prototyping and testing of solutions.10 Recently, House et al applied human-centered design to generate solutions for pediatric mental health boarding, culminating in the design of a modular digital mental health curriculum and a clinical practice guideline.11 Moving forward, application of human-centered design principles alongside QI efforts could drive patient-centeredness to the forefront.
Taken together, the studies by Afzal et al5 and Winner et al7 illustrate how QI enables evaluation of iterative changes to improve pediatric acute mental health services within complex health systems. Using robust QI methods, including clear descriptions of context, illustrations of theory through driver diagrams, and rigorous analysis of time-series data with control charts, the authors carefully tested and implemented local changes that led to important improvements in outcomes. At the same time, it is essential to understand the limitations of QI and the relative advantages of alternative methodologies such as research and human-centered design. Ultimately, addressing the multifaceted and unprecedented pediatric mental health crisis will require a combination of approaches. As we work toward larger efforts as a health care community to address the root causes of the pediatric mental health crisis, these QI reports should encourage all of us to examine our local care systems for improvement opportunities that can be undertaken now to help keep children with mental health conditions out of the hospital and with their families.
Dr Hoffmann and Dr Patel drafted the commentary and reviewed it critically for important intellectual content; Both authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
CONFLICT OF INTEREST DISCLOSURES: The authors have no conflicts of interest to disclose.
FUNDING: This publication was supported by the National Institutes of Mental Health of the National Institutes of Health under Award Number K23MH135206 [to JAH]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
COMPANION PAPERS: Companions to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2023-063262 and www.pediatrics.org/cgi/doi/10.1542/peds.2023-064917.
Comments