BACKGROUND AND OBJECTIVES:

The prevalence of mental health problems among adolescents in the United States is a major public health concern. However, the uptake of mental health treatment is low. Integrating behavioral health into primary care is 1 research-informed strategy to increase engagement in treatment. Facilitators of and barriers to implementation of integrated behavioral health in a pediatric primary care setting are not well delineated.

METHODS:

We examined the effectiveness of 2 mental health quality improvement strategies: an electronic referral and a social work follow-up protocol. We analyzed the following measures: uptake rate of first mental health appointments, overall use of mental health appointments, and first and overall mental health appointment show rate.

RESULTS:

Overall use rate improved after implementation of electronic referral, with 13 consecutive points above the median. First appointment show rates improved with a special cause run occurring after adding social work students to the mental health quality improvement team. First appointment show rate improved from a monthly average of 51% (November 2014 to March 2016) to 78% (April 2016 to December 2016). Use rate improved initially with increased efforts in assisting patients with scheduling; show rate improved more slowly after an emphasis on scheduling patients exhibiting treatment readiness.

CONCLUSIONS:

Findings suggest that a number of facilitators can increase the effective use of mental health services in an integrated adolescent clinical setting. These include an electronic referral through a shared electronic health record, multidisciplinary collaboration, and care management by social workers equipped with a variety of clinical and care coordination skills.

The prevalence of mental health problems among adolescents in the United States is a major public health concern. Of adolescents aged 13 to 18, almost 32% experience anxiety disorders, 19% experience behavior disorders, 14.3% experience mood disorders, and 11.4% experience substance use disorders.1 Approximately 22% of all adolescents surveyed experienced mental health disorders with severe impairment or distress.1 Uptake of mental health treatment of adolescents is low, despite the availability of effective identification2 and treatment methods.3 Only 36% of adolescents with a major depressive episode between the ages of 12 to 15 and 40% between 16 and 17 use treatment.4 

In our primary care and specialty adolescent medicine setting, the clinic confronted a number of problems with mental health service delivery despite having colocated therapists. In December 2013, a 2-week analysis was used to identify inadequate use of mental health therapists with monthly use rates (the number of appointments scheduled out of the total available appointments) as low as 52%. There lacked a formal process to refer patients or to track referrals. The mental health quality improvement (MHQI) team was formed to address these service gaps and to move toward an integrated model of behavioral health.

The effectiveness of integrated care in children and adolescents is demonstrated in emerging research.5 Asarnow et al’s6 systematic review revealed that, compared with youth receiving usual care, youth receiving integrated care were 66% more likely to have better behavioral health outcomes. A systematic review examining the integration of behavioral health within clinical settings7 found the number of studies on adolescents and young adults was low; accordingly, the authors recommend research on strategies to increase the likelihood of integrated care adoption and implementation.

Guidelines in research identify a number of facilitators of integrated or coordinated care, including the following: (1) early detection and screening, (2) care coordination, (3) protocols for identifying mental illness, (4) data management methods for monitoring referrals, and (5) access to psychiatric and/or mental health consultations.5,8,10 These factors were used to inform Richardson et al’s11 randomized controlled trial, in which they found adolescents receiving care through an integrated primary care model compared with those receiving standard care experienced greater decreases in depressive symptoms. Further research is needed on which barriers and facilitators contribute to successful implementation of an integrated behavioral health care model in pediatric primary care settings.

The MHQI project employs strategies on the basis of guidelines for implementing integrated care noted above. The project uses social workers to fulfill the care coordination role, which enhances care coordination to include the following: motivational interviewing to address treatment readiness and barriers; assessment for mental health disorders, suicidality, and safety; and brief counseling to bridge gaps in care.

The purpose of the MHQI project is to implement and evaluate a system for improving use and show rates of mental health appointments within an integrated behavioral health care model. We describe our methods of implementing these strategies in an academic clinical setting providing primary and specialty care to adolescents and young adults. Addressing the gaps in the research literature on implementation of integrated behavioral health models with this practice-based description, we illustrate the use of specific strategies to address common operational barriers to integrated care and to improve uptake of mental health treatment.

The Center for Adolescent and Young Adult Health (CAYAH) of the University of Pittsburgh Medical Center (UPMC) Children’s Hospital of Pittsburgh is a primary care and specialty integrated health care clinic serving patient ages 12 to 26. Patients present at the clinic for primary care or through self-referral or referral for consultation by a pediatrician for specialty adolescent concerns such as confidential sexual and reproductive health care, menstrual concerns, sexuality and gender identity concerns, eating disorders, behavioral health concerns, and complex psychosocial situations or chronic care management.

CAYAH uses an integrated health care model of interdisciplinary providers. Behavioral health providers embedded in the clinic share an electronic health record (EHR) with the medical providers, allowing for continuous collaboration. CAYAH is a multidisciplinary training site for trainees who collaborate while being exposed to the benefits of the integrated health care model.

The complex nature of the specialty care provided, the embedded mental health providers, and the presence of social work trainees with time and skills to complete project tasks created a unique opportunity to implement a quality improvement (QI) project to improve uptake of behavioral health services.

MHQI Project

In November 2014, a multidisciplinary team formed that included an attending adolescent medicine physician, a psychiatrist, a licensed social worker, graduate social work students, and an administrative assistant. CAYAH’s division head supported the team’s efforts through institutional advocacy and resources. The team implemented 2 strategies to enhance the system through which patients were referred to mental health and social work services. More essential to the MHQI project, however, are the monthly meetings when the team evaluates the effectiveness of the model for any current barriers to integration. The team collaborates to propose modifications to address identified barriers, allowing for the success of this iterative project.

Mental Health–Social Work Electronic Referral

The first key MHQI strategy was a mental health–social work (MH-SW) electronic referral (Fig 1) through the shared EHR, implemented in September 2014. The MHQI team worked diligently with the hospital data center over several months to develop the electronic referral; once developed, the team’s physician liaison trained providers in using the referral before official implementation in September 2014. The electronic referral allows medical providers to make a direct referral to on-site mental health or social work services. During a patient appointment, a provider may determine a patient has mental health concerns through assessment and screening. New and annual patients are routinely screened for depressive symptoms with Patient Health Questionnaire-2 screen12; if results are positive, providers use more extensive screening tools for depression (Patient Health Questionnaire-9)13,14 and anxiety (Generalized Anxiety Disorder-7)15 and a brief clinical interview.

FIGURE 1

Electronic referral through the shared EHR.

FIGURE 1

Electronic referral through the shared EHR.

Close modal

In many cases, the provider may use a warm referral, an in-person introduction to mental health providers or social workers if they are available in clinic. The mental health provider or social worker further assesses the patients’ symptoms and treatment readiness through clinical interview and motivational interviewing. If no warm referral occurs, the medical provider receives consent from the patient to receive a call from the social worker who also helps with appointment scheduling. In all cases, the medical provider submits an electronic referral form through the patient’s chart in the EHR. The social worker then accesses a report of all referrals generated weekly by the EHR.

Initially, the team focused efforts on encouraging provider engagement with warm referrals and the electronic referral, including social work trainees present for warm referrals, residents and medical trainees informed about the role of social work, and discussing progress of this project at biweekly staff meetings.

The electronic referral has 2 key functions: first, it ensures that every patient referred to mental health or social work will be contacted by the social work team to assist with arranging follow-up care, and second, it enables the MHQI team to collect data and analyze outcomes for all referred patients.

Social Work Follow-up Protocol

The second significant MHQI strategy was a social work follow-up protocol, first implemented in December 2014, which has iteratively evolved over time (Fig 2). This begins with either a warm referral or through accessing the weekly electronic referral report. Then, the social worker communicates with each patient and offers options on the basis of the patient’s needs, which may include referrals to outside resources, care coordination, or brief counseling or therapy within the clinic and assistance with appointment scheduling. In cases in which the patients are not ready to engage in treatment, the social worker advises the referring provider to follow-up with the patient at the next medical visit.

FIGURE 2

Social work follow-up protocol. SW, social work(er).

FIGURE 2

Social work follow-up protocol. SW, social work(er).

Close modal

Figure 2 details the social work follow-up protocol for patients referred to services.

Measures

Data collection began in November 2014 with tracking new referrals through the MH-SW electronic referral. Each month, social workers collect data for uptake and show rate for first appointments as well as overall (not only first appointment) use and show rates. Use rate and overall show rate data were available through the EMR beginning in September 2013 and were collected to present baseline data.

We analyzed and evaluated 4 key outcomes: (1) rate of patients who schedule the first appointment (number of first or new appointments scheduled out of the new MH-SW referrals made in the past month), (2) first or new appointment show rate (number of appointments attended out of first MH-SW appointments scheduled in the past month), (3) use rate of all mental health appointments (the number of all [first or new and returning] mental health appointments scheduled with the embedded therapist out of the total available appointments slots), and (4) overall show rate of all mental health appointments (number of all [first or new and returning] mental health appointments attended out of the total number scheduled).

Analysis

We used statistical process control charts generated by QI Macros, a program extension of Microsoft Excel.16 Control charts display data over time in a way that allows QI teams to evaluate variability of outcomes, resulting from interventions in a systematic way. Cheung et al16 outline the analysis of control charts for QI interventions using 4 rules or special causes, indicating that the outcomes did not occur randomly. We analyzed the charts for any rules and any corresponding interventions.

Ethical Consideration

The project was approved by the UPMC QI Committee. There were no ethical concerns.

There were no special causes in the study data of the rates of patients who schedule a first appointment (Fig 3). This rate averaged 65% over the study period. We hypothesize that a variety of factors impact the variability of this outcome, including patient treatment readiness and patients seeking treatment elsewhere.

FIGURE 3

Rate of patients who schedule first appointment out of those referred. LCL, lower control limit; UCL, upper control limit.

FIGURE 3

Rate of patients who schedule first appointment out of those referred. LCL, lower control limit; UCL, upper control limit.

Close modal

In Fig 4, we display the first appointment show rate of referrals made through the electronic referral. The improvement in first appointment show rate occurred gradually after adding social work students to the MHQI team to provide warm referrals and care coordination, resulting in a special cause run from April 2016 to November 2016. Show rates remained unchanged during the first year of the project. Later, the team shifted efforts to scheduling patients who displayed motivation for treatment through clinical interview with their medical provider or social worker. The patient’s readiness to engage in treatment was assessed most often during the patient’s medical appointment by the medical provider and social worker. This shift is reflected in the first appointment show rate that improved from a monthly average of 51% (November 2014 to March 2016) to 78% (April 2016 to December 2016).

FIGURE 4

First new appointment show rate of mental health or social work appointments.

FIGURE 4

First new appointment show rate of mental health or social work appointments.

Close modal

The 13 consecutive points from October 2014 to October 2015 above the median indicate a run and overall improvement to use rate due to the electronic referral implementation (Fig 5). During the beginning stages of the project, the MHQI team’s efforts focused on increasing the number of patients who scheduled their first MH-SW appointment. These efforts included care coordination with each patient referred for a social work or mental health need; the team called patients who missed their first appointment to assess barriers and helped them to reschedule. As mentioned, as the team prioritized increasing show rate by assessing patients’ readiness before scheduling appointments, the use rate lowered. The use rate decreased in August and September 2016. This could be due to lower patient volume over the previous summer months and/or due to fewer social work trainees present between semesters to facilitate referrals. The overall show rate remained high during these months. (Fig 6).

FIGURE 5

Use of mental health appointments.

FIGURE 5

Use of mental health appointments.

Close modal
FIGURE 6

Overall (new and returning) show rate of mental health appointments.

FIGURE 6

Overall (new and returning) show rate of mental health appointments.

Close modal

There was no improvement in overall show rate immediately after project implementation (Fig 6). The initial decrease reflects the team’s initial emphasis on scheduling patients (to increase use rate) rather than assess patient’s treatment readiness first. Additionally, the overall show rate remained steady until new patients who were part of the intervention had time to establish regular care. Beginning ∼1 year later, the 14 consecutive points from November 2015 to December 2016 above the median indicate an overall improvement in show rate of first and returning appointments from a monthly average of 67% (September 2013 to October 2015) to 77% (November 2015 to December 2016).

The purpose of this quality report is to describe how our practice implemented strategies to improve uptake of mental health treatment in the context of an integrated behavioral health model for adolescents and young adults and how this led to improved use of mental health treatment. Quantitative data analysis revealed that first appointment show rate and overall show rate for mental health appointments at CAYAH, since the start of the MHQI project, have steadily improved. The first appointment show rate improved from a monthly average of 51% to 78%; the overall (new and returning) show rate improved from 67% to 77%. Use rates initially improved and then decreased because of an emphasis on scheduling only patients ready to engage in treatment, determined by provider interview.

The improved first appointment show rate, overall show rate, and initial improvement in use rate of mental health appointments are likely attributable to these specific QI strategies and iterative changes made throughout the MHQI project. First, the use rate improved immediately after the introduction of the MH-SW electronic referral in the EHR and the social work follow-up protocol. This protocol involved early efforts in assisting patients with appointment scheduling. Next, the first appointment and overall show rates improved later, in January 2016 and November 2015, respectively. This improvement reflects facilitating uptake of treatment by increasing warm referrals as well as addressing potential barriers to treatment through motivational interviewing during appointment reminder calls.

These improvements are likely due to facilitating factors in CAYAH’s integrated model that are consistent with published standards for behavioral health integration. First, the social work team filled the care management and coordination responsibilities; care management facilitates implementation of integrated care with adults and adolescents.5,8,9,17,19 Second, CAYAH implemented an electronic referral to monitor behavioral health referrals; at CAYAH, it facilitated (1) improved communication between multidisciplinary providers, (2) a process for data management of referrals and social work follow-up, and (3) a method to track and evaluate MHQI interventions.5,8,10,17,18,20,22 Third, the process of integrating behavioral health was facilitated by a QI approach; this involved regularly scheduled meetings between a multidisciplinary team, including a physician as the liaison between the MHQI team and the clinical providers and staff, resulting in the ongoing identification of potential challenges that may otherwise have been missed.10,17,19,22 Finally, this project was greatly facilitated by the use of information technology, including an electronic referral within a shared EHR.5,21,22 

The MHQI project is a single case study, thus possibly limiting generalizability of the findings to other settings or patient populations. The process described provides an example that can be replicated in primary care settings that have access to similar resources such as medical and social work trainees, an EHR, and mental health therapists. For some settings, investing in additional specialized staff may be unrealistic, although medical staff or peer workers overseen by a care manager or social worker may be able to accomplish similar tasks.

There are potential limitations to the data collected and analyzed. First, in some cases, referrals may have occurred through warm referral but may not have been entered through the electronic order set. Second, the outcomes measured from the electronic referrals (uptake rate and first appointment show rate) may not fully represent the effectiveness of connecting patients to outside MH services. Future iterations of the MHQI project plan to collect this data as well.

In the project discussed, we strived to improve uptake rate, use rate, and show rate of mental health appointments through the use of an electronic referral and a social work follow-up protocol. In the findings, an improvement in first appointment show rate, use rate, and overall show rate of mental health and social work appointments over the 26 months after project implementation is displayed. The positive changes to these outcomes helped justify the presence for mental health providers in the clinic space and supported negotiation for the hospital administrative investment in future expansion of the mental health team.

In this article, we outline a concrete approach for improving mental health service delivery using an electronic referral through a shared EHR and social workers to fulfill a care coordinator role. Although the practice is especially relevant in clinical settings serving adolescents and young adults, this approach could be replicated in other primary care settings. In addition, with the results outlined here, we underscore the critical role of social workers in fulfilling the role of care coordinator. The skills of social workers allowed them to provide a range of support, including care management, assessing psychosocial barriers to treatment, connecting patients to community resources, and providing brief intervention counseling when needed. This created a more youth-centered and tailored model of care, from the initial warm referral to using motivational interviewing techniques to assess readiness, providing brief counseling, and connecting patients to higher levels of care in the community as appropriate.

With this project, we highlight the critical role of social workers in implementing a behavioral health model. The addition of QI strategies to increase uptake of mental health treatment can be implemented in primary care settings and help to justify the need to include social workers in primary care settings. The QI process and training social workers in QI strategies can support the sustainability of such models; for example, our results helped to justify the need to hire a second full-time social worker toward the end of this project. We suggest next steps to incentivize integrating mental health services into primary care settings and employing social workers to facilitate monitoring service delivery.

     
  • CAYAH

    Center for Adolescent and Young Adult Health

  •  
  • EHR

    electronic health record

  •  
  • MHQI

    mental health quality improvement

  •  
  • MH-SW

    mental health–social work

  •  
  • UPMC

    University of Pittsburgh Medical Center

Ms Peters collected data, supervised data collection, conducted data analysis, drafted the initial manuscript, and reviewed and revised the manuscript; Mr Sadler designed methods for data collection, collected data, supervised data collection, and reviewed and revised the manuscript; Dr Miller contributed to the project’s initiation and design and critically reviewed and revised the manuscript; Dr Radovic conceptualized and initiated the implementation of the project, supervised the project, and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: Supported by Children’s Hospital of Pittsburgh. Social work students involved in data collection were supported by a training grant to the University of Pittsburgh School of Social Work (Copeland, PI: Health Resources and Services Administration, G02HP27994).

We thank all the staff, providers, and administrators at the Center for Adolescent and Young Adult Health. We also thank the University of Pittsburgh’s School of Social Work, especially the 2015–2016 and 2016–2017 Cannon Fellows who, in addition to the primary author, included the following: Leslie Caresse, Amie DiTomasso, Aisling McIntyre, Edwin Sanchez, and Gregory Valdisera. We thank the UPMC Children’s Hospital of Pittsburgh’s Data Warehouse and Clinical Informatics; Dr Sarah Homitsky for help with initial pilot data and order set implementation; initial QI team members Dr Jonathan Pletcher, Dr Joanna Quigley, Paul Tedesco, Sarah Ingram, and Dr Orquidia Torres; and Dr Johanna Rosen for consultation on QI methods.

1
Merikangas
KR
,
He
JP
,
Burstein
M
, et al
.
Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A).
J Am Acad Child Adolesc Psychiatry
.
2010
;
49
(
10
):
980
989
[PubMed]
2
Wissow
LS
,
Brown
J
,
Fothergill
KE
, et al
.
Universal mental health screening in pediatric primary care: a systematic review [published correction appears in J Am Acad Child Adolesc Psychiatry. 2014;53(3):382].
J Am Acad Child Adolesc Psychiatry
.
2013
;
52
(
11
):
1134
1147.e23
3
March
J
,
Silva
S
,
Petrycki
S
, et al;
Treatment for Adolescents With Depression Study (TADS) Team
.
Fluoxetine, cognitive-behavioral therapy, and their combination for adolescents with depression: Treatment for Adolescents With Depression Study (TADS) randomized controlled trial.
JAMA
.
2004
;
292
(
7
):
807
820
[PubMed]
4
Substance Abuse and Mental Health Services Administration (SAMHS)
.
The CBHSQ Report: Serious Mental Health Challenges Among Older Adolescents and Young Adults
.
Rockville, MD
:
Substance Abuse and Mental Health Services Administration
;
2014
5
Wren
FJ
,
Foy
JM
,
Ibeziako
PI
.
Primary care management of child & adolescent depressive disorders.
Child Adolesc Psychiatr Clin N Am
.
2012
;
21
(
2
):
401
419, ix–x
[PubMed]
6
Asarnow
JR
,
Rozenman
M
,
Wiblin
J
,
Zeltzer
L
.
Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: a meta-analysis.
JAMA Pediatr
.
2015
;
169
(
10
):
929
937
[PubMed]
7
Richardson
LP
,
McCarty
CA
,
Radovic
A
,
Suleiman
AB
.
Research in the integration of behavioral health for adolescents and young adults in primary care settings: a systematic review.
J Adolesc Health
.
2017
;
60
(
3
):
261
269
[PubMed]
8
American Academy of Child and Adolescent Psychiatry
.
A Guide to Building Collaborative Mental Health Care Partnerships in Pediatric Primary Care
.
Washington, DC
:
American Academy of Child and Adolescent and Psychiatry
;
2010
9
Martini
R
,
Hilt
R
,
Marx
L
, et al
.
Best Principles for Integration of Child Psychiatry Into the Pediatric Health Home
.
Washington, DC
:
American Academy of Child & Adolescent Psychiatry
;
2012
10
Wissow
LS
,
van Ginneken
N
,
Chandna
J
,
Rahman
A
.
Integrating children’s mental health into primary care.
Pediatr Clin North Am
.
2016
;
63
(
1
):
97
113
[PubMed]
11
Richardson
LP
,
Ludman
E
,
McCauley
E
, et al
.
Collaborative care for adolescents with depression in primary care: a randomized clinical trial.
JAMA
.
2014
;
312
(
8
):
809
816
[PubMed]
12
Richardson
LP
,
Rockhill
C
,
Russo
JE
, et al
.
Evaluation of the PHQ-2 as a brief screen for detecting major depression among adolescents.
Pediatrics
.
2010
;
125
(
5
). Available at: www.pediatrics.org/cgi/content/full/125/5/e1097
[PubMed]
13
Kroenke
K
,
Spitzer
RL
,
Williams
JB
.
The PHQ-9: validity of a brief depression severity measure.
J Gen Intern Med
.
2001
;
16
(
9
):
606
613
[PubMed]
14
Johnson
JG
,
Harris
ES
,
Spitzer
RL
,
Williams
JB
.
The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients.
J Adolesc Health
.
2002
;
30
(
3
):
196
204
[PubMed]
15
Spitzer
RL
,
Kroenke
K
,
Williams
JB
,
Löwe
B
.
A brief measure for assessing generalized anxiety disorder: the GAD-7.
Arch Intern Med
.
2006
;
166
(
10
):
1092
1097
[PubMed]
16
Cheung
YY
,
Jung
B
,
Sohn
JH
,
Ogrinc
G
.
Quality initiatives: statistical control charts: simplifying the analysis of data for quality improvement.
Radiographics
.
2012
;
32
(
7
):
2113
2126
[PubMed]
17
Eghaneyan
BH
,
Sanchez
K
,
Mitschke
DB
.
Implementation of a collaborative care model for the treatment of depression and anxiety in a community health center: results from a qualitative case study.
J Multidiscip Healthc
.
2014
;
7
:
503
513
[PubMed]
18
Bauer
AM
,
Azzone
V
,
Goldman
HH
, et al
.
Implementation of collaborative depression management at community-based primary care clinics: an evaluation.
Psychiatr Serv
.
2011
;
62
(
9
):
1047
1053
[PubMed]
19
Whitebird
RR
,
Solberg
LI
,
Jaeckels
NA
, et al
.
Effective Implementation of collaborative care for depression: what is needed?
Am J Manag Care
.
2014
;
20
(
9
):
699
707
[PubMed]
20
Beach
SR
,
Walker
J
,
Celano
CM
,
Mastromauro
CA
,
Sharpe
M
,
Huffman
JC
.
Implementing collaborative care programs for psychiatric disorders in medical settings: a practical guide.
Gen Hosp Psychiatry
.
2015
;
37
(
6
):
522
527
[PubMed]
21
Peek
C
.
Lexicon for Behavioral Health and Primary Care Integration: Concepts and Definitions Developed by Expert Consensus
.
Rockville, MD
:
Agency for Healthcare Research and Quality
;
2013
22
Wagner
EH
,
Austin
BT
,
Von Korff
M
.
Organizing care for patients with chronic illness.
Milbank Q
.
1996
;
74
(
4
):
511
544
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.