Video Abstract

Video Abstract

Close modal
OBJECTIVES

To evaluate changes in access to and utilization of behavioral health (BH) services after the integration of psychologists into primary care clinics compared with clinics without integrated psychologists.

METHODS

We integrated 4 of 12 primary care clinics within our academic health system. We used the median wait time for BH services as a proxy for changes in access and defined BH utilization as the percentage of primary care visits that resulted in contact with a BH clinician within 180 days. We compared changes in access and utilization from the year before integration (September 2015 to September 2016) with the 2 years after integration (October 2016 to October 2018) within integrated clinics and between integrated and nonintegrated clinics. We used difference-in-difference analysis to test the association of study outcomes with the presence of integrated psychologists.

RESULTS

Access and utilization were similar across all practices before integration. After integration, BH utilization increased by 143% in integrated clinics compared with 12% in nonintegrated clinics. The utilization of BH services outside of the medical home (ie, specialty BH service) decreased for integrated clinics only. In clinics with integrated psychologists, 93% of initial BH visits happened on the same day as a need was identified. The median wait time for the 7% in integrated clinics who were not seen on the same day was 11.4 days (interquartile range = 5.3–17.7) compared with 48.3 days (interquartile range = 20.4–93.6) for nonintegrated clinics.

CONCLUSIONS

A team-based integration model increased BH utilization and access.

What’s Known on This Subject:

Various models of integration have revealed improved access and utilization of care; however, few studies have evaluated single-component, highly integrated, team-based models of care within urban pediatric clinics.

What This Study Adds:

A highly integrated, team-based model of integrated primary care increases access to and utilization of behavioral health services in contrast to clinics without integrated behavioral health services.

Behavioral health (BH) disorders affect approximately one-half of youth, and 20% will experience severe functional impairment as a result.1  When BH concerns arise, most patients and families seek the advice of their primary care clinician (PCC).24  Although some BH problems can be managed by a PCC, many children may benefit from working with a BH clinician. Unfortunately, the facilitation of referrals to specialty BH services from primary care has not sufficiently met pediatric BH needs, even when enhanced referral pathways are created.5,6  Up to one-half of youth with a BH disorder never receive treatment, and there are significant inequities in who receives care.710  For those who receive treatment, the median latency between symptom onset and treatment is approximately a decade, and many BH problems become more difficult to treat the longer treatment is delayed.1114 

PCCs are increasingly managing BH needs due to rising demand and limited access to BH services.15,16  However, compared with visits addressing medical concerns, primary care visits addressing BH concerns are significantly longer with lower rates of reimbursement,17  and PCCs report inadequate training in treating BH problems beyond attention-deficit/hyperactivity disorder (ADHD).1821  Thus, it is not surprising that patients and families often report that BH needs go unmet in primary care2,3,22  despite their preference to receive BH care in this setting.2,4 

There is growing evidence that the integration of BH providers within pediatric primary care (referred to herein as integrated primary care [IPC]) may increase access to BH services, improve outcomes, and reduce racial and ethnic disparities.2328  Although promising, existing research on pediatric IPC models has focused on delivering traditional outpatient therapy in the primary care setting separate from the rest of the primary care team and workflow (ie, co-location) or for a narrow range of clinical disorders (eg, Collaborative Care Model).23,2629  This is despite effective adult IPC models that prioritize generalist, team-based care integrated within the primary care workflow (eg, joint appointments, shared treatment planning).30,31  Although the number of studies is limited, a previous meta-analysis found that more highly integrated pediatric integration models may outperform less collaborative co-located models.28  The few articles that do describe higher levels of integration often use multiple interventions simultaneously (eg, screening, psychiatric consultation, PCC training), which makes it difficult to determine the unique effects of BH integration.24,32,33  In addition, pediatric IPC studies rarely include patients <4 years of age, patients living in geographically urban locations, or minoritized youth.27 

We implemented a team-based integration model within 4 of our urban pediatric primary care clinics to increase access to and utilization of BH services. The specific aims of this project were to evaluate the impact of this care delivery model on:

  1. Access to BH services as measured by the change in wait to first BH contact for patients seen by integrated psychologists compared with specialty BH.

  2. Utilization of BH services as measured by the percentage of primary care appointments that result in contact with a BH provider within 180 days.

  3. Demographic differences in the utilization of BH services.33 

  4. The utilization of BH services outside of the primary care setting (ie, specialty BH services).

During our evaluation period (September 2015 to October 2018), our academic health center’s primary care network included 12 clinics. All clinics are located in underserved urban communities. The primary care network averaged 134 992 visits per year for a total of 404 977 patient visits over the course of the project. Most patients are Medicaid-enrolled and members of our accountable care organization (see Table 1 for demographic details). In October 2016, a psychologist was integrated into each of the 4 primary care clinics 4 days per week, with 1 day each week for administrative activities. At the start of integration, our institution employed 325.8 full-time equivalent (FTE) specialty psychologists and therapists who worked in the hospital, local schools, and community outpatient centers. Total effort increased by 61.5 by the end of the evaluation period (October 2018) to a total of 387.3 FTE. We ended the evaluation period in 2018 because of later changes that occurred across the primary care network, including the expansion of the number of clinics, total patient volumes, and processes. Integrated clinics were larger, serving an average of 11 047 unique patients per clinic site compared with 8001 unique patients over the evaluation period. All clinics were located in the same metropolitan area and were similarly distanced from specialty BH locations.

TABLE 1

Demographic Characteristics of Integrated and Non-Integrated Clinics Across the Entire Evaluation Period (Baseline and Integration)

Integrated Clinics (n = 44 188)Nonintegrated Clinics (n = 64 011)
Ages 
 0–2 y 31.0% 30.3% 
 3–5 y 17.8% 18.3% 
 6–12 y 33.2% 32.5% 
 ≥13 18% 19.0% 
Race and ethnicity 
 Non-Hispanic Black 55.4% 52.4% 
 Non-Hispanic White 17.1% 17.8% 
 Hispanic/Latino* 16.1% 14.1% 
 Other* 11.3% 15.6% 
Language 
 English 68.7% 66.3% 
 Spanish 13.6% 12.2% 
 Somali* 10.3% 6.9% 
 Nepali* 1.0% 5.1% 
 Other* 6.4% 9.4% 
Insurance 
 Commercial* 8.5% 10.2% 
 Medicaid 83.3% 83% 
 Self/other 7.6% 6.7% 
Integrated Clinics (n = 44 188)Nonintegrated Clinics (n = 64 011)
Ages 
 0–2 y 31.0% 30.3% 
 3–5 y 17.8% 18.3% 
 6–12 y 33.2% 32.5% 
 ≥13 18% 19.0% 
Race and ethnicity 
 Non-Hispanic Black 55.4% 52.4% 
 Non-Hispanic White 17.1% 17.8% 
 Hispanic/Latino* 16.1% 14.1% 
 Other* 11.3% 15.6% 
Language 
 English 68.7% 66.3% 
 Spanish 13.6% 12.2% 
 Somali* 10.3% 6.9% 
 Nepali* 1.0% 5.1% 
 Other* 6.4% 9.4% 
Insurance 
 Commercial* 8.5% 10.2% 
 Medicaid 83.3% 83% 
 Self/other 7.6% 6.7% 
*

Indicates significant differences at the P < .05 level.

TABLE 2

Billing Codes

Behavioral Health Billing Codes Used in Integrated Primary Care and Specialty Behavioral Health
Psychotherapy CPT codes 
 90791 Intake interview 
 90832, 90834, 90837 Individual psychotherapy 
 90839, 90840 Crisis 
 90845 Psychoanalysis 
 90846, 90847 Family therapy 
 90848, 90849 Multiple family group psychotherapy 
 90853 Group psychotherapy 
 90875, 90876 Individual psychophysiological therapy incorporating biofeedback training of any modality 
 90880 Other psychiatric services or procedures 
Heath and behavior assessment and intervention codes 
 96156 Health behavior assessment or reassessment 
 96158, 96159 Individual health behavior intervention 
 96164, 96165 Group health behavior intervention 
 96167, 96168, 96170, 96171 Family health behavior intervention 
Nonbillable encounters 
 “No Bill” “dummy” codes 
Behavioral Health Billing Codes Used in Integrated Primary Care and Specialty Behavioral Health
Psychotherapy CPT codes 
 90791 Intake interview 
 90832, 90834, 90837 Individual psychotherapy 
 90839, 90840 Crisis 
 90845 Psychoanalysis 
 90846, 90847 Family therapy 
 90848, 90849 Multiple family group psychotherapy 
 90853 Group psychotherapy 
 90875, 90876 Individual psychophysiological therapy incorporating biofeedback training of any modality 
 90880 Other psychiatric services or procedures 
Heath and behavior assessment and intervention codes 
 96156 Health behavior assessment or reassessment 
 96158, 96159 Individual health behavior intervention 
 96164, 96165 Group health behavior intervention 
 96167, 96168, 96170, 96171 Family health behavior intervention 
Nonbillable encounters 
 “No Bill” “dummy” codes 

We integrated 4 of 12 primary care clinics to test the feasibility and impact of integration. The 8 nonintegrated clinics served as comparators. We chose clinics to integrate on the basis of several factors, including patient volume, community needs, and matching psychologists to the communities served (eg, race, ethnicity, and language when possible). When designing our integration model, we considered the needs of our community, the clinical and operational needs of PCCs, as well as gaps in our existing BH system. PCCs reported high rates of BH needs within the community, insufficient time to address BH concerns, stigma, long wait times, and family contexts that limited follow-through with referrals (eg, financial, transportation, and employment challenges).

We selected the Primary Care Behavioral Health (PCBH)27  integration model because it prioritizes joint, team-based visits to maximize the care families get during their primary care visits with brief follow-ups on the BH clinician’s schedule (typically 3 to 5 30- to 45-minute sessions) when indicated. Integrated psychologists provided generalist BH care to children aged 0 to 21.34  Patients who did not respond to IPC interventions, needed psychological testing, or had acute needs were referred to specialty BH services.

Compared with colocated models of care, the PCBH approach affords greater population reach, as well as requiring fewer transportation demands and fewer days off work and school because families do not have to return for separate BH and medical appointments.27  PCBH was chosen over the Collaborative Care Model because our system already had a telephonic psychiatric consultation line to address psychotropic medication questions, PCCs wanted support with the full range of patients rather than a specific clinical population, and it was deemed more feasible to employ a psychologist who can bill for their services opposed to a care manager, consulting psychiatrist, and regularly scheduled consultation time for the PCC. Our model met the criteria for level 6, the highest level of integration, on the Integrated Practice Assessment Tool35  and included all components of the PCBH model (see Supplemental Table 7 for PCBH components and Reiter & colleagues30  for an overview and how it contrasts with other IPC models).

After selecting the clinical model, we mapped existing clinic processes, developed new workflows that prioritized joint, team-based visits, created a data dashboard to track implementation metrics, and conducted regular implementation meetings. Psychologists billed current procedural terminology (CPT) for mental health diagnoses, health and behavior billing codes for physical health diagnoses (eg, adherence to asthma treatment plan), and used a “non-billable” code to track clinically meaningful encounters that did not meet billing requirements (wellness promotion,34  subclinical problems, and visits not meeting minimum time; see Table 3 for billing codes). Psychology charges were self-billed, not billed as “incident to.” IPC psychologists had the same institutional productivity expectation as specialty BH providers and delivered additional nonbillable services as need and capacity allowed.

TABLE 3

Most Common Diagnoses

ConditionPercentage of Patients
Adjustment 23.5% 
Behavior problems 17.9% 
Attention Deficit Hyperactivity Disorder (ADHD) 14.5% 
Medical concerns 7.6% 
Mood 6.6% 
Anxiety 6.4% 
General prevention 6.3% 
Feeding concerns 4.4% 
Sleep 4.0% 
Learning/school concerns 2.7% 
Autism Spectrum Disorder (ASD)/ Developmental Delays (DD) 2.3% 
ConditionPercentage of Patients
Adjustment 23.5% 
Behavior problems 17.9% 
Attention Deficit Hyperactivity Disorder (ADHD) 14.5% 
Medical concerns 7.6% 
Mood 6.6% 
Anxiety 6.4% 
General prevention 6.3% 
Feeding concerns 4.4% 
Sleep 4.0% 
Learning/school concerns 2.7% 
Autism Spectrum Disorder (ASD)/ Developmental Delays (DD) 2.3% 

Percentage of patients with primary diagnostic code associated with CPT code.

Our project includes a 1-year baseline period from October 2015 to September 2016 and 2 years of evaluation data from October 2016 to October 2018. Changes in access to and utilization of BH services were compared between integrated and nonintegrated clinics to account for the growth in specialty BH services and other institutional interventions to increase access to and utilization of BH services.

Demographic Characteristics

Given documented inequities in access to BH care6  and our mission to increase the equity of BH care, we disaggregated data by race, ethnicity, preferred language, age, and insurance type. Demographic data were reported by families at the time of the appointment and entered into the medical record by registration staff as a standard part of clinical care.

Utilization of Behavioral Health Services

We measured the utilization of BH treatment by calculating the percentage of primary care appointments each month that resulted in contact with a non-prescribing BH provider (masters- or doctoral-level clinician providing either specialty BH or IPC services) within 180 days from the primary care visit. BH contact was identified by a BH billing code associated with a visit within our health care system occurring within 180 days of an appointment in primary care. We selected 180 days to account for the delays in accessing specialty BH and to align with previous studies.6  The percentage of visits was calculated rather than using raw volume to account for the difference in visit volume between integrated (n = 193 541) and nonintegrated clinics (n = 211 436). We included all primary care appointments (for any presenting concern or age) in our calculations consistent with our generalist clinical model.

Access to Behavioral Health Services

We used wait times as a proxy for access to care. Wait times were defined as the number of days from the date a referral was placed to attendance at the first appointment with a non-prescribing BH provider as defined above.

Data for analysis were acquired from the electronic health record data warehouse. Descriptive statistics were used to describe the sample and summarize variables.

Aim 1: Assessing for Increased Access to BH Services

Access to BH services was assessed by comparing the median wait time from identification of a BH concern to initial contact with a BH provider (specialty and IPC) between integrated and non-integrated clinics and the interquartile range (IQR; IQR = Q1–Q3).

Aim 2: Assessing for Change in Utilization of BH Services

To test for significant change in BH utilization, we used a comparative interrupted time-series analysis with ordinary least-squares regression-based approach estimate using Newey-West standard errors in Stata 15.0.36  This allowed for both pre and post comparisons within integrated clinics as well as comparisons between integrated and control clinics. Autocorrelation was present at lag 1 but not at any higher lag orders (up to 6 tested); thus, our model specifying lag(1) correctly accounted for autocorrelation.37 

Aim 3: Assessing for Demographic Differences in Utilization of BH Services

We assessed differences in the utilization rate of BH services across demographics at baseline using the χ-square test with regression-based planned contrasts.33  Demographic changes in utilization postintegration were assessed by using a cross-sectional difference-in-difference comparison.33  This was accomplished by using regressions with Bonferroni corrections to test for differences in utilization rates pre and post-integration within each demographic category (ie, age, race, ethnicity, language, and insurance) between integrated and nonintegrated clinics.

Aim 4: Assessing for Reduced Use of Specialty Behavioral Health Services

We compared cross-sectional differences in the percentage of specialty BH contact pre- and post-integration within and between integrated and nonintegrated clinics.

This project was considered exempt by the institutional review board.

Integrated psychologists were involved in ∼5.7% of all primary care visits in integrated clinics, and 74% percent of all patient contacts (initial and follow-up) occurred as a joint, team-based appointment; 89.1% of contacts were billed. Of the billable contacts, 89.2% were mental health CPT codes and 10.8% were health and behavior. The 3 most common diagnoses were adjustment, behavioral problems, and ADHD (see Table 4). IPC psychologists billed 12% less than specialty psychologists in the first year, and 6% less in the second year.

TABLE 4

Comparative Time-Series Analysis Results

CoefficienttP95% Confidence Interval
Preintegration comparison of BH utilization between integrated and non-integrated clinics 
 Mean difference 0.331 1.11 .270 −0.261 to 0.921 
 Slope difference 0.017 0.40 .694 −0.067 to 0.105 
Postintegration comparison of BH utilization between integrated and non-integrated clinics 
 Mean difference 2.685 4.42 <.001 2.685 to 7.053 
 Slope difference 0.038 0.48 .634 −0.121 to 0.197 
CoefficienttP95% Confidence Interval
Preintegration comparison of BH utilization between integrated and non-integrated clinics 
 Mean difference 0.331 1.11 .270 −0.261 to 0.921 
 Slope difference 0.017 0.40 .694 −0.067 to 0.105 
Postintegration comparison of BH utilization between integrated and non-integrated clinics 
 Mean difference 2.685 4.42 <.001 2.685 to 7.053 
 Slope difference 0.038 0.48 .634 −0.121 to 0.197 

Overall model test characteristics. 10 obs F(7101) = 175.9, Prob > F<0.001.

Within integrated clinics, 93% of initial IPC contacts occurred during their primary care appointment, resulting in no wait time. Of the 7% of patients whose first contact occurred on the psychologist’s schedule, the median wait time to see an integrated BH psychologist was 11.4 days (IQR = 5.3–17.7) compared with 48.3 days (IQR = 20.4–93.6) for specialty BH. Appointments scheduled from referrals to integrated psychologists were completed within 30 days 90.8% of the time compared with 36.2% for specialty BH.

Comparative interrupted time series analysis (see Fig 1 and Table 5 for details) indicated there were no statistically significant differences in the utilization of BH services between integrated and nonintegrated clinics during the baseline period. In the 2 years after integration, IPC clinics experienced an immediate and sustained increase in the utilization of BH services. The percentage of primary care visits that resulted in contact with a BH clinician in 180 days increased from an average of 4.6% in the year before integration to an average of 11.2% during the first 2 years of integration. During this same time period, there was no significant change in utilization for nonintegrated clinics (4.0% to 4.4%).

FIGURE 1

Percentage of primary care visits that resulted in contact with a BH provider within 180 days of the appointment.

FIGURE 1

Percentage of primary care visits that resulted in contact with a BH provider within 180 days of the appointment.

Close modal
TABLE 5

Characteristics of Patients Seen by Behavioral Health (Specialty BH And IPC) in Integrated Compared With Non-Integrated Clinics Before and After Integration

Integrated ClinicsNon-Integrated Clinics
% with BH contact pre-integration% with any BH contact post– integrationChange% with BH contact pre-integration% with any BH contact post– integrationChange
Ages 
 0–2 y* 2.2a 9.9d 7.7 1.8h 4.1z 2.3 
 3–5 y* 7.2b 15.6e 8.4 6.6i 8.1y 1.6 
 6–12 y* 9.6c 19.9f 10.3 8.2j 11.4x 3.2 
 ≥13* 10.3c 20.4f 10.1 8.8c 11.4x 2.6 
Race and ethnicity 
 Non-Hispanic Black* 6.7a 14.9d 8.2 5.4h 7.8z 2.4 
 Non-Hispanic White* 10.7b 21.3e 10.7 9.3i 12.9y 3.7 
 Hispanic/Latino* 4.8c 14.6d 9.8 4.8h 7.6z 2.8 
 Other* 7.1a 16.4f 9.3 6.0h 7.6z 1.6 
 Not reported* 4.8c 11.6g 6.8 1.4j 2.8x 1.4 
Language 
 English* 8.7a 18.5d 9.8 7.3h 10.3z 3.0 
 Spanish* 4.9b 14.5e 9.6 4.6i 7.3y 2.7 
 Somali* 2.6c 6.4f 3.8 1.6j 3.8x 2.2 
 Nepali* 2.9c 14.4e 11.5 2.0j 4.1x 2.1 
 Other* 3.1c 10.0g 6.9 3.7k 4.5x 0.8 
Insurance 
 Commercial* 7.9a 15.9d 8.0 6.7h 9.9z 3.2 
 Medicaid* 7.2a 16.7d 9.5 6.2h 8.7z 2.5 
 Self/other* 4.0b 10.6e 6.6 3.5i 5.0y 1.5 
Integrated ClinicsNon-Integrated Clinics
% with BH contact pre-integration% with any BH contact post– integrationChange% with BH contact pre-integration% with any BH contact post– integrationChange
Ages 
 0–2 y* 2.2a 9.9d 7.7 1.8h 4.1z 2.3 
 3–5 y* 7.2b 15.6e 8.4 6.6i 8.1y 1.6 
 6–12 y* 9.6c 19.9f 10.3 8.2j 11.4x 3.2 
 ≥13* 10.3c 20.4f 10.1 8.8c 11.4x 2.6 
Race and ethnicity 
 Non-Hispanic Black* 6.7a 14.9d 8.2 5.4h 7.8z 2.4 
 Non-Hispanic White* 10.7b 21.3e 10.7 9.3i 12.9y 3.7 
 Hispanic/Latino* 4.8c 14.6d 9.8 4.8h 7.6z 2.8 
 Other* 7.1a 16.4f 9.3 6.0h 7.6z 1.6 
 Not reported* 4.8c 11.6g 6.8 1.4j 2.8x 1.4 
Language 
 English* 8.7a 18.5d 9.8 7.3h 10.3z 3.0 
 Spanish* 4.9b 14.5e 9.6 4.6i 7.3y 2.7 
 Somali* 2.6c 6.4f 3.8 1.6j 3.8x 2.2 
 Nepali* 2.9c 14.4e 11.5 2.0j 4.1x 2.1 
 Other* 3.1c 10.0g 6.9 3.7k 4.5x 0.8 
Insurance 
 Commercial* 7.9a 15.9d 8.0 6.7h 9.9z 3.2 
 Medicaid* 7.2a 16.7d 9.5 6.2h 8.7z 2.5 
 Self/other* 4.0b 10.6e 6.6 3.5i 5.0y 1.5 

Categories with the same superscript letter within each demographic variable are equal. Different letter groups were used for each column to avoid direct comparisons between columns. “Any contact” means contact with an integrated psychologist or specialty BH provider.

*

P < .05 for difference of difference comparison between integrated and non-integrated clinics.

TABLE 6

Percentage of Demographic Categories Seen by SBH or IPC Within Integrated Clinics During the Integration Period

Specialty BH (n = 2652)IPC (n = 4469)
Ages 
 0–2 y 2.3%a 7.6%z 
 3–5 y 6.1%b 9.5%y 
 6–12 y 7.8%c 12.1%x 
 ≥13 8.9%d 11.5%x 
Language 
 English 7.1%a 11.3%z 
 Spanish 4.4%b 10.1%z 
 Somali 2.7%c 3.8%y 
 Nepali 2.7%c 11.7%z 
Race 
 Non-Hispanic White 9.0%a 12.4%z 
 Non-Hispanic Black 5.5%b 9.4% y 
 Hispanic/Latino 4.4%c 10.2%x 
Insurance 
 Commercial 6.2%a 10.5%z 
 Medicaid 6.6%a 9.3%z 
 Self/other <0.01%b 7.2%y 
Specialty BH (n = 2652)IPC (n = 4469)
Ages 
 0–2 y 2.3%a 7.6%z 
 3–5 y 6.1%b 9.5%y 
 6–12 y 7.8%c 12.1%x 
 ≥13 8.9%d 11.5%x 
Language 
 English 7.1%a 11.3%z 
 Spanish 4.4%b 10.1%z 
 Somali 2.7%c 3.8%y 
 Nepali 2.7%c 11.7%z 
Race 
 Non-Hispanic White 9.0%a 12.4%z 
 Non-Hispanic Black 5.5%b 9.4% y 
 Hispanic/Latino 4.4%c 10.2%x 
Insurance 
 Commercial 6.2%a 10.5%z 
 Medicaid 6.6%a 9.3%z 
 Self/other <0.01%b 7.2%y 

Within each demographic variable, percentages without a common superscript differ at P < .01. We used a, b, c for specialty behavioral health contact and z, y, x for integrated primary care contact. Data represents a cross-sectional view of data during the integration period.

Table 1 reveals population-level demographic differences between integrated and nonintegrated clinics during the evaluation period, and Table 2 reveals BH utilization data across measured demographics. Of note, non-English-speaking parents and patients 0 to 2 years of age had lower rates of BH utilization, and non-Hispanic White patients had the highest levels of utilization within both integrated and nonintegrated clinics at baseline. Difference-of-difference analysis indicated that patients in integrated clinics had significantly more growth in utilization compared with nonintegrated clinics across all demographic categories, with the largest growth occurring for Nepali-speaking patients in integrated clinics. Somali-speaking parents did experience significantly more utilization in integrated clinics compared with nonintegrated clinics, but growth was notably smaller for these patients. Although integration significantly increased utilization for all demographics, non-Hispanic White and English-speaking youth were still more likely to receive contact with any type of BH provider (ie, specialty and IPC combined) within integrated clinics during the study period overall. However, there were significantly fewer and sometimes no demographic differences in the utilization of the IPC psychologist (see Supplemental Table 7).

Patients in integrated clinics were less likely to use specialty BH services (ie, non-prescribing BH services outside of the primary care clinic) compared with baseline and compared with nonintegrated clinics during the evaluation period. Specifically, contact with specialty BH reduced from 4.6% before integration to 3.5% after integration (25% reduction) and was statistically lower than the 4.4% of patients seen by specialty BH in nonintegrated clinics. Approximately 14% of patients in integrated clinics were seen by both IPC and specialty BH.

Integration resulted in a significant and immediate increase in the percentage of primary care visits that resulted in contact with a BH provider within 180 days. Given the long delays between symptom onset and treatment,11  it is notable that ∼25% of patients seen by a BH provider within integrated clinics were ≤5 years of age and 10% were ≤3 years of age. Patients accessed IPC psychologists quickly, with 93% of initial contacts occurring during the primary care appointment when a need was identified. Patients in integrated clinics were also less likely to use specialty BH services after integration. Thus, the PCBH model improved access to integrated, team-based care at the population level and reduced the need for specialty services consistent with the mission of primary care.38,39,40 

This study adds to the literature in several important ways. First, we used a highly integrated, team-based IPC model (PCBH). Second, our clinics serve a diverse and geographically urban population, an understudied population in IPC research.27,40  Third, we used a single-component approach to assess the unique effects of integration. Finally, we included comparator clinics within the same network to help account for system-level and historical confounds. Notably, the use of comparator clinics led to the surprising finding that substantial growth of specialty BH clinicians over the 2-year integration period did not lead to a significant increase in the utilization of BH services for patients in nonintegrated clinics. Given the large improvement in BH utilization observed after investing a single integrated FTE per clinic, integration proved to be a highly efficient model for increasing the utilization of BH services for our urban primary care population.

Given existing racial and ethnic disparities in care, we designed our services to be culturally informed and connected to the community (eg, participating in community events, partnering with community organizations).34  We prioritized team-based, same-day visits because of structural barriers limiting follow-up and matched psychologist demographics to the community when possible. These efforts likely contributed to the reduction of demographic disparities in BH care utilization within integrated clinics. However, disparities still remained in some areas, including in the use of specialty BH services, and we found only modest increases in utilization for Somali-speaking patients in integrated clinics. To address the modest improvements for Somali-speaking patients, we have engaged with Somali-oriented community organizations and hired a Somali-speaking therapist. Although integration typically improves linkage to specialty BH care,5  further investigation is needed to understand how integrated psychologists can address inequities in who accesses specialty BH services.

Although the use of comparator clinics is an advantage of this study, clinics were not randomly assigned, and some small baseline differences between integrated and nonintegrated clinics were observed. This quasi-experimental design has limits to its causal interpretations, although the observed effects were large. In addition, increased access to and utilization of BH care does not guarantee improved outcomes. Despite growing evidence supporting the effectiveness of single-session41  and brief co-located interventions,42  further research on clinical outcomes in pediatric PCBH is needed. Although most patients receive care within our large BH system of care, we were unable to track the use of specialty care outside of our institution. Additionally, self-reported language data kept within the medical record do not allow us to account for the possible differences for families who speak English as a second language and those who do not, as well as families with generational differences in language barriers, both of which are areas for future research to explore. Although the evaluation of the financial impact is beyond the scope of this paper, it is worth noting, given the challenges of funding IPC models (including PCBH43 ), that integrated psychology billing was similar to specialty psychologists in the second year of integration. Future studies should further explore the financial impacts of pediatric PCBH models.

This study adds to a growing body of literature that reveals the benefit of shifting some of the BH workforce “upstream” to support primary care in managing pediatric BH needs. Although the workforce is limited, integrating BH clinicians within primary care teams can result in patients being seen more quickly, equitably, and efficiently44  in the setting they prefer2  while reducing the demand for specialty BH services and making them more accessible for patients with more acute needs.45 

Drs Hostutler and Wolf conceptualized and designed the study, conducted the data analysis, and drafted the initial manuscript; Drs Snider, Butter, Butz, and Kemper conceptualized and designed the study and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no potential conflicts of interest relevant to this article to disclose.

BH

behavioral health

CPT

current procedural terminology

FTE

full-time equivalent

IPC

integrated primary care

IQR

interquartile range

PCBH

Primary Care Behavioral Health model

PCC

primary care clinician

1
Merikangas
KR
,
He
JP
,
Burstein
M
, et al
.
Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A)
.
J Am Acad Child Adolesc Psychiatry
.
2010
;
49
(
10
):
980
989
2
Riley
AR
,
Walker
BL
,
Wilson
AC
, et al
.
Parents’ consumer preferences for early childhood behavioral intervention in primary care
.
J Dev Behav Pediatr
.
2019
;
40
(
9
):
669
678
3
Combs-Orme
T
,
Holden Nixon
B
,
Herrod
HG
.
Anticipatory guidance and early child development: pediatrician advice, parent behaviors, and unmet needs as reported by parents from different backgrounds
.
Clin Pediatr (Phila)
.
2011
;
50
(
8
):
729
737
4
Petts
RA
,
Walker
BL
,
Hails
KA
, et al
.
Parents’ preferences for behavioral services in primary care during the COVID-19 pandemic
.
J Dev Behav Pediatr
.
2022
;
43
(
5
):
291
296
5
Petts
RA
,
Shahidullah
JD
.
Engagement interventions delivered in primary care to improve off-site pediatric mental health service initiation: a systematic review
.
Fam Syst Health
.
2020
;
38
(
3
):
310
322
6
Hacker
K
,
Arsenault
L
,
Franco
I
, et al
.
Referral and follow-up after mental health screening in commercially insured adolescents
.
J Adolesc Health
.
2014
;
55
(
1
):
17
23
7
Cunningham
PJ
.
Beyond parity: primary care physicians’ perspectives on access to mental health care
.
Health Aff (Millwood)
.
2009
;
28
(
3
):
w490
w501
8
Whitney
DG
,
Peterson
MD
.
US national and state-level prevalence of mental health disorders and disparities of mental health care use in children
.
JAMA Pediatr
.
2019
;
173
(
4
):
389
391
9
Merikangas
KR
,
He
J-P
,
Burstein
M
, et al
.
Service utilization for lifetime mental disorders in U.S. adolescents: results of the National Comorbidity Survey-Adolescent Supplement (NCS-A)
.
J Am Acad Child Adolesc Psychiatry
.
2011
;
50
(
1
):
32
45
10
Coker
TR
,
Elliott
MN
,
Toomey
SL
, et al
.
Racial and ethnic disparities in ADHD diagnosis and treatment
.
Pediatrics
.
2016
;
138
(
3
):
e20160407
11
Wang
PS
,
Berglund
PA
,
Olfson
M
,
Kessler
RC
.
Delays in initial treatment contact after first onset of a mental disorder
.
Health Serv Res
.
2004
;
39
(
2
):
393
415
12
Shaw
M
,
Hodgkins
P
,
Caci
H
, et al
.
A systematic review and analysis of long-term outcomes in attention deficit hyperactivity disorder: effects of treatment and non-treatment
.
BMC Med
.
2012
;
10
(
1
):
99
13
Post
RM
,
Leverich
GS
,
Kupka
RW
, et al
.
Early-onset bipolar disorder and treatment delay are risk factors for poor outcome in adulthood
.
J Clin Psychiatry
.
2010
;
71
(
7
):
864
872
14
Ghio
L
,
Gotelli
S
,
Marcenaro
M
, et al
.
Duration of untreated illness and outcomes in unipolar depression: a systematic review and meta-analysis
.
J Affect Disord
.
2014
;
152–154
:
45
51
15
Olfson
M
.
The rise of primary care physicians in the provision of US mental health care
.
J Health Polit Policy Law
.
2016
;
41
(
4
):
559
583
16
Olfson
M
,
Kroenke
K
,
Wang
S
,
Blanco
C
.
Trends in office-based mental health care provided by psychiatrists and primary care physicians
.
J Clin Psychiatry
.
2014
;
75
(
3
):
247
253
17
Meadows
T
,
Valleley
R
,
Haack
MK
, et al
.
Physician “costs” in providing behavioral health in primary care
.
Clin Pediatr (Phila)
.
2011
;
50
(
5
):
447
455
18
Horwitz
SM
,
Kelleher
KJ
,
Stein
RE
, et al
.
Barriers to the identification and management of psychosocial issues in children and maternal depression
.
Pediatrics
.
2007
;
119
(
1
):
e208
e218
19
Horwitz
SM
,
Storfer-Isser
A
,
Kerker
BD
, et al
.
Barriers to the identification and management of psychosocial problems: Changes from 2004 to 2013
.
Acad Pediatr
.
2015
;
15
(
6
):
613
620
20
McMillan
JA
,
Land
M
,
Leslie
LK
.
Pediatric residency education and the behavioral and mental health crisis: a call to action
.
Pediatrics
.
2017
;
139
(
1
):
e20162141
21
Nasir
A
,
Watanabe-Galloway
S
,
DiRenzo-Coffey
G
.
Health services for behavioral problems in pediatric primary care
.
J Behav Health Serv Res
.
2016
;
43
(
3
):
396
401
22
Inkelas
M
,
Raghavan
R
,
Larson
K
, et al
.
Unmet mental health need and access to services for children with special health care needs and their families
.
Ambul Pediatr
.
2007
;
7
(
6
):
431
438
23
Njoroge
WF
,
Hostutler
CA
,
Schwartz
BS
,
Mautone
JA
.
Integrated behavioral health in pediatric primary care
.
Curr Psychiatry Rep
.
2016
;
18
(
12
):
106
24
Walter
HJ
,
Vernacchio
L
,
Trudell
EK
, et al
.
Five-year outcomes of behavioral health integration in pediatric primary care
.
Pediatrics
.
2019
;
144
(
1
):
e20183243
25
Weersing
VR
,
Brent
DA
,
Rozenman
MS
, et al
.
Brief behavioral therapy for pediatric anxiety and depression in primary care: a randomized clinical trial
.
JAMA Psychiatry
.
2017
;
74
(
6
):
571
578
26
Burkhart
K
,
Asogwa
K
,
Muzaffar
N
,
Gabriel
M
.
Pediatric integrated care models: a systematic review
.
Clin Pediatr (Phila)
.
2019
;
59
(
2
):
148
153
27
Yonek
J
,
Lee
C-M
,
Harrison
A
, et al
.
Key components of effective pediatric integrated mental health care models: a systematic review
.
JAMA Pediatr
.
2020
;
174
(
5
):
487
498
28
Asarnow
JR
,
Rozenman
M
,
Wiblin
J
,
Zeltzer
L
.
Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: a meta-analysis
.
JAMA Pediatr
.
2015
;
169
(
10
):
929
937
29
Kolko
DJ
,
Hart
JA
,
Campo
J
, et al
.
Effects of collaborative care for comorbid attention deficit hyperactivity disorder among children with behavior problems in pediatric primary care
.
Clin Pediatr (Phila)
.
2020
;
59
(
8
):
787
800
30
Reiter
JT
,
Dobmeyer
AC
,
Hunter
CL
.
The primary care behavioral health (PCBH) model: An overview and operational definition
.
J Clin Psychol Med Settings
.
2018
;
25
(
2
):
109
126
31
Hunter
CL
,
Funderburk
JS
,
Polaha
J
, et al
.
Primary care behavioral health (PCBH) model research: current state of the science and a call to action
.
J Clin Psychol Med Settings
.
2018
;
25
(
2
):
127
156
32
Sheldrick
RC
,
Bair-Merritt
MH
,
Durham
MP
, et al
.
Integrating pediatric universal behavioral health care at federally qualified health centers
.
Pediatrics
.
2022
;
149
(
4
):
e2021051822
33
Lion
KC
,
Faro
EZ
,
Coker
TR
.
All quality improvement is health equity work: Designing improvement to reduce disparities
.
Pediatrics
.
2022
;
149
(
Suppl 3
):
e2020045948E
34
Snider
TC
,
Raglin Bignall
WJ
,
Hostutler
CA
, et al
.
Development and implementation of a culturally tailored early childhood program in an integrated pediatric primary care practice
.
Clin Pract Pediatr Psychol
.
2020
;
8
(
3
):
288
297
35
Waxmonsky
J
,
Auxier
A
,
Romero
PW
,
Heath
B
;
SAMHSA Center for Integrated Health Solutions
. Integrated practice assessment tool. Available at: https://www.thenationalcouncil.org/resources/integration-practice-assessment-tool-ipat/. Accessed July 17, 2016
36
Linden
A
.
Conducting interrupted time-series analysis for single-and multiple-group comparisons
.
Stata J
.
2015
;
15
(
2
):
480
500
37
Cumby
RE
,
Huizinga
J
.
Testing the autocorrelation structure of disturbances in ordinary least squares and instrumental variables regressions
.
Econometrica
.
1992
;
60
(
1
):
185
195
38
Foy
JM
,
Green
CM
,
Earls
MF
;
Committee on Psychosocial Aspects of Child and Family Health, Mental Health Leadership Work Group
.
Mental health competencies for pediatric practice
.
Pediatrics
.
2019
;
144
(
5
):
e20192757
39
National Academies of Sciences, Engineering, and Medicine
;
Health and Medicine Division
;
Board on Health Care Services
;
Committee on Implementing High-Quality Primary Care
.
Implementing High-Quality Primary Care: Rebuilding the Foundation of Health Care
.
Robinson
SK
,
Meisnere
M
,
Phillips
RL
. Jr,
McCauley
L
, eds.
Washington, DC
:
National Academies Press
;
2021
40
Lewis
FJ
,
Rappleyea
D
,
Didericksen
K
, et al
.
Bringing inclusion into pediatric primary health care: a systematic review of the behavioral health treatment of racial and ethnic minority youth
.
J Pediatr Health Care
.
2021
;
35
(
6
):
e32
e42
42
Schleider
JL
,
Weisz
JR
.
Little treatments, promising effects? Meta-analysis of single-session interventions for youth psychiatric problems
.
J Am Acad Child Adolesc Psychiatry
.
2017
;
56
(
2
):
107
115
41
Schleider
JL
,
Weisz
JR
.
Little treatments, promising effects? Meta-analysis of single-session interventions for youth psychiatric problems
.
J Am Acad Child Adolesc Psychiatry
.
2017
;
56
(
2
):
107
115
43
Valleley
RJ
,
Meadows
TJ
,
Burt
J
, et al
.
Demonstrating the impact of colocated behavioral health in pediatric primary care
.
Clin Pract Pediatr Psychol
.
2020
;
8
(
1
):
13
24
44
Freeman
DS
,
Manson
L
,
Howard
J
,
Hornberger
J
.
Financing the primary care behavioral health model
.
J Clin Psychol Med Settings
.
2018
;
25
(
2
):
197
209
45
Riley
AR
,
Paternostro
JK
,
Walker
BL
,
Wagner
DV
.
The impact of behavioral health consultations on medical encounter duration in pediatric primary care: a retrospective match-controlled study
.
Fam Syst Health
.
2019
;
37
(
2
):
162
166

Supplementary data