Cost, Utilization, and Patient and Family Experience With ACO-Based Pediatric Care Management Free

The pediatric integrated care management program is embedded in primary care practices owned by or affiliated with a large health system in Massachusetts, including academic and community hospitals, primary, and subspecialty care. The system participates in Medicare, Medicaid, and commercial ACOs with over 700 000 covered lives, including 160 000 children.

Care managers who are registered nurses (RNs), social workers (SWs), and community health workers (CHWs) deliver services to patients and their caregivers through shared office visits with the primary care provider (PCP), telephone calls, and visits to the home or community, including schools. Care managers initially contact families through telephone or PCP warm-handoff, and caregivers must agree to program participation. Care managers assess patient needs using a structured High-Risk Assessment (Supplement 1) and develop family-centered care plans. They coordinate care with specialists and inpatient teams, conduct postdischarge follow-up calls, find community resources for medical, behavioral health, educational, or social needs, and coach caregivers in managing their child’s health conditions. Patients are discharged when their care needs are met, they are no longer able to advance their goals with program resources, or they leave the health system.

Eligible patients are identified either by a high-risk algorithm followed by PCP validation or by direct PCP referral. The high-risk algorithm identifies members of commercial or Medicaid ACO risk contracts with high Johns Hopkins Adjusted Clinical Group (ACG) total medical expense prospective risk scores,^25,26  high Emergency Department (ED) or inpatient (IP) utilization, multiple prescriptions, and specific combinations of chronic conditions. PCP validation or referral leverages their established relationship to identify patients using a multidimensional lens, including social need and family stress, that is not available through algorithms alone. Patients are paired with an appropriate discipline care manager lead based on complex medical, behavioral health, or social needs, and may transfer care to a different discipline lead as needed.

We conducted a claims-based analysis to understand the effects of the program on TME and utilization. Included patients entered the program January 1, 2017 to December 31, 2020, had a risk-based commercial or Medicaid insurance plan with a PCP in the ACO, and had ≥1 month of claims before and after enrollment. Analyses included data from July 1, 2016, 6 months before the first potential enrollment, through January 31, 2021, 1 month after the last potential enrollment. Patients younger than 1-year at enrollment were excluded because of unique utilization patterns in the first year of life.¹⁹ 

Primary outcomes included per member per month (PMPM) TME with secondary outcomes of ED, inpatient (IP), observation, and PCP utilization. TME included fully adjudicated claims paid by insurance, patient deductibles, and copays. Cost is from the health system perspective and includes fully adjudicated paid claims rather than charges. TME excluded pharmacy costs because of data limitations and Medicaid substance use disorder costs because of confidentiality regulations; all other costs were included. We conducted sensitivity analyses including available pharmacy claims (Supplement 2B), which yielded similar results. ED visit counts excluded those resulting in observation or inpatient stays, and observation visit counts excluded those transferring to inpatient admission. Inpatient stays included all medical, surgical, and psychiatric hospitalizations, excluding postacute care. Same-day transfers were grouped into a single admission. PCP visits included visits to any primary care provider. For stratified analyses, we used 1 combined utilization outcome of ED and IP.

We used a quasi-stepped-wedged design including a fixed-effects, repeated measures model that leveraged the differential timing between program eligibility and enrollment across patients to approximate a series of difference-in-differences (Supplement 2A). This approximation of difference-in-differences has been applied in the evaluation of adult care management programs.¹⁶  It was not possible to identify a separate control population for analysis because the program was universally available. To create a comparison group, the analytic model used patient-level fixed effects to estimate within-person changes in utilization from preenrollment to postenrollment while including monthly fixed effects to control for temporal trends, such as seasonal utilization differences caused by external forces. In this manner, patients contributed to a control population in the months they were eligible but not yet enrolled in the program and to an intervention population in the months after they enrolled. The monthly indicator removed the effect of utilization changes over time that were not specific to the intervention. Time-invariant patient characteristics and interactions between these characteristics were not included in the model because they would not contribute to the intervention effect determined by within-person change. Instead, we used stratification to explore impact in specific patient groups.

Patients were eligible for enrollment at care initiation with the ACO PCP. Monthly data were aggregated into 6-month intervals from 1 month to 2 years postenrollment. Data were restricted to patients with at least 1 member-month (MM) in the eligibility period and 1 MM in the post-enrollment period. Patients only contributed data to the intervention effect in the postenrollment period months where they had data; no missing data were imputed. TME was analyzed using a generalized linear model with log link and Tweedie distribution, with mean-variance relationship chosen using the modified Park test and 80% of potential MMs available for analysis. PMPM utilization was analyzed using generalized linear models, and negative binomial models provided rate reduction estimates. In all linear models, absolute change estimates were anchored to baseline TME or utilization in 2019. We conducted sensitivity analyses by varying length of eligibility, minimum ACO MMs, and excluding patients without at least 1 eligibility MM and 1 postenrollment MM before coronavirus disease 2019 (COVID-19) (March 2020), which yielded consistent results (Supplement 2B).

To explore program impact on subpopulations, we stratified results by care manager lead discipline, behavioral health diagnosis at enrollment, ACG risk score, and payor. Diagnoses were ascertained by ACGs, with data available for 94% patients in the claims-based analysis and 67% patients in the survey analysis. Strata were not mutually exclusive. In this exploratory analysis, each group was modeled separately to consider the differential efficacy of the program by patient characteristics. Direct statistical comparison between groups was not possible because of small size and power limitations.

We conducted a cross-sectional survey of program participants enrolled ≥6 months to assess perceived quality of care coordination, unmet needs, and patient and family satisfaction. The survey instrument included 19 structured and 1 free-text questions (Supplement 3), including validated items from the National Survey of Children’s Health (NSCH) and internal items previously used for program quality improvement. Quality was assessed using measures of experience with care management support for medical, behavioral, educational, and health-related social needs. Caregivers and patients ≥18 years were surveyed using e-mail, phone, and paper questionnaires in English or Spanish.

The survey was administered by a specialized team not involved in program operations from July to October 2021. Patients who preferred a language other than English or Spanish, discharged from the program, or without viable contact information were ineligible. To minimize caregiver survey burden, 1 child per household was randomly selected. Data were summarized using descriptive statistics, and we compared subgroups using χ-squared tests.

The survey team conducted semistructured interviews with a purposefully-sampled subset of caregivers who participated in the survey. Caregivers were sampled for range of program satisfaction, patient clinical and demographic characteristics, insurance, length of enrollment, and care manager lead discipline. Participation was limited to English speakers because of the language capabilities of the interviewers. Two reviewers on this team independently coded the transcriptions to identify key themes.

For all analyses, statistical evaluation was performed using SAS 9.4 software. A 2-sided P ≤ .05 defined statistical significance and P values were not adjusted for multiple comparisons with exploratory stratification analyses. The claims-based evaluation was approved by the Mass General Brigham Institutional Review Board; the survey and interviews were conducted primarily for quality improvement purposes.

The study population included 1321 children for claims analysis, 276 survey respondents, and 13 interview participants (Table 1, Supplement 2 C and D). The claims population was majority male (61.2%) and was generally representative of the race and ethnicity composition of the health system pediatric ACO population, including over half non-Hispanic (54.7%) and white (55.0%). The majority were in a Medicaid ACO (70.6%) and more than half had a behavioral health diagnosis at enrollment (60.6%). SWs were the most common care manager lead discipline (48.5%), followed by CHWs and RNs.

Survey respondents included 276 patients and caregivers (48.3% response rate). The survey was predominantly completed by caregivers (96.9%) and 10.6% completed in Spanish. Compared with the claims population, survey participants had longer median program enrollment (22.3 vs 8.9 months), higher median ACG risk scores (2.1 vs 1.2), and majority had a RN care manager lead (51.4% vs 24.6%).

Of survey respondents, 42 caregivers agreed to participate in semistructured interviews. Purposefully sampled interviewees included 13 caregivers of a diverse group of patients (54% female, 46% non-Hispanic white, 62% Medicaid enrolled, mean age 10).

TME significantly declined within 6 months after program enrollment and continued to significantly decline through 19 to 24 months postenrollment (−$645.48 PMPM, P < .001; 95% CI −$589.78 to −$695.69) (Fig 1). ED and IP utilization significantly declined 7 to 12 months postenrollment compared with eligibility and persisted to 24 months post-enrollment (Fig 2). At 19 to 24 months postenrollment, the rate of ED utilization had decreased by 29% (P = .012; 95% CI −46% to −7%) and the rate of IP utilization had decreased by 82% (P < .001; 95% CI −91% to −65%). There was no effect on observation stay utilization. Notably, PCP visits significantly decreased immediately after enrollment and continued to have significant rate reductions through 24 months postenrollment (−44%, P < .001; 95% CI −51% to −35%).

Caregivers and patients were generally very satisfied with the program: 67.6% reported usually getting as much help as desired coordinating care and 87.2% were very or somewhat satisfied with the help they received (Table 2). Respondents reported speaking with care managers about many care coordination topics, most frequently primary care follow-up (77.1%), plan if feeling worse after-hours (73.7%), and family stress (71.8%). Although there was a range in the topics discussed, survey respondents almost always found these conversations to be very helpful (72.4% to 85.1%). Most respondents (56.1%) reported care managers contacted the school when needed and 90.9% were very satisfied with this interaction. Despite program enrollment, 18.5% of respondents reported unmet care needs, the majority of which were in mental health services.

Caregivers who participated in interviews described positive experiences with a personal care manager who knew their family and the health system well, coordinated different pieces of their child’s care, and supported caregivers to address their own stress and advocate for their child (Table 3). They noted improvement opportunities to better describe the scope of program offerings, connect to community resources, and improve pediatric to adult transitions.

We stratified claims and survey results into key patient subgroups: care manager lead discipline, behavioral health diagnosis at enrollment, ACG risk score, and payor (Table 4). There were no significant differences in acute utilization (ED or IP) based on care manager lead discipline, with similar significant reductions beginning at 7 to 12 months and persisting through 19 to 24 months in all groups. There were no differences between groups in top-box survey responses for overall program satisfaction, perceived quality as measured by educational communication, unmet needs, or frustration getting care; however, the CHW sample size was small (n = 15).

The program significantly reduced acute utilization much earlier, at 7 to 12 months post-enrollment, for patients with a behavioral health diagnosis compared with 19 to 24 months postenrollment for those without behavioral health conditions. At 7 to 12 months postenrollment, patients with a behavioral health diagnosis had 37% lower acute utilization (P < .01; 95% CI −50% to −22%) relative to eligibility. There were no significant differences in top-box survey responses for satisfaction, educational communication, unmet needs, or frustration. However, a small group of respondents for patients with behavioral health diagnoses reported being very unsatisfied (9% vs 2%, P = .05) and always frustrated getting care (9% vs 2%, P = .06), and interviewed caregivers also noted difficulty accessing behavioral health services (Table 3).

The program significantly reduced acute utilization only for those patients with an above-median (>1.2) ACG risk score, with significant reductions observed within the first 6 months (−20%, P = .02; 95% CI −37% to −4%) and rising to 54% reduction (P < .01; 95% CI −67% to −37%) in months 19 to 24 postenrollment relative to eligibility. No reductions in acute utilization were observed among children with below-median risk scores. Significantly fewer survey respondents for children with above-median (>2.1) ACG risk scores reported no unmet needs or never being frustrated getting care relative to the below-median group. Top-box results for satisfaction and educational communication were similar across groups.

Care management significantly reduced acute utilization for patients with both Medicaid and commercial insurance beginning 7 to 12 months postenrollment. The statistically significant effect persisted to 19 to 24 months in Medicaid and was near-significant in the smaller commercial population. There were no differences between payors in top-box survey responses for satisfaction, educational communication, unmet needs, or frustration getting care.

In this study, we evaluated the impact of an ACO-based, multipayor, primary care-embedded pediatric care management program on utilization, unmet needs, and patient and family experience using a mixed-methods approach. The program, including structured assessment, care planning, multisectorial coordination, and longitudinal follow-up, effectively reduced TME and utilization, while providing perceived high-quality care coordination with high patient and family satisfaction. To our knowledge, this is the first evaluation of a broad pediatric care management program based in primary care for both commercial and Medicaid ACO populations, validating the generalizability of this approach across patient groups. The operating cost of the program, including administrative and clinical staff, was approximately $120 PMPM, leading to >3:1 return on investment in a blended payor model.

Our intervention differs from those of prior published programs in several ways. First, the program was embedded in primary care and available to patients with complex and noncomplex chronic conditions. Groups including Bergman et al presented effective programs for children with medical complexity delivered in specialized clinics,²²  which may not be generalizable to broader ACO populations. Second, our program showed efficacy in both Medicaid and commercial ACO populations. Although Rubin et al described an ACO primary care-based program that significantly reduced inpatient utilization in the 2 years post-enrollment, only children with Medicaid were included.²¹  Third, our program enrolled patients with both disease-based risk and high utilization, and it required primary care engagement and active caregiver participation. Caskey et al described a payor-based program for children with medical, behavioral, and social needs in a randomized-controlled evaluation up to 12 months post-enrollment; however, it did not significantly reduce TME or utilization.²⁰  Patients were eligible by diagnosis and passively enrolled, potentially selecting lower-utilizing and nonengaged patients.

In our evaluation of multipayor, primary care-based care management, the program significantly reduced TME and ED or IP utilization through 19 to 24 months post-enrollment. Because median enrollment was 8.9 months, this represents persistent post-program reduction. We hypothesize this is because the program enabled caregivers to better manage their child’s health condition, which is substantiated in interviews where caregivers describe feeling more empowered to manage conditions independently. Notably, PCP utilization decreased immediately after enrollment, where effects on outpatient visits were mixed in other studies.^21,22  On average, care managers had 8 annual encounters with families, including 5 in the first 6 months of enrollment. In the survey, caregivers reported discussing topics that may have been previously addressed by PCPs and interviewed caregivers described the multidisciplinary team, effectively substituting some PCP visits. This decreased PCP utilization can add capacity and potentially decrease burnout in primary care.

In stratified analyses, the program only effectively reduced utilization for patients with above-median ACG risk scores. We hypothesize the lower-risk group may have had less utilization before enrollment, or because ACGs include both medical and behavioral health conditions, these patients may have had primarily social risk not included in the ACGs, such as health-related social needs or increased family stress. As we better understand the needs of specific patient subgroups, there may be opportunities to provide lower intensity interventions. Additional evaluation is needed on where the program becomes effective as this analysis was limited to a median cut-off.

The program effectively reduced utilization for patients with and without behavioral health conditions, though more quickly for those with behavioral needs. We hypothesize that patients with behavioral health diagnoses may have had more modifiable utilization or needs more readily addressed by care managers. This is substantiated both by prior evaluations of the NSCH where children with mental health diagnoses were more likely to have unmet care coordination needs and with national provider surveys describing systemic challenges accessing pediatric mental health care.^27–29  There is also increasing evidence that behavioral health conditions may drive pediatric utilization, and care management for cooccurring behavioral health needs is important for improving pediatric spending and population health.^30–32 

Despite the intervention, some surveyed caregivers of patients with behavioral health needs described low satisfaction and high frustration, showing persistent gaps in the child mental health system. Until there is improvement in specialty access, patient and family experience in care management may be improved through better communication, use of psychiatry access programs, and integrated primary care resources.^33–35 

This study has several potential limitations. First, because universal program availability within the ACO was prioritized, randomization was not possible, and a retrospective matched cohort of nonparticipants could not be created because of discretion in PCP referrals. It was not possible to identify whether participants were first identified by algorithm or by their PCP and which components of the intervention correlated most strongly with the positive outcomes. TME estimates may also be biased if earlier enrolled children were different from those enrolled later because of factors affecting TME. Second, data were collected during routine clinical care rather than for research, leading to potential misclassification and missing data, including substance use disorder claims. Third, the analysis was limited to patients with insurance participating in primary care in 1 health system and may not be generalizable to all geographies and care settings. However, despite these limitations the study offers a research model for rigorous evaluation of the multidimensional impact of clinically implemented pediatric care management programs.

Pediatric care management based in primary care across a large health system participating in multipayor ACOs can effectively reduce TME and utilization and can provide high-quality care coordination with high patient and family satisfaction. Such programs can provide multifactorial benefits for CYSHCN, including reducing care fragmentation, unmet needs, and unnecessary utilization. Health systems including children may implement multidisciplinary pediatric care management as a key strategy across payor groups.

ACO

accountable care organization

ACG

adjusted clinical group

CHW

Community Health Worker

CYSHCN

Children and Youth with Special Health Care Needs

ED

emergency department

IP

inpatient

MM

member month

NSCH

National Survey of Children’s Health

PCP

primary care provider

PMPM

per member per month

RN

registered nurse

SW

social worker

TME

total medical expense