OBJECTIVES:

A national quality measure in the Child Core Set is used to assess whether pediatric patients hospitalized for a mental illness receive timely follow-up care. In this study, we examine the relationship between adherence to the quality measure and repeat use of the emergency department (ED) or repeat hospitalization for a primary mental health condition.

METHODS:

We used the Truven MarketScan Medicaid Database 2015–2016, identifying hospitalizations with a primary diagnosis of depression, bipolar disorder, psychosis, or anxiety for patients aged 6 to 17 years. Primary predictors were outpatient follow-up visits within 7 and 30 days. The primary outcome was time to subsequent mental health–related ED visit or hospitalization. We conducted bivariate and multivariate analyses using Cox proportional hazard models to assess relationships between predictors and outcome.

RESULTS:

Of 22 844 hospitalizations, 62.0% had 7-day follow-up, and 82.3% had 30-day follow-up. Subsequent acute use was common, with 22.4% having an ED or hospital admission within 30 days and 54.8% within 6 months. Decreased likelihood of follow-up was associated with non-Hispanic or non-Latino black race and/or ethnicity, fee-for-service insurance, having no comorbidities, discharge from a medical or surgical unit, and suicide attempt. Timely outpatient follow-up was associated with increased subsequent acute care use (hazard ratio [95% confidence interval]: 7 days: 1.20 [1.16–1.25]; 30 days: 1.31 [1.25–1.37]). These associations remained after adjusting for severity indicators.

CONCLUSIONS:

Although more than half of patients received follow-up within 7 days, variations across patient population suggest that care improvements are needed. The increased hazard of subsequent use indicates the complexity of treating these patients and points to potential opportunities to intervene at follow-up visits.

What’s Known on This Subject:

A nationally endorsed pediatric quality measure is used to assess rates of timely follow-up for patients hospitalized for a mental health diagnosis. Rates range from 48% to 67%, but little is known about variations by patient characteristics or about the relationship between follow-up and a subsequent psychiatric crisis.

What This Study Adds:

Adherence to the national quality measure ranged from 62% to 82% but varied by patient and hospital characteristics. Adherence was associated with an increased risk of a subsequent mental health crisis, even after controlling for severity of illness.

The prevalence of children in the United States with a serious mental health disorder is estimated at 10%1 ; these children and youth are high users of emergency department (ED) and inpatient hospitalization, but many do not receive ongoing outpatient mental health care.26 

National quality measures for assessing quality of mental health care delivery may be critical to improving outcomes for children and youth with serious psychiatric disorders. In recognition of this, the Children’s Core Set of quality measures, created as a part of the Children’s Health Insurance Program Reauthorization Act of 2009, includes follow-up after hospitalization for mental illness at 7 and 30 days as a national quality measure. This is a Healthcare Effectiveness Data and Information Set (HEDIS) measure developed by the National Committee for Quality Assurance.7 

Rates of performance nationally for this measure range from 47.5% of children and youth hospitalized for a mental illness having a follow-up visit within 7 days of hospital discharge to 66.7% having at least one follow-up visit within 30 days of discharge.7,8  However, there are few studies on how this quality measure’s performance varies by race and/or ethnicity and socioeconomic, clinical, or hospital characteristics and whether this adherence to the quality measure is associated with improved patient outcomes. Answering these questions can inform efforts to improve quality, identifying patients at higher risk of poor-quality care and potentially worse outcomes.

To answer these questions and fill the gap in the literature, our study objectives, by using Medicaid data from 11 states, were to (1) assess outpatient follow-up after discharge for a mental health hospitalization for children ages 6 to 17, (2) assess variations in measure performance by child race and/or ethnicity and socioeconomic status and by clinical and hospital characteristics, and (3) assess how having follow-up care relates to repeat ED use and hospitalizations. For objective 3, we hypothesized that having follow-up within 7 or 30 days of discharge would be associated with a decreased risk of subsequent hospitalization or ED visit for a mental health disorder.

We used the Truven MarketScan Medicaid data from 2015 to 2016. The Truven database contains medical claims for inpatient, outpatient, and pharmaceutical use for Medicaid enrollees in 11 states representing all geographic regions of the United States (the Truven data use agreements preclude publication of state names). We used data only for patients with mental health and substance abuse coverage (98% of enrollees in the database).

We focused on mental health diagnoses that were found in our previous work to be common and costly diagnoses for pediatric inpatient mental health use: depression, bipolar disorder, psychosis, and anxiety.9  Our study cohort consisted of patients with a hospitalization (the index admission) for one of these as a primary diagnosis between January 2015 and June 2016. We identified these patients using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) codes, as done previously.10,11  The ICD-9-CM codes were aligned with the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition main diagnostic groups,10  and, subsequently, the ICD-10-CM codes were mapped from the ICD-9-CM and tested by using a national administrative database after the switch to the ICD-10-CM.11  We restricted the population to patients aged 6 to 17 years at the time of their hospitalization. This age range is consistent with previous claims-based studies of the mental health diagnoses of focus, reflecting the low prevalence in young patients and, therefore, small sample size in this group.9,12  To ensure completeness of use estimates after hospitalization, only children with continuous Medicaid enrollment for 6 months after discharge were included. Transfers from one inpatient setting to another (eg, general hospital to inpatient psychiatric facility) were treated as one episode. Finally, to ensure consistent measurement of outpatient use, patients who were not discharged from the hospital to home (eg, those discharged to a partial hospitalization program) were excluded.

Our predictors of interest were whether the patient had a mental health outpatient follow-up visit within 7 or 30 days after discharge from the index admission. The visit was defined as any Medicaid-reimbursed visit with one or more of the following: (1) a mental health diagnosis defined by using the International Classification of Diseases code definitions noted above10,11 ; (2) a visit defined by using the “standard providers” variable in the database, by using values for psychiatry, psychology, therapy (excluding physical, occupational, or speech), and mental health facility; or (3) a visit defined by using Truven’s proprietary binary variable indicating a service subcategory of mental health (see Supplemental Table 5 for the distribution of these definitions and Supplemental Table 6 for the most common Current Procedural Terminology [CPT] codes associated with these visits). We chose the 2 follow-up windows (7 and 30 days) because they are consistent with the HEDIS quality measure of follow-up after hospitalization for mental illness.7 

Because the therapeutic effect of follow-up care has a potential dose-response relationship, we also measured the number of subsequent follow-up outpatient mental health visits between 8 days and 6 months after the index discharge as an indicator of dose of care over time.

Our primary outcome was time to subsequent psychiatric crisis up to 6 months after discharge. We defined psychiatric crisis as an ED visit or a hospitalization with an assigned major diagnostic category of mental diseases and disorders; alcohol or drug use or induced mental disorders; or injuries, poison, and toxic effect of drugs. As secondary outcomes, we assessed 2 binary outcomes: whether a crisis occurred within 30 days and whether a crisis occurred within 6 months.

We included in our analyses several covariates to adjust for potential confounding and as potential explanatory variables. Timely follow-up visits may be driven by clinical severity. To account for this, we included 2 covariates to approximate severity of illness: psychiatric comorbidities and length of stay (LOS) of the index hospitalization. For psychiatric comorbidities, we assessed the number of psychiatric comorbidities for each patient, defined by using ICD-9-CM and ICD-10-CM codes as noted above.10,11  After examining the data, we defined LOS using the following categories: <4, 4 to 6, and >6 days.

Other covariates included age (categorized as 6–11 or 12–17 years), sex, race and/or ethnicity, insurance type (fee for service or managed care), having a comorbid nonpsychiatric chronic disease, and hospital type (psychiatric hospital, psychiatric unit within a general hospital, or general medical hospital unit). To adjust for nonpsychiatric comorbidities, we used the Pediatric Medical Complexity Algorithm (PMCA), which categorizes patients into 3 categories using ICD-9-CM and ICD-10-CM codes: no chronic conditions, noncomplex chronic conditions, or complex chronic conditions.13,14 

We used bivariate analyses to determine characteristics associated with follow-up within 7 or 30 days and characteristics associated with having a subsequent psychiatric crisis. For statistical significance testing, we used Kruskal-Wallis and χ2 analyses.

We used a survival analysis, with Kaplan-Meier curves and Cox proportional hazards multivariable models, to assess relationships between follow-up visits at the 2 time periods and subsequent psychiatric crisis. This approach allows the 7- and 30-day visits to be time varying. For example, if an enrollee had a crisis on day 4 and a follow-up visit on day 6, they would be considered as not having had a visit on days 0 to 4. Their contributions to the Cox model would stop at day 4, and their follow-up would not be considered for the analysis. This approach allowed us to include psychiatric crises that occurred at any time point after discharge.

We used multivariable analyses with a stepwise approach to assess whether the relationship between follow-up and subsequent crisis was confounded by comorbidities or severity of disease. We first tested the association by only including covariates that do not indicate comorbidity or severity of illness in the model: mental health diagnosis for index admission, age, sex, race and/or ethnicity, and insurance plan type (fee for service versus capitated plan). Then we added into the model covariates for comorbidity and severity: PMCA category, LOS category, number of comorbid mental health conditions, and suicide attempt or ideation.

For sensitivity analyses, we tested an alternative predictor definition, defining follow-up using 2 variables, active medication use and active outpatient mental health visits, included in a time-dependent model. To define these time-dependent variables, we identified the first medication fill for an antidepressant or antipsychotic and considered the time-dependent medication variable active if another prescription was filled within 30 days of the end of the prescribed fill. The time-dependent variable for outpatient visits was considered active if there was a visit within 30 days of the previous visit. An additional sensitivity analysis was used to assess for potential confounding effects of autism (n = 970; 4.2%), intellectual disability (n = 553; 2.4%), and substance abuse (n = 3678; 16.1%) on the relationship between outpatient follow-up and subsequent crises by including the presence of those diagnoses as covariates in the model.

A total of 22 844 children aged 6 to 17 years old were admitted with one of the conditions of interest as a primary diagnosis. Three-quarters of these children had depressive disorders, whereas 20% had bipolar or related disorders, and few had either schizophrenia (3.6%) or anxiety (1.6%). Most patients were 12 to 17 years of age. Racial and ethnic distribution was similar across 3 of the 4 categories (Table 1). Non-Hispanic or non-Latino white patients made up 63% of admissions and non-Hispanic or non-Latino black patients made up 25%.

TABLE 1

Characteristics of Pediatric Admissions for Primary Diagnoses of Depression, Bipolar Disorder, Psychosis, or Anxiety (N = 22 844)

Depression (n = 17 199)Bipolar Disorder (n = 4424)Psychosis (n = 835)Anxiety (n = 386)
Age, y, median (IQR) 15 (13–16) 15 (13–16) 15 (12–16) 14 (12–16) 
Age category, y, n (%)     
 6–11 1882 (10.9) 784 (17.7) 151 (18.1) 95 (24.6) 
 12–17 15 317 (89.1) 3640 (82.3) 684 (81.9) 291 (75.4) 
Male sex, n (%) 5694 (33.1) 2341 (52.9) 503 (60.2) 169 (43.8) 
Race and/or ethnicity, n (%)     
 Non-Hispanic or non-Latino white 11 208 (65.2) 2701 (61.1) 348 (41.7) 261 (67.6) 
 Non-Hispanic or non-Latino black 4060 (23.6) 1239 (28.0) 358 (42.9) 71 (18.4) 
 Hispanic or Latino 639 (3.7) 83 (1.9) 35 (4.2) 16 (4.1) 
 Other 1292 (7.5) 401 (9.1) 94 (11.3) 38 (9.8) 
Capitated plan, n (%) 12 962 (75.4) 3149 (71.2) 629 (75.3) 277 (71.8) 
PMCAa (non-MH), n (%)     
 Nonchronic 10 636 (61.8) 2475 (55.9) 470 (56.3) 229 (59.3) 
 Non–complex chronic 4054 (23.6) 1082 (24.5) 210 (25.1) 75 (19.4) 
 Complex chronic 2509 (14.6) 867 (19.6) 155 (18.6) 82 (21.2) 
LOS, d, median (IQR) 5 (4–7) 6 (4–8) 6 (4–9) 5 (3–7) 
LOS group, d, n (%)     
 <4 3616 (21.0) 594 (13.4) 117 (14.0) 135 (35.0) 
 4–6 8165 (47.5) 1831 (41.4) 314 (37.6) 154 (39.9) 
 >6 5418 (31.5) 1999 (45.2) 404 (48.4) 97 (25.1) 
Hospital type, n (%)     
 Psychiatric hospital 7443 (43.3) 1822 (41.2) 315 (37.7) 121 (31.3) 
 Psychiatric unit within a general hospital 9456 (55.0) 2542 (57.5) 488 (58.4) 210 (54.4) 
 General medical or surgical unit 300 (1.7) 60 (1.4) 32 (3.8) 55 (14.2) 
Comorbid MH conditions, n (%)     
 0 1137 (6.6) 266 (6.0) 85 (10.2) 50 (13.0) 
 1 3815 (22.2) 544 (12.3) 174 (20.8) 52 (13.5) 
 2 5012 (29.1) 935 (21.1) 184 (22.0) 92 (23.8) 
 3 3834 (22.3) 1096 (24.8) 187 (22.4) 79 (20.5) 
 4+ 3401 (19.8) 1583 (35.8) 205 (24.6) 113 (29.3) 
Depression (n = 17 199)Bipolar Disorder (n = 4424)Psychosis (n = 835)Anxiety (n = 386)
Age, y, median (IQR) 15 (13–16) 15 (13–16) 15 (12–16) 14 (12–16) 
Age category, y, n (%)     
 6–11 1882 (10.9) 784 (17.7) 151 (18.1) 95 (24.6) 
 12–17 15 317 (89.1) 3640 (82.3) 684 (81.9) 291 (75.4) 
Male sex, n (%) 5694 (33.1) 2341 (52.9) 503 (60.2) 169 (43.8) 
Race and/or ethnicity, n (%)     
 Non-Hispanic or non-Latino white 11 208 (65.2) 2701 (61.1) 348 (41.7) 261 (67.6) 
 Non-Hispanic or non-Latino black 4060 (23.6) 1239 (28.0) 358 (42.9) 71 (18.4) 
 Hispanic or Latino 639 (3.7) 83 (1.9) 35 (4.2) 16 (4.1) 
 Other 1292 (7.5) 401 (9.1) 94 (11.3) 38 (9.8) 
Capitated plan, n (%) 12 962 (75.4) 3149 (71.2) 629 (75.3) 277 (71.8) 
PMCAa (non-MH), n (%)     
 Nonchronic 10 636 (61.8) 2475 (55.9) 470 (56.3) 229 (59.3) 
 Non–complex chronic 4054 (23.6) 1082 (24.5) 210 (25.1) 75 (19.4) 
 Complex chronic 2509 (14.6) 867 (19.6) 155 (18.6) 82 (21.2) 
LOS, d, median (IQR) 5 (4–7) 6 (4–8) 6 (4–9) 5 (3–7) 
LOS group, d, n (%)     
 <4 3616 (21.0) 594 (13.4) 117 (14.0) 135 (35.0) 
 4–6 8165 (47.5) 1831 (41.4) 314 (37.6) 154 (39.9) 
 >6 5418 (31.5) 1999 (45.2) 404 (48.4) 97 (25.1) 
Hospital type, n (%)     
 Psychiatric hospital 7443 (43.3) 1822 (41.2) 315 (37.7) 121 (31.3) 
 Psychiatric unit within a general hospital 9456 (55.0) 2542 (57.5) 488 (58.4) 210 (54.4) 
 General medical or surgical unit 300 (1.7) 60 (1.4) 32 (3.8) 55 (14.2) 
Comorbid MH conditions, n (%)     
 0 1137 (6.6) 266 (6.0) 85 (10.2) 50 (13.0) 
 1 3815 (22.2) 544 (12.3) 174 (20.8) 52 (13.5) 
 2 5012 (29.1) 935 (21.1) 184 (22.0) 92 (23.8) 
 3 3834 (22.3) 1096 (24.8) 187 (22.4) 79 (20.5) 
 4+ 3401 (19.8) 1583 (35.8) 205 (24.6) 113 (29.3) 

MH, mental health.

a

The PMCA,13,14  which uses ICD-9-CM and ICD-10-CM definitions of chronic disease to create a categorical variable, was used to determine chronic disease.

Sixty-two percent of patients had follow-up within 7 days, and 82.3% had follow-up within 30 days. Follow-up rates were similar across most patient characteristics (Table 2), with <6-percentage-point differences in follow-up between categories (although most met statistical significance because of large sample size). However, those discharged from general medical or surgical units were substantially less likely to have follow-up at both time points (47.2% at 7 days and 73.2% at 30 days) compared with patients discharged from psychiatric units within general hospitals (61.1% at 7 days and 81.3% at 30 days) or freestanding psychiatric hospitals (63.8% at 7 days and 84.1% at 30 days; Table 2). Fewer non-Hispanic or non-Latino black patients compared with white patients had follow-up at both 7 and 30 days after discharge (57.9% vs 63.2% at 7 days and 77.4% vs 84.1% at 30 days). Those with ≥4 mental health comorbidities had a higher follow-up percentage compared with those without these comorbidities (66.4% vs 59.0% at 7 days and 85.6% vs 78.4% at 30 days; Table 2).

TABLE 2

Frequency of Follow-up Visits Within 7 and 30 Days by Patient and Hospital Characteristics

CharacteristicsVisit Within 7 d, %Visit Within 30 d, %Visit Within 7 d, AdjustedaVisit Within 30 d, Adjusteda
Overall 62.0 82.3 — — 
Condition     
 Depression 60.7 81.5 59.7 80.7 
 Bipolar disorder 66.1 85.1 62.9 82.7 
 Psychosis 64.8 85.3 62.5 83.4 
 Anxiety 65.0 83.4 62.7 82.5 
Age category, y     
 6–11 66.7 85.4 63.9 83.5 
 12–17 61.3 81.9 60.0 81.1 
Sex     
 Male 62.5 82.8 61.4 82.0 
 Female 61.7 82.1 62.6 82.8 
Race and/or ethnicity     
 Non-Hispanic or non-Latino white 63.2 84.1 62.3 83.7 
 Non-Hispanic or non-Latino black 57.9 77.4 57.2 77.2 
 Hispanic or Latino 65.7 84.1 65.9 85.1 
 Other 63.4 83.2 62.6 83.0 
Plan type     
 Fee for service 59.2 79.1 59.3 79.3 
 Capitated 62.9 83.5 64.7 85.4 
PMCAb (non-MH)     
 Nonchronic 60.1 80.7 58.9 79.6 
 Non–complex chronic 63.2 83.9 61.6 82.4 
 Complex chronic 67.5 86.5 65.3 84.7 
LOS group, d     
 <4 60.6 79.9 61.2 80.2 
 4–6 61.4 82.4 61.4 82.4 
 >6 63.5 83.7 63.4 84.3 
Hospital type     
 Psychiatric hospital 63.8 84.1 68.4 86.0 
 Psychiatric unit within a general hospital 61.1 81.3 66.2 84.1 
 General medical or surgical unit 47.2 73.2 51 76.0 
Comorbid MH conditions     
 0 59.0 78.4 59.4 79.0 
 1 59.6 79.8 60.3 80.6 
 2 60.3 81.4 61.1 82.0 
 3 62.4 83.6 62.8 84.0 
 4+ 66.4 85.6 66.1 85.5 
CharacteristicsVisit Within 7 d, %Visit Within 30 d, %Visit Within 7 d, AdjustedaVisit Within 30 d, Adjusteda
Overall 62.0 82.3 — — 
Condition     
 Depression 60.7 81.5 59.7 80.7 
 Bipolar disorder 66.1 85.1 62.9 82.7 
 Psychosis 64.8 85.3 62.5 83.4 
 Anxiety 65.0 83.4 62.7 82.5 
Age category, y     
 6–11 66.7 85.4 63.9 83.5 
 12–17 61.3 81.9 60.0 81.1 
Sex     
 Male 62.5 82.8 61.4 82.0 
 Female 61.7 82.1 62.6 82.8 
Race and/or ethnicity     
 Non-Hispanic or non-Latino white 63.2 84.1 62.3 83.7 
 Non-Hispanic or non-Latino black 57.9 77.4 57.2 77.2 
 Hispanic or Latino 65.7 84.1 65.9 85.1 
 Other 63.4 83.2 62.6 83.0 
Plan type     
 Fee for service 59.2 79.1 59.3 79.3 
 Capitated 62.9 83.5 64.7 85.4 
PMCAb (non-MH)     
 Nonchronic 60.1 80.7 58.9 79.6 
 Non–complex chronic 63.2 83.9 61.6 82.4 
 Complex chronic 67.5 86.5 65.3 84.7 
LOS group, d     
 <4 60.6 79.9 61.2 80.2 
 4–6 61.4 82.4 61.4 82.4 
 >6 63.5 83.7 63.4 84.3 
Hospital type     
 Psychiatric hospital 63.8 84.1 68.4 86.0 
 Psychiatric unit within a general hospital 61.1 81.3 66.2 84.1 
 General medical or surgical unit 47.2 73.2 51 76.0 
Comorbid MH conditions     
 0 59.0 78.4 59.4 79.0 
 1 59.6 79.8 60.3 80.6 
 2 60.3 81.4 61.1 82.0 
 3 62.4 83.6 62.8 84.0 
 4+ 66.4 85.6 66.1 85.5 

MH, mental health; —, not applicable.

a

Adjusted for all variables shown. All comparisons were statistically significantly different except for comparing unadjusted rates of follow-up visits for boys and girls (P = .19 for 7-d visits and P = .21 for 30-d visits).

b

The PMCA,13,14  which uses ICD-9-CM and ICD-10-CM definitions of chronic disease to create a categorical variable, was used to determine chronic disease.

In assessing differences in the dose of follow-up, or the number of follow-up visits within 6 months, we found that patients with follow-up by 7 days had more subsequent follow-up visits (median: 17 visits within 6 months of discharge; interquartile range [IQR] 8–35), whereas those without follow-up by 7 days had fewer subsequent visits (median: 8 visits; IQR 3–17).

Patients frequently had psychiatric crises (mental health–related ED visits or hospitalizations) after discharge, with 22.4% with crises within 30 days and 54.8% with crises within 6 months. Primary diagnosis, race and/or ethnicity, medical and mental health comorbidities, and hospital unit type were associated with subsequent crises (Table 3). Of those with a crisis, 63% had ED visits and 37% were hospitalized.

TABLE 3

Patients With a Psychiatric Crisis Within 30 Days and Within 6 Months

Patient CharacteristicsCrisis Within 30 d, %Crisis Within 6 mo, %
Overall 22.4 54.8 
Condition   
 Depression 21.0 53.0 
 Bipolar disorder 26.9 61.3 
 Psychosis 27.2 57.8 
 Anxiety 22.0 53.4 
Age category, y   
 6–11 23.6 52.5 
 12–17 22.2 55.1 
Sex   
 Male 22.3 52.3 
 Female 22.5 56.3 
Race and/or ethnicity   
 Non-Hispanic or non-Latino white 22.0 54.5 
 Non-Hispanic or non-Latino black 23.5 55.6 
 Hispanic or Latino 18.6 50.8 
 Other 24.2 56.4 
Plan type   
 Fee for service 20.8 54.1 
 Capitated 23.0 55.0 
PMCAa (non-MH)   
 Nonchronic 20.5 51.2 
 Non–complex chronic 24.3 58.1 
 Complex chronic 27.0 63.4 
LOS group, d   
 <4 25.5 55.2 
 4–6 19.1 52.4 
 >6 25.1 57.7 
Hospital type   
 Psychiatric hospital 20.8 53.7 
 Psychiatric unit within a general hospital 23.6 55.6 
 General medical or surgical unit 24.8 56.2 
Comorbid MH conditions   
 0 19.8 50.6 
 1 19.5 50.4 
 2 21.8 52.7 
 3 23.2 56.7 
 4+ 25.6 60.3 
Patient CharacteristicsCrisis Within 30 d, %Crisis Within 6 mo, %
Overall 22.4 54.8 
Condition   
 Depression 21.0 53.0 
 Bipolar disorder 26.9 61.3 
 Psychosis 27.2 57.8 
 Anxiety 22.0 53.4 
Age category, y   
 6–11 23.6 52.5 
 12–17 22.2 55.1 
Sex   
 Male 22.3 52.3 
 Female 22.5 56.3 
Race and/or ethnicity   
 Non-Hispanic or non-Latino white 22.0 54.5 
 Non-Hispanic or non-Latino black 23.5 55.6 
 Hispanic or Latino 18.6 50.8 
 Other 24.2 56.4 
Plan type   
 Fee for service 20.8 54.1 
 Capitated 23.0 55.0 
PMCAa (non-MH)   
 Nonchronic 20.5 51.2 
 Non–complex chronic 24.3 58.1 
 Complex chronic 27.0 63.4 
LOS group, d   
 <4 25.5 55.2 
 4–6 19.1 52.4 
 >6 25.1 57.7 
Hospital type   
 Psychiatric hospital 20.8 53.7 
 Psychiatric unit within a general hospital 23.6 55.6 
 General medical or surgical unit 24.8 56.2 
Comorbid MH conditions   
 0 19.8 50.6 
 1 19.5 50.4 
 2 21.8 52.7 
 3 23.2 56.7 
 4+ 25.6 60.3 

All comparisons were statistically significantly different except for comparing age categories for a crisis within 30 d (P = .19), boys and girls for a crisis within 30 d (P = .22), and plan types for a crisis within 6 mo (P = .36). MH, mental health.

a

The PMCA,13,14  which uses ICD-9-CM and ICD-10-CM definitions of chronic disease to create a categorical variable, was used to determine chronic disease.

Having a follow-up visit within 7 or 30 days was associated with an increased hazard ratio (HR) for psychiatric crisis (HR adjusted for base characteristics: 7 days: 1.20 [95% confidence interval (CI) 1.16–1.25]; 30 days: 1.31 [95% CI 1.25–1.37]; Kaplan-Meier curves in Figs 1 and 2). This relationship was slightly attenuated when severity-related variables (LOS, comorbidities related and not related to mental health) were included (HR adjusted for base characteristics and severity indicators: 7 days: 1.18 [95% CI 1.13–1.22]; 30 days: 1.27 [95% CI 1.21–1.33]; Table 4), but the association remained statistically significant in the same direction, with CIs overlapping results from the baseline model.

FIGURE 1

Kaplan-Meier survival curves revealing the relationship between having a mental health follow-up visit within 7 days after hospitalization for depression, bipolar disorder, psychosis, or anxiety and a subsequent psychiatric crisis (hospitalization or ED visit) within 6 months from discharge.

FIGURE 1

Kaplan-Meier survival curves revealing the relationship between having a mental health follow-up visit within 7 days after hospitalization for depression, bipolar disorder, psychosis, or anxiety and a subsequent psychiatric crisis (hospitalization or ED visit) within 6 months from discharge.

Close modal
FIGURE 2

Kaplan-Meier survival curves revealing the relationship between having a mental health follow-up visit within 30 days after hospitalization for depression, bipolar disorder, psychosis, or anxiety and a subsequent psychiatric crisis (hospitalization or ED visit) within 6 months from discharge.

FIGURE 2

Kaplan-Meier survival curves revealing the relationship between having a mental health follow-up visit within 30 days after hospitalization for depression, bipolar disorder, psychosis, or anxiety and a subsequent psychiatric crisis (hospitalization or ED visit) within 6 months from discharge.

Close modal
TABLE 4

Relationships Between Follow-up Visits at 7 and 30 Days and Subsequent Psychiatric Crises, Adjusting for Base Characteristics and Then Including Severity Measures

Adjusted HR for Time to Crisis (95% CI)
Follow-up Visit Within 7 dFollow-up Visit Within 30 d
Base Adjusted ModelBase + Severity MeasuresBase Adjusted ModelBase + Severity Measures
Follow-up visit 1.20 (1.16–1.25) 1.18 (1.13–1.22) 1.31 (1.25–1.37) 1.27 (1.21–1.33) 
Severity measures     
 PMCAa (non-MH)     
  Nonchronic — Reference — Reference 
  Non–complex chronic — 1.20 (1.14–1.25) — 1.19 (1.14–1.25) 
  Complex chronic — 1.34 (1.27–1.4) — 1.34 (1.27–1.4) 
 LOS group, d     
  <4 — Reference — Reference 
  4–6 — 1.05 (0.99–1.10) — 1.04 (0.99–1.09) 
  >6 — 1.23 (1.16–1.30) — 1.22 (1.16–1.29) 
 Comorbid MH conditions     
  0 — Reference — Reference 
  1 — 1.00 (0.92–1.09) — 1.00 (0.92–1.08) 
  2 — 1.07 (0.98–1.15) — 1.06 (0.98–1.15) 
  3 — 1.18 (1.09–1.28) — 1.18 (1.08–1.27) 
  4+ — 1.27 (1.17–1.38) — 1.26 (1.17–1.37) 
Adjusted HR for Time to Crisis (95% CI)
Follow-up Visit Within 7 dFollow-up Visit Within 30 d
Base Adjusted ModelBase + Severity MeasuresBase Adjusted ModelBase + Severity Measures
Follow-up visit 1.20 (1.16–1.25) 1.18 (1.13–1.22) 1.31 (1.25–1.37) 1.27 (1.21–1.33) 
Severity measures     
 PMCAa (non-MH)     
  Nonchronic — Reference — Reference 
  Non–complex chronic — 1.20 (1.14–1.25) — 1.19 (1.14–1.25) 
  Complex chronic — 1.34 (1.27–1.4) — 1.34 (1.27–1.4) 
 LOS group, d     
  <4 — Reference — Reference 
  4–6 — 1.05 (0.99–1.10) — 1.04 (0.99–1.09) 
  >6 — 1.23 (1.16–1.30) — 1.22 (1.16–1.29) 
 Comorbid MH conditions     
  0 — Reference — Reference 
  1 — 1.00 (0.92–1.09) — 1.00 (0.92–1.08) 
  2 — 1.07 (0.98–1.15) — 1.06 (0.98–1.15) 
  3 — 1.18 (1.09–1.28) — 1.18 (1.08–1.27) 
  4+ — 1.27 (1.17–1.38) — 1.26 (1.17–1.37) 

Base models were adjusted for condition, age category, sex, chronic condition indicator, and insurance type. MH, mental health; —, not applicable.

a

The PMCA,13,14  which uses ICD-9-CM and ICD-10-CM definitions of chronic disease to create a categorical variable, was used to determine chronic disease.

There was no difference in the proportion of patients with a subsequent hospitalization between groups with and without follow-up within 7 days of discharge (34.7% vs 35.7%; P > .05), and there was a slight increase in the proportion of patients without follow-up within 30 days (34.1% vs 37.3%; P = .01).

The findings were similar in magnitude and direction in a time-dependent model used to assess active medication use (HR: 1.25; 95% CI 1.20–1.31) and active outpatient mental health visits (HR: 1.34; 95% CI 1.29–1.40). Models that included autism, intellectual disabilities, and substance abuse as covariates also had similar findings in direction and magnitude (data not shown).

With this study, we are the first to examine variations in 7- and 30-day follow-up among a sample of children and adolescents with mental health admissions across psychiatric and general hospitals in the United States and the associated risk of subsequent ED revisit or rehospitalization. In our analysis of Medicaid data from 11 states, we found that a majority of children aged 6 to 17 years had any mental health outpatient follow-up, with higher rates than previously found for this national HEDIS quality measure.7,8  Just under two-thirds had follow-up within 7 days, and >80% had follow-up within 30 days. Lower rates of follow-up were seen for certain groups, including non-Hispanic or non-Latino black children and those children discharged from a general medical or surgical unit for mental health illness as opposed to a psychiatric unit or hospital. Contrary to our hypothesis, we found that having a follow-up within 7 or 30 days of discharge was associated with an increased risk of a subsequent hospitalization or ED visit for a mental health illness, even after controlling for potential confounders.

Our finding that follow-up rates for black children were ∼6 percentage points less than follow-up rates for white children supports findings from a previous study that revealed a relationship between black race and lower follow-up rates for children admitted for mood disorders (depression and bipolar disorders) in 2009–2010.15  Fee-for-service health plan coverage and discharge from a medical or surgical unit were also associated with lower follow-up rates at 7 and 30 days. This suggests that to improve performance on the national measure from the Child Core Set, health plans and providers may need to increase outreach and access to services for these populations.

Our a priori hypothesis was that having a 7- or 30-day outpatient mental health follow-up after a mental health hospitalization would be associated with lower risk for subsequent hospitalizations or ED visits for mental illness. However, we found the opposite to be true. Previous studies examining the risk of readmission from a mental health hospitalization have produced mixed results. A study by Carlisle et al16  revealed that outpatient follow-up with a primary care provider or psychiatrist within a month after a psychiatric hospitalization for 15- to 19-year-olds in Ontario, Canada, increased the risk of readmission and ED visits combined and readmission alone but was not significantly associated with ED visits. Two other smaller studies also revealed that outpatient follow-up after hospitalization for a mental health disorder was associated with increased risk for readmission, including a study of 522 adolescents with psychiatric hospitalizations in 3 Maryland hospitals17  and a study by Gearing et al18  of 87 patients <18 years old with a first episode of psychosis. A study of 500 children and adolescents (ages 3–21) who were wards of the court in Illinois revealed that posthospitalization services (including case management) were associated with lower rates of rehospitalization.19  A study of Children’s Health Insurance Program recipients in Alabama revealed low rates of follow-up at 7 and 30 days (33% and 53%, respectively) and no association with subsequent ED use or rehospitalization within 60, 90, or 120 days in adjusted models.20 

Although we do not believe that there is causation in this relationship, we have several hypotheses to explain our findings. First, it is possible that we did not sufficiently account for severity of illness or psychosocial complexity in our models. Discharging providers may be more likely to ensure outpatient follow-up for children with more-severe illness (either medical or psychiatric) or greater psychosocial complexity, thereby driving the relationship between follow-up and crisis. We did find that children with ≥4 comorbid mental health conditions were more likely to have a 7- and 30-day follow-up than those with no comorbid mental health conditions, but this variable was included in our multivariable models.

There is also the possibility that having a 7- or 30-day follow-up outpatient mental health visit leads to increased recognition of a crisis and thus increases the risk of ED use or rehospitalization. A recent study by Joyce et al21  revealed that a lack of treatment alliance with a mental health provider was associated with decreased psychiatric rehospitalization after discharge; the authors suggest as an explanation that outpatient care leads to increased recognition of emotional dysregulation and, hence, to appropriate referrals to ED or inpatient care.

Finally, there is also the possibility that psychiatric crises (leading to an ED visit or admission) after the index mental health illness hospital admission are not readily prevented by better access to outpatient follow-up care and that follow-up care is beneficial to these patients who are severely ill but cannot reduce the severity of their disease. This is supported by the finding that almost one-quarter of patients have subsequent crises within 30 days and that more than half have subsequent crises within 6 months. Our finding of similar rates of rehospitalization for children with and without a 7-day follow-up after the index admission indicates that the more-severe crises are difficult to prevent, regardless of outpatient follow-up; however, we did find that follow-up within 30 days is associated with a slightly lower percentage of subsequent hospitalizations (34.1% vs 37.3%, respectively; P = .01), indicating some protective effect for these more-severe crises for patients seen within 30 days.

This study has several limitations to consider. Although administrative claims analyses can capture the number of outpatient follow-up visits,22  they cannot assess the quality of the visits; it is possible that high-quality visits might reveal different results. ICD-9-CM and ICD-10-CM codes may be limited in their sensitivity for correctly identifying all patients with mental health diagnoses. This may have led to omission of patients from the population (likely those with less-severe illness) or omission of outpatient follow-up visits, although the latter may be curtailed by the HEDIS quality measure incentive to appropriately code for these visits. In addition, we do not have access to state identifiers, so we cannot assess geographical variations. Although the included states represent all regions nationally, our findings may not be generalizable to all states. Finally, we may not have been able to fully adjust for severity of illness or psychosocial complexity. Clinical data derived from electronic medical records may provide more extensive risk-adjustment variables. A future randomized trial to assess the effectiveness of an intervention to improve follow-up after hospitalization could more conclusively address this limitation.

In this study of Medicaid patients across 11 states, we found that more than one-third of children with a mental health hospitalization did not receive care that aligns with the national quality measure of follow-up within 7 and 30 days post discharge. Adherence to the quality measure varied by patient and hospital characteristics. In addition, adherence was associated with an increased risk of a subsequent mental health crisis, even after controlling for severity of illness. Implications of these findings are that targeted interventions are needed to address lower rates of follow-up for specific populations and that interventions for more-intensive symptom monitoring and outpatient treatment of crises may be needed to optimize outpatient care after hospitalization.

Drs Bardach and Coker contributed to study conceptualization and design, interpreted the results, drafted the initial manuscript, and revised the manuscript; Dr Doupnik assisted in study design, interpretation of results, and review of later versions of the manuscript; Mr Rodean assisted in statistical approach and analyses, interpretation of the results, and critical review of the manuscript; Drs Zima and Gay, Ms Nash, and Dr Tanguturi contributed to study design, interpretation of the results, and critical review of the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

     
  • CI

    confidence interval

  •  
  • CPT

    Current Procedural Terminology

  •  
  • ED

    emergency department

  •  
  • HEDIS

    Healthcare Effectiveness Data and Information Set

  •  
  • HR

    hazard ratio

  •  
  • ICD-9-CM

    International Classification of Diseases, Ninth Revision, Clinical Modification

  •  
  • ICD-10-CM

    International Classification of Diseases, 10th Revision, Clinical Modification

  •  
  • IQR

    interquartile range

  •  
  • LOS

    length of stay

  •  
  • PMCA

    Pediatric Medical Complexity Algorithm

1
Williams
NJ
,
Scott
L
,
Aarons
GA
.
Prevalence of serious emotional disturbance among US children: a meta-analysis
.
Psychiatr Serv
.
2018
;
69
(
1
):
32
40
2
Briggs-Gowan
MJ
,
Horwitz
SM
,
Schwab-Stone
ME
,
Leventhal
JM
,
Leaf
PJ
.
Mental health in pediatric settings: distribution of disorders and factors related to service use
.
J Am Acad Child Adolesc Psychiatry
.
2000
;
39
(
7
):
841
849
3
Kataoka
SH
,
Zhang
L
,
Wells
KB
.
Unmet need for mental health care among U.S. children: variation by ethnicity and insurance status
.
Am J Psychiatry
.
2002
;
159
(
9
):
1548
1555
4
US Public Health Service
.
Mental Health: A Report of the Surgeon General
.
Rockville, MD
:
US Department of Health and Human Services
;
1999
5
Leaf
PJ
,
Alegria
M
,
Cohen
P
, et al
.
Mental health service use in the community and schools: results from the four-community MECA study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study
.
J Am Acad Child Adolesc Psychiatry
.
1996
;
35
(
7
):
889
897
6
Gardner
W
,
Pajer
K
,
Cloutier
P
, et al
.
Health outcomes associated with emergency department visits by adolescents for self-harm: a propensity-matched cohort study
.
CMAJ
.
2019
;
191
(
44
):
E1207
E1216
7
National Committee for Quality Assurance
.
Follow-up after hospitalization for mental illness (FUH). 2019
.
8
Zima
BT
,
Edgcomb
JB
,
Shugarman
SA
.
National child mental health quality measures: adherence rates and extent of evidence for clinical validity
.
Curr Psychiatry Rep
.
2019
;
21
(
1
):
6
9
Bardach
NS
,
Coker
TR
,
Zima
BT
, et al
.
Common and costly hospitalizations for pediatric mental health disorders
.
Pediatrics
.
2014
;
133
(
4
):
602
609
10
Zima
BT
,
Rodean
J
,
Hall
M
,
Bardach
NS
,
Coker
TR
,
Berry
JG
.
Psychiatric disorders and trends in resource use in pediatric hospitals
.
Pediatrics
.
2016
;
138
(
5
):
e20160909
11
Zima
BT
,
Gay
JC
,
Rodean
J
, et al
.
Classification system for ICD-9 and ICD-10 pediatric mental health disorders
.
JAMA Pediatr
.
In press
12
Frayne
SM
,
Miller
DR
,
Sharkansky
EJ
, et al
.
Using administrative data to identify mental illness: what approach is best?
Am J Med Qual
.
2010
;
25
(
1
):
42
50
13
Simon
TD
,
Haaland
W
,
Hawley
K
,
Lambka
K
,
Mangione-Smith
R
.
Development and validation of the Pediatric Medical Complexity Algorithm (PMCA) version 3.0
.
Acad Pediatr
.
2018
;
18
(
5
):
577
580
14
Simon
TD
,
Cawthon
ML
,
Stanford
S
, et al
;
Center of Excellence on Quality of Care Measures for Children with Complex Needs (COE4CCN) Medical Complexity Working Group
.
Pediatric medical complexity algorithm: a new method to stratify children by medical complexity
.
Pediatrics
.
2014
;
133
(
6
).
15
Fontanella
CA
,
Hiance-Steelesmith
DL
,
Bridge
JA
, et al
.
Factors associated with timely follow-up care after psychiatric hospitalization for youths with mood disorders
.
Psychiatr Serv
.
2016
;
67
(
3
):
324
331
16
Carlisle
CE
,
Mamdani
M
,
Schachar
R
,
To
T
.
Aftercare, emergency department visits, and readmission in adolescents
.
J Am Acad Child Adolesc Psychiatry
.
2012
;
51
(
3
):
283
293.e4
17
Fontanella
CA
.
The influence of clinical, treatment, and healthcare system characteristics on psychiatric readmission of adolescents
.
Am J Orthopsychiatry
.
2008
;
78
(
2
):
187
198
18
Gearing
RE
,
Mian
I
,
Sholonsky
A
, et al
.
Developing a risk-model of time to first-relapse for children and adolescents with a psychotic disorder
.
J Nerv Ment Dis
.
2009
;
197
(
1
):
6
14
19
Romansky
JB
,
Lyons
JS
,
Lehner
RK
,
West
CM
.
Factors related to psychiatric hospital readmission among children and adolescents in state custody
.
Psychiatr Serv
.
2003
;
54
(
3
):
356
362
20
Blackburn
J
,
Sharma
P
,
Corvey
K
, et al
.
Assessing the quality measure for follow-up care after children’s psychiatric hospitalizations
.
Hosp Pediatr
.
2019
;
9
(
11
):
834
843
21
Joyce
VW
,
King
CD
,
Nash
CC
,
Lebois
LAM
,
Ressler
KJ
,
Buonopane
RJ
.
Predicting psychiatric rehospitalization in adolescents
.
Adm Policy Ment Health
.
2019
;
46
(
6
):
807
820
22
Zima
BT
,
Bussing
R
,
Tang
L
, et al
.
Quality of care for childhood attention-deficit/hyperactivity disorder in a managed care Medicaid program
.
J Am Acad Child Adolesc Psychiatry
.
2010
;
49
(
12
):
1225
1237, 1237.e1–1237.e11

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data