Although there have been a number of clinical trials evaluating treatments for adolescents with major depressive disorder (MDD), the generalizability of those trials to samples of depressed adolescents who present for routine clinical care is unknown. Examining the generalizability of clinical trials of pharmacological and psychotherapy interventions for adolescent depression can help administrators and frontline practitioners determine the relevance of these studies for their patients and may also guide eligibility criteria for future clinical trials in this clinical population.
Data on nationally representative adolescents were derived from the National Comorbidity Survey: Adolescent Supplement. To assess the generalizability of adolescent clinical trials for MDD, we applied a standard set of eligibility criteria representative of clinical trials to all adolescents in the National Comorbidity Survey: Adolescent Supplement with a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition diagnosis of MDD (N = 592).
From the overall MDD sample, 61.9% would have been excluded from a typical pharmacological trial, whereas 42.2% would have been excluded from a psychotherapy trial. Among those who sought treatment (n = 412), the corresponding exclusion rates were 72.7% for a pharmacological trial and 52.2% for a psychotherapy trial. The criterion leading to the largest number of exclusions was “significant risk of suicide” in both pharmacological and psychotherapy trials.
Pharmacological and, to a lesser extent, psychotherapy clinical trials likely exclude most adolescents with MDD. Careful consideration should be given to balancing eligibility criteria and internal validity with applicability in routine clinical care while ensuring patient safety.
Over the last 2 decades, in several studies, researchers have evaluated treatments to guide clinical management of adolescents with depression. However, the generalizability of these clinical trial samples to the broader population of adolescents with depression is unknown.
We estimated the generalizability of clinical trials of pharmacological and psychotherapy interventions for adolescent depression. An understanding of the generalizability of these interventions is important to help administrators and front-line practitioners determine the relevance of these studies for their patients.
Major concerns in any randomized controlled trial (RCT) include ensuring the validity of the data being collected as well as the protection of the rights and safety of study participants. In response to these concerns, researchers conducting RCTs apply inclusion and exclusion criteria to ensure that studies meet ethical standards, that participation in them is safe, and that participants have the clinical and sociodemographic characteristics needed to answer the research question. In particular, eligibility criteria are used to ensure participants’ safety, prevent exploitation of vulnerable persons, reduce study costs and attrition rate, allow for adequate evaluation of the effect of a treatment on a specific disorder and increase the likelihood of generating reliable and reproducible results, and comply with guidelines of regulatory agencies.1,–7 However, authors of previous research1,–7 suggest that the use of restrictive eligibility criteria may result in research samples that often do not adequately represent the range of patients seen in routine clinical care. As concerns have emerged regarding the use of stringent exclusion criteria in clinical trials, there has been growing interest in quantifying the generalizability of clinical trial results to the broader target population8 and in more optimally balancing internal validity (ie, the extent to which a causal conclusion based on a study is warranted) and external validity (ie, the applicability of clinical trial results to routine clinical settings) while ensuring participant safety.9,–12
In several influential studies over the last 2 decades, researchers have evaluated treatments to guide the clinical management of adolescents with major depressive disorder (MDD),13 but the generalizability of these RCT samples to the broader population of adolescents with MDD is unknown. Examining the generalizability of RCTs of pharmacological and psychotherapy interventions for adolescent depression can help administrators and frontline practitioners determine the relevance of these studies for their patients. An understanding of the generalizability of clinical trials might also assist research funding agencies in identifying gaps in knowledge and help guide eligibility criteria for future clinical trials in this clinical population.
In this study, we apply exclusion criteria commonly used in adolescent pharmacological and psychotherapy clinical trials for MDD to a large, nationally representative adolescent population sample of the United States to assess the generalizability of the criteria to adolescents with MDD and to a subsample of adolescents seeking treatment for depression.
Methods
Sample
Data were drawn from the National Comorbidity Survey: Adolescent Supplement (NCS-A), a nationally representative, face-to-face survey of 10 123 adolescents aged 13 to 18 years conducted between February 2001 and January 2004 in the continental United States and described in detail elsewhere.14,–17 The survey was administered by the professional interview staff of the Institute for Social Research at the University of Michigan. The NCS-A was conducted in a dual-frame sample that included a household subsample and a school subsample. These recruitment and consent procedures received full human subjects review and approval from the human subjects committees of Harvard Medical School and the University of Michigan.
The overall NCS-A adolescent response rate after combining the 2 subsamples was 82.9%. One parent or parent surrogate of each participant was asked to complete a self-administered questionnaire (SAQ) that contained informant questions about the adolescent’s developmental history and mental health. The full SAQ was completed by 6491 parents. In the present report, we focus on the 6483 adolescent-parent pairs for which complete data are available from both adolescents and parents. From this group, we selected all participants with a 12-month Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnosis of MDD (N = 592) and a subsample who sought mental health treatment (n = 412).
DSM-IV Diagnostic Interview
Lifetime and 12-month psychiatric diagnoses were made according to DSM-IV18 criteria by using a modified version of the World Health Organization Composite International Diagnostic Interview (CIDI) 3.0, a fully structured interview administered by trained lay interviewers.19 The CIDI was modified to enhance the wording and appropriateness of the instrument for the assessment of adolescents.20,21 Suicidal ideation was assessed by asking, “Have you ever seriously thought about killing yourself?” Adolescents who endorsed lifetime suicidal ideation were questioned about lifetime suicide attempts by asking, “Have you ever attempted suicide?”
Parents who completed the SAQ provided full DSM-IV diagnostic information about MDD, whereas those completing the abbreviated SAQ only reported on attention-deficit/hyperactivity disorder. Information from both the parent and adolescent were combined for major depressive episode (MDE) and behavioral problems and classified as positive if either informant endorsed the diagnostic criteria.22,23 Definitions of all psychiatric disorders followed DSM-IV criteria, and diagnostic hierarchy rules were applied for every disorder, with the exception of oppositional defiant disorder and substance use disorders. Psychiatric disorder diagnoses derived from the modified CIDI (including MDD) had good concordance with a clinical reappraisal subsample.15 There was good concordance between the CIDI Version 3.0 diagnoses from the NCS-A and from the Schedule for Affective Disorders and Schizophrenia for School-Age Children,24 including under the receiver operating characteristic curve of 0.88 for any anxiety disorder, 0.89 for any mood disorder, 0.84 for any disruptive behavior disorder, and 0.94 for any substance disorder.
Clinical Trials’ Exclusion Criteria
Exclusion criteria commonly used in adolescent pharmacological and psychotherapy RCTs for MDD were applied to the full NCS-A sample of individuals with self-report and parental information who met the criteria for the current DSM-IV diagnosis of MDD. The same exclusion criteria were applied to the subsample of adolescents who sought treatment of MDD to investigate potential differences in eligibility between treatment-seeking and non–treatment-seeking adolescents with MDD.1
We collected the exclusion criteria from adolescent clinical trials for MDD included in 2 recent meta-analyses examining the effects of pharmacological25 and psychotherapy26 treatments for adolescents with MDD. Only trials published after 1997 were included in our analyses because journals strengthened their policies after this date, requiring the reporting of exclusion criteria.27 The analysis of exclusion criteria included all 34 trials published after 1997 in the meta-analysis of psychotherapy treatments and all 25 trials published after 1997 in the meta-analysis of pharmacological treatments.
Two coders (S.F. and N.H.) independently collected all eligibility criteria from the clinical trials (Tables 1 and 2). Intercoder reliability was adequate28 (intraclass correlation coefficient, 0.84; 95% confidence interval, 0.68–0.90). Disagreement was resolved by consensus. The median number of exclusion criteria applied was 11 for the pharmacological clinical trials and 4 for the psychotherapy clinical trials. To estimate the representativeness of a typical pharmacological and psychotherapy clinical trial with traditional exclusion criteria, we applied, respectively, the 11 and the 4 most commonly used exclusion criteria to adolescents in the NCS-A who met the criteria for past-year MDD (Tables 3 and 4).
Exclusion Criteria in 25 Clinical Trials Examining the Effects of Pharmacological Treatments for Adolescents With MDD
| Exclusion Criteria, Ranked by Frequencya . | Studies Using the Criteria, Reference No. . | No. of Studies Using the Criteria (25) . |
|---|---|---|
| 1. Lifetime psychotic features | 29,–51 | 23 |
| 2. Lifetime bipolar I or II disorder | 29,–33,35,–52 | 23 |
| 3. Currently taking any psychotropic medication | 30,–36,38,–45,47,–52 | 21 |
| 4. Alcohol or drug abuse and/or dependence (within the last 6 mo) | 29,–32,34,35,37,–43,45,–50,52 | 20 |
| 5. Significant risk of suicideb | 29,–33,35,38,–49,52 | 20 |
| 6. Any current significant physical condition | 30,–32,34,–41,43,44,46,47,49,51 | 17 |
| 7. Developmental disorder or mental deficiency | 30,32,–35,37,–40,42,45,47,–50,52 | 17 |
| 8. Pregnant, breastfeeding, or sexually active without contraception | 31,34,35,38,39,41,–45,47,–52 | 16 |
| 9. Lifetime eating disorder | 29,34,37,–39,41,42,44,–50 | 14 |
| 10. Lifetime obsessive-compulsive disorder | 29,31,32,35,41,42,44,–47,49,51 | 12 |
| 11. Current panic disorder | 31,32,34,35,41,42,44,47,49,51 | 10 |
| 12. Conduct disorder | 30,35,41,42,47,–49,51,52 | 9 |
| 13. Currently receiving psychotherapy | 30,35,38,–40,42,48,49,52 | 9 |
| 14. Neurologic condition | 32,34,38,39,44,45,47,50 | 8 |
| 15. Current generalized anxiety disorder | 31,32,34,35,42,47,49 | 7 |
| 16. Attention-deficit/hyperactivity disorder | 30,31,42,47,–50 | 7 |
| 17. Current social anxiety disorder | 31,32,35,42,47,49 | 6 |
| 18. Current specific phobia | 31,32,35,42,47,49 | 6 |
| 19. Current dysthymia | 31,32,35,42,47,49 | 6 |
| 20. Severe personality disorder | 38,39,42,47,–49 | 6 |
| 21. Current posttraumatic stress disorder | 35,42,45,47,49 | 5 |
| 22. Oppositional defiant disorder | 30,42,48,49 | 4 |
| 23. Severe malnutrition | 47,49 | 2 |
| 24. Low English proficiency | 52 | 1 |
| Exclusion Criteria, Ranked by Frequencya . | Studies Using the Criteria, Reference No. . | No. of Studies Using the Criteria (25) . |
|---|---|---|
| 1. Lifetime psychotic features | 29,–51 | 23 |
| 2. Lifetime bipolar I or II disorder | 29,–33,35,–52 | 23 |
| 3. Currently taking any psychotropic medication | 30,–36,38,–45,47,–52 | 21 |
| 4. Alcohol or drug abuse and/or dependence (within the last 6 mo) | 29,–32,34,35,37,–43,45,–50,52 | 20 |
| 5. Significant risk of suicideb | 29,–33,35,38,–49,52 | 20 |
| 6. Any current significant physical condition | 30,–32,34,–41,43,44,46,47,49,51 | 17 |
| 7. Developmental disorder or mental deficiency | 30,32,–35,37,–40,42,45,47,–50,52 | 17 |
| 8. Pregnant, breastfeeding, or sexually active without contraception | 31,34,35,38,39,41,–45,47,–52 | 16 |
| 9. Lifetime eating disorder | 29,34,37,–39,41,42,44,–50 | 14 |
| 10. Lifetime obsessive-compulsive disorder | 29,31,32,35,41,42,44,–47,49,51 | 12 |
| 11. Current panic disorder | 31,32,34,35,41,42,44,47,49,51 | 10 |
| 12. Conduct disorder | 30,35,41,42,47,–49,51,52 | 9 |
| 13. Currently receiving psychotherapy | 30,35,38,–40,42,48,49,52 | 9 |
| 14. Neurologic condition | 32,34,38,39,44,45,47,50 | 8 |
| 15. Current generalized anxiety disorder | 31,32,34,35,42,47,49 | 7 |
| 16. Attention-deficit/hyperactivity disorder | 30,31,42,47,–50 | 7 |
| 17. Current social anxiety disorder | 31,32,35,42,47,49 | 6 |
| 18. Current specific phobia | 31,32,35,42,47,49 | 6 |
| 19. Current dysthymia | 31,32,35,42,47,49 | 6 |
| 20. Severe personality disorder | 38,39,42,47,–49 | 6 |
| 21. Current posttraumatic stress disorder | 35,42,45,47,49 | 5 |
| 22. Oppositional defiant disorder | 30,42,48,49 | 4 |
| 23. Severe malnutrition | 47,49 | 2 |
| 24. Low English proficiency | 52 | 1 |
Derived from the review of 25 clinical trials (method described in the article).
Of the 20 studies using this criterion, 4 specified “any previous suicide attempt,” 3 “any previous suicide attempt or any suicide plan,” 3 “any previous suicide attempt or suicidal ideation,” and 10 did not detail this criterion.
Exclusion Criteria in 34 Clinical Trials Examining the Effects of Psychological Treatments for Adolescents With MDD
| Exclusion Criteria Present in >10% of the Studies, Ranked by Frequencya . | Studies Using the Criteria, Reference No. . | No. of Studies Using the Criteria (34) . |
|---|---|---|
| 1. Lifetime psychotic features | 53,–74 | 22 |
| 2. Lifetime bipolar I or II disorder | 53,–58,60,–68,71,72,74,75 | 19 |
| 3. Current psychoactive medication | 55,57,58,60,61,63,65,66,68,70,72,73,75,–77 | 15 |
| 4. Significant risk of suicideb | 53,55,–58,60,–63,66,73,78,–80 | 14 |
| 5. Currently receiving psychotherapy | 55,57,58,60,61,63,65,70,72,73,75,–77,79 | 14 |
| 6. Alcohol or drug abuse and/or dependence (within the last 6 mo) | 56,–58,60,–66,70,–72,74 | 14 |
| 7. Developmental disorder or mental deficiency | 57,60,62,63,66,67,69,72,76,80 | 10 |
| 8. Conduct disorder | 53,–55,57,58,61,62,72,78 | 9 |
| 9. Lifetime obsessive-compulsive disorder | 53,54,61,63,–65,71,74 | 8 |
| 10. Current panic disorder | 53,54,62,–65,71,72 | 8 |
| 11. Any current significant physical condition | 56,60,61,65,66,72,74 | 7 |
| 12. Current dysthymia | 53,55,63,–65,71 | 6 |
| 13. Current generalized anxiety disorder | 62,–65,71,72 | 6 |
| 14. Need of hospitalization | 55,56,62,66,69 | 5 |
| 15. Current social anxiety disorder | 63,–65,71 | 4 |
| 16. Current specific phobia | 63,–65,71 | 4 |
| 17. Neurologic condition | 58,62,66 | 3 |
| 18. Lifetime eating disorder | 61,66,74 | 3 |
| 19. Oppositional defiant disorder | 53,54,78 | 3 |
| 20. No parental history of depression or dysthymia | 81,82 | 2 |
| 21. Ongoing physical or sexual abuse | 66,74 | 2 |
| 22. Current posttraumatic stress disorder | 53,54 | 2 |
| 23. Low English proficiency | 59,63 | 2 |
| 24. Attention-deficit hyperactivity disorder | 53,54 | 2 |
| 25. Litigation | 58,59 | 2 |
| 26. Charges of first degree assault, robbery, homicide or rape | 57,59 | 2 |
| 27. Pregnancy or breastfeeding | 74 | 1 |
| 28. Aggressive behavior | 57 | 1 |
| 29. Cluster B personality disorders | 56 | 1 |
| Exclusion Criteria Present in >10% of the Studies, Ranked by Frequencya . | Studies Using the Criteria, Reference No. . | No. of Studies Using the Criteria (34) . |
|---|---|---|
| 1. Lifetime psychotic features | 53,–74 | 22 |
| 2. Lifetime bipolar I or II disorder | 53,–58,60,–68,71,72,74,75 | 19 |
| 3. Current psychoactive medication | 55,57,58,60,61,63,65,66,68,70,72,73,75,–77 | 15 |
| 4. Significant risk of suicideb | 53,55,–58,60,–63,66,73,78,–80 | 14 |
| 5. Currently receiving psychotherapy | 55,57,58,60,61,63,65,70,72,73,75,–77,79 | 14 |
| 6. Alcohol or drug abuse and/or dependence (within the last 6 mo) | 56,–58,60,–66,70,–72,74 | 14 |
| 7. Developmental disorder or mental deficiency | 57,60,62,63,66,67,69,72,76,80 | 10 |
| 8. Conduct disorder | 53,–55,57,58,61,62,72,78 | 9 |
| 9. Lifetime obsessive-compulsive disorder | 53,54,61,63,–65,71,74 | 8 |
| 10. Current panic disorder | 53,54,62,–65,71,72 | 8 |
| 11. Any current significant physical condition | 56,60,61,65,66,72,74 | 7 |
| 12. Current dysthymia | 53,55,63,–65,71 | 6 |
| 13. Current generalized anxiety disorder | 62,–65,71,72 | 6 |
| 14. Need of hospitalization | 55,56,62,66,69 | 5 |
| 15. Current social anxiety disorder | 63,–65,71 | 4 |
| 16. Current specific phobia | 63,–65,71 | 4 |
| 17. Neurologic condition | 58,62,66 | 3 |
| 18. Lifetime eating disorder | 61,66,74 | 3 |
| 19. Oppositional defiant disorder | 53,54,78 | 3 |
| 20. No parental history of depression or dysthymia | 81,82 | 2 |
| 21. Ongoing physical or sexual abuse | 66,74 | 2 |
| 22. Current posttraumatic stress disorder | 53,54 | 2 |
| 23. Low English proficiency | 59,63 | 2 |
| 24. Attention-deficit hyperactivity disorder | 53,54 | 2 |
| 25. Litigation | 58,59 | 2 |
| 26. Charges of first degree assault, robbery, homicide or rape | 57,59 | 2 |
| 27. Pregnancy or breastfeeding | 74 | 1 |
| 28. Aggressive behavior | 57 | 1 |
| 29. Cluster B personality disorders | 56 | 1 |
Derived from the review of 34 clinical trials (method described in the article).
Of the 15 studies using this criterion, 3 specified “any previous suicide attempt,” 3 “suicidal ideation,” and 9 did not detail this criterion.
Estimated Percentage of Adolescents With 12-Month MDE in NCS-A Excluded by Traditional Eligibility Criteria in 25 Adolescent Pharmacological Trials for MDD
| Exclusion Criteria . | 12-mo DSM-IV MDE Cases . | Group Difference (Treatment-Seeking Versus Non–Treatment Seeking) . | |||||
|---|---|---|---|---|---|---|---|
| Total (N = 592) . | Treatment-Seeking Groupa (n = 412) . | Non–Treatment Seeking (n = 180) . | |||||
| n . | % (SE) . | n . | % (SE) . | n . | % (SE) . | Wald χ21 (P) . | |
| 1. Lifetime psychotic features | NA | — | |||||
| 2. Lifetime bipolar I or II disorder | 62 | 11.21 (2.27) | 51 | 12.83 (2.83) | 11 | 6.80 (3.05) | 1.8 (.176) |
| 3. Past-year use of psychotropic medication | 112 | 19.73 (2.75) | 109 | 25.95 (3.09) | 3 | 2.75 (1.72) | 15.9 (.000) |
| 4. Past-year alcohol or drug abuse and/or dependence | 107 | 24.47 (3.43) | 90 | 29.18 (4.13) | 17 | 11.62 (4.59) | 5.1 (.023) |
| 5. Significant risk of suicideb | 100 | 25.34 (4.94) | 91 | 32.74 (5.86) | 9 | 5.15 (2.20) | 19.6 (.000) |
| 6. Past-year significant physical conditionc | 52 | 8.89 (1.69) | 43 | 11.07 (2.24) | 9 | 2.95 (1.25) | 8.5 (.003) |
| 7. Developmental disorderd | 2 | 0.16 (0.13) | 2 | 0.22 (0.18) | 0 | 0.00 (0.00) | — |
| 8. Pregnant, breastfeeding, or sexually active without contraception | NA | — | |||||
| 9. Lifetime eating disorder | 60 | 13.10 (3.38) | 49 | 15.81 (4.39) | 11 | 5.69 (2.47) | 3.7 (.054) |
| 10. Lifetime obsessive-compulsive disorder | NA | — | |||||
| 11. Past-year panic disorder | 41 | 5.82 (1.51) | 33 | 6.39 (1.96) | 8 | 4.28 (2.75) | 0.3 (.592) |
| At least 1 criterion | 325 | 61.88 (3.63) | 270 | 72.71 (3.16) | 55 | 32.31 (5.71) | 32.4 (.000) |
| Exclusion Criteria . | 12-mo DSM-IV MDE Cases . | Group Difference (Treatment-Seeking Versus Non–Treatment Seeking) . | |||||
|---|---|---|---|---|---|---|---|
| Total (N = 592) . | Treatment-Seeking Groupa (n = 412) . | Non–Treatment Seeking (n = 180) . | |||||
| n . | % (SE) . | n . | % (SE) . | n . | % (SE) . | Wald χ21 (P) . | |
| 1. Lifetime psychotic features | NA | — | |||||
| 2. Lifetime bipolar I or II disorder | 62 | 11.21 (2.27) | 51 | 12.83 (2.83) | 11 | 6.80 (3.05) | 1.8 (.176) |
| 3. Past-year use of psychotropic medication | 112 | 19.73 (2.75) | 109 | 25.95 (3.09) | 3 | 2.75 (1.72) | 15.9 (.000) |
| 4. Past-year alcohol or drug abuse and/or dependence | 107 | 24.47 (3.43) | 90 | 29.18 (4.13) | 17 | 11.62 (4.59) | 5.1 (.023) |
| 5. Significant risk of suicideb | 100 | 25.34 (4.94) | 91 | 32.74 (5.86) | 9 | 5.15 (2.20) | 19.6 (.000) |
| 6. Past-year significant physical conditionc | 52 | 8.89 (1.69) | 43 | 11.07 (2.24) | 9 | 2.95 (1.25) | 8.5 (.003) |
| 7. Developmental disorderd | 2 | 0.16 (0.13) | 2 | 0.22 (0.18) | 0 | 0.00 (0.00) | — |
| 8. Pregnant, breastfeeding, or sexually active without contraception | NA | — | |||||
| 9. Lifetime eating disorder | 60 | 13.10 (3.38) | 49 | 15.81 (4.39) | 11 | 5.69 (2.47) | 3.7 (.054) |
| 10. Lifetime obsessive-compulsive disorder | NA | — | |||||
| 11. Past-year panic disorder | 41 | 5.82 (1.51) | 33 | 6.39 (1.96) | 8 | 4.28 (2.75) | 0.3 (.592) |
| At least 1 criterion | 325 | 61.88 (3.63) | 270 | 72.71 (3.16) | 55 | 32.31 (5.71) | 32.4 (.000) |
Percentages are weighted values. NA, not available in NCS-A; —, not applicable.
Defined as receiving any treatment of emotional and behavioral problems in the mental health specialty sector, general medical sector, or school services system during the year preceding the interview.
Defined as having a lifetime history of suicide attempt.
Approximated by series of questions on 12-mo diabetes, HIV/AIDS, and cancer diagnosis (reported by adolescents); frequent high fever (reported by parents); and heart problems and epilepsy and/or seizure (reported by adolescents or parents).
Approximated from parent’s reports about child’s developmental history.
Estimated Percentage of Adolescents With 12-Month MDE in NCS-A Excluded by Traditional Eligibility Criteria in 34 Adolescent Psychological Trials for MDD
| Exclusion Criteria . | 12-mo DSM-IV MDE Cases . | Group Difference (Treatment-Seeking Group Versus Non–Treatment Seeking) . | |||||
|---|---|---|---|---|---|---|---|
| Total (N = 592) . | Treatment-Seeking Groupa (n = 412) . | Non–Treatment Seeking (n = 180) . | |||||
| n . | % (SE) . | n . | % (SE) . | n . | % (SE) . | Wald χ21 (P) . | |
| 1. Lifetime psychotic features | NA | — | |||||
| 3. Lifetime bipolar I or II disorder | 62 | 11.21 (2.27) | 51 | 12.83 (2.83) | 11 | 6.80 (3.05) | 1.8 (.176) |
| 3. Past-year use of psychotropic medication | 112 | 19.73 (2.75) | 109 | 25.95 (3.09) | 3 | 2.75 (1.72) | 15.9 (.000) |
| 4. Significant risk of suicideb | 100 | 25.34 (4.94) | 91 | 32.74 (5.86) | 9 | 5.15 (2.20) | 19.6 (.000) |
| At least 1 criterion | 215 | 42.15 (4.71) | 192 | 52.20 (4.95) | 23 | 14.70 (3.84) | 30.8 (.000) |
| Exclusion Criteria . | 12-mo DSM-IV MDE Cases . | Group Difference (Treatment-Seeking Group Versus Non–Treatment Seeking) . | |||||
|---|---|---|---|---|---|---|---|
| Total (N = 592) . | Treatment-Seeking Groupa (n = 412) . | Non–Treatment Seeking (n = 180) . | |||||
| n . | % (SE) . | n . | % (SE) . | n . | % (SE) . | Wald χ21 (P) . | |
| 1. Lifetime psychotic features | NA | — | |||||
| 3. Lifetime bipolar I or II disorder | 62 | 11.21 (2.27) | 51 | 12.83 (2.83) | 11 | 6.80 (3.05) | 1.8 (.176) |
| 3. Past-year use of psychotropic medication | 112 | 19.73 (2.75) | 109 | 25.95 (3.09) | 3 | 2.75 (1.72) | 15.9 (.000) |
| 4. Significant risk of suicideb | 100 | 25.34 (4.94) | 91 | 32.74 (5.86) | 9 | 5.15 (2.20) | 19.6 (.000) |
| At least 1 criterion | 215 | 42.15 (4.71) | 192 | 52.20 (4.95) | 23 | 14.70 (3.84) | 30.8 (.000) |
Percentages are weighted values. NA, not available in NCS-A; —, not applicable.
Defined as receiving any treatment of emotional and behavioral problems in the mental health specialty sector, general medical sector, or school services system during the year preceding the interview.
Defined as having a lifetime history of suicide attempt.
The percentage of participants excluded was estimated from responses to the modified version of the CIDI 3.0.19 The criteria “lifetime bipolar I or II disorder,” “lifetime eating disorder,” and “past-year panic disorder” were diagnosed by using DSM-IV criteria. The criterion “current or past 6-month drug or alcohol abuse and/or dependence” was defined as having a DSM-IV diagnosis of dependence or abuse on alcohol or drug within the past 12 months. The criterion “significant risk of suicide” was considered met if the adolescent ever attempted suicide. The criterion “current psychotropic medication” was considered present if adolescents reported taking psychotropic medication within the last 12 months. “Developmental disorder” was assessed from parents’ reports of their child’s behavior and developmental history. The criterion “significant physical condition” was indexed by a series of self-report questions regarding the presence of HIV/AIDS, cancer, diabetes, heart problems, and epilepsy and/or seizure in the past year or parental report of an adolescent heart problem, frequent high fever, and epilepsy and/or seizure. Information to approximate the criteria “lifetime psychotic features,” “pregnant, breastfeeding, or sexually active without contraception,” and “lifetime obsessive-compulsive disorder” was not available in the NCS-A.
Among respondents with current MDD, treatment seeking was defined by report of any treatment of emotional and behavioral problems in the mental health specialty sector, general medical sector, or school services system during the year preceding the interview.
Statistical Analyses
We first defined the percentages (and their SEs) of survey participants with MDD who would have been excluded by applying each one of the exclusion criteria in clinical trials for MDD. Because individuals could have been excluded by more than 1 criterion, we also calculated the overall percentage of subjects who would have been excluded by the simultaneous application of all available criteria. We conducted these analyses for all participants with a 12-month DSM-IV diagnosis of MDE (N = 592) and for the subsample who sought treatment (n = 412). Analyses were performed by using SUDAAN 10.0.183 (Research Triangle Institute, Research Triangle Park, NC) to take into account the complex survey design. Wald statistics were used to examine potential differences between adolescents who would have been included in a typical clinical trial for MDD and those who would have been excluded. Significance tests were calculated by using Wald χ2 tests based on coefficient variance-covariance matrices that were adjusted for design effects by using the Taylor series method. Statistical significance was based on 2-sided design-based tests evaluated at a level of significance of .05.
Supplementary Analyses
To determine if the median number of exclusion criteria and the generalizability rate differed between recent and older pharmacological and psychotherapy clinical trials for MDD, we applied the method described above separately for recent trials (ie, published after 2006) and those older (ie, published in 2006 or before).
Results
In the full sample of 592 participants who met DSM-IV criteria for MDE, 61.9% (SE = 3.6) would have been excluded by at least 1 of the 8 most common and operationalizable criteria in pharmacological trials, and 72.7% (SE = 3.2) would have been excluded from the subsample of 412 participants who sought treatment (Table 3). In both the full sample and the treatment seeking subsample, the criterion leading to the largest number of exclusions was “significant risk of suicide.” The criteria “past-year alcohol or drug abuse and/or dependence” and “past-year use of psychotropic medication” also excluded a substantial proportion of respondents.
In psychotherapy trials, the percentage of adolescents excluded by at least 1 of the 3 most common and operationalizable criteria was 42.2% (SE = 4.7) in the full sample and 52.2% (SE = 5.0) in the treatment-seeking subsample (Table 4). In both the full sample and the treatment-seeking subsample, the criterion resulting in the largest number of exclusions was “significant risk of suicide.”
The overall exclusion rate was significantly higher in participants who sought treatment than in those who did not (P < .001 for pharmacological and psychotherapy trials). The prevalence of 12-month alcohol or drug abuse and/or dependence, significant risk of suicide, use of psychotropic medication, and significant medical conditions within the past year were significantly higher in treatment seekers compared with non–treatment seekers in pharmacological trials (Table 3). As compared with the non–treatment-seeking subsample, the treatment-seeking sample in psychological clinical trials also had a significantly higher prevalence of 12-month use of psychotropic medication and a significantly higher proportion of adolescents at risk for suicide (Table 4).
The overall exclusion rate was within the same range in older and recent pharmacological trials (60.2% [SE = 3.6] vs 65.6% [SE = 3.7] with a maximum number of exclusion criteria that could be approximated of 7 out of 11 and 8 out of 11, respectively), and the median number of exclusion criteria was identical (ie, 11) (Supplemental Tables 5 and 6). By contrast, the median number of exclusion criteria and the overall exclusion rate were substantially lower in recent than in older psychotherapy trials (3 vs 5; 32.7% [SE = 4.7] vs 45.6% [SE = 4.4] with a maximum number of exclusion criteria that could be approximated of 2 out of 3 and 3 out of 5, respectively) (Supplemental Tables 7 and 8).
Discussion
In a typical pharmacological trial for MDD, >6 out of 10 adolescents with MDD in the general population and >7 out of 10 among those seeking treatment would have been excluded by at least 1 commonly used study exclusion criteria. In a typical psychotherapy trial for MDD, >4 of 10 adolescents with MDD in the general population and >5 of 10 among those seeking treatment of MDD would have been excluded by at least 1 commonly used study exclusion criteria. Consistent with previous studies in adult samples,2,84,–88 we found that exclusion criteria commonly used in adolescent clinical trials for MDD exclude a substantial proportion of adolescents from participation, particularly those seeking treatment. The representativeness of psychotherapy trials for MDD tended to be higher in recent years, whereas that of pharmacological trials for MDD did not appear to increase over time.
Adolescents with MDD commonly present with comorbid medical and psychiatric disorders,16,88,–93 which would frequently lead to their exclusion from typical MDD clinical trials. In our findings, we suggest that clinical trial results examining the effects of pharmacological and psychotherapy treatments in adolescents with MDD may have limited generalizability to community settings because they tend to include “pure” rather than “typical” patients.94 The higher overall exclusion rate found in typical pharmacological trials compared with typical psychotherapy trials is consistent with recent findings in adults87 and may be partially explained by the larger number of exclusion criteria applied in pharmacological trials.
Pharmacologic trials excluded >60% of the adolescents with MDD. This proportion is comparable to studies in which researchers have assessed the generalizability of pharmacological trials in adults.3,6,88 The use of exclusion criteria in response to concerns about patient safety (eg, pregnancy, significant medical conditions), study feasibility, or interpretability of results is justified.95 However, the use of other criteria may mostly reflect a tradition that has evolved over time within a particular research area,96,97 resulting in a progressive, unnecessary narrowing of the population of eligible patients (eg, substance use disorders or anxiety disorders).98 Our results suggest that this may be particularly true for pharmacological trials whose generalizability did not increase in recent years, in contrast to psychotherapy trials. Those trials might prioritize the inclusion of narrowly defined uncomplicated subjects in an effort to maximize treatment effects. Indeed, comorbid psychiatric disorders and medical conditions are an important source of heterogeneity in treatment response,99,–101 and authors of previous research suggest that patients with psychiatric or general medical comorbidities tend to have poorer treatment outcomes.6,101,–104 This incentive may be greater because the current Food and Drug Administration labeling does not reflect the study subject selection process. Specifically, the Food and Drug Administration indication drawn from these trials are for “adolescent major depressive disorder” and not for “uncomplicated adolescent major depressive disorder.” There is an inherent tension between the wish of investigators and manufacturers to obtain positive results in their trials and the clinical interests of patients and their providers who seek information from broadly representative studies. In this context, there is a need to carefully consider the advantages and disadvantages of applying each exclusion criterion that is not dedicated to increase patient safety and to balance internal and external validity.
We found that the criterion resulting in the largest number of exclusions in pharmacological and psychotherapy trials was “significant risk of suicide.” Given the public health importance of suicide prevention in young people, the exclusion of adolescents with a significant risk of suicide is an especially important example of the tension between the need for more inclusive eligibility criteria to better inform clinical practice and the constraints (such as the stringent safety standards applied to clinical research) that limit the application of those broader criteria.105,106 To address this issue, a potential approach may be to establish efficacy of a specific pharmacological or psychotherapy intervention in a subset of adolescents with MDD and a low risk of suicide and then seek to evaluate its effectiveness in a subset of more vulnerable adolescents.
Application of the eligibility criteria to the treatment-seeking subsample excluded a significantly greater proportion of depressed adolescents from psychotherapy and pharmacological clinical trials for MDD. Consistent with previous research with adults,107,–112 this suggests that individuals with a given disorder who seek treatment tend to have greater illness severity and more psychiatric and medical comorbidities than those who do not seek treatment. Furthermore, high rates of psychiatric and medical comorbidities may increase the perceived need for care, which in turn may influence treatment-seeking behavior in adolescents and their parents.113
For adolescent MDD treatment trials to adequately inform clinical practice, the eligibility fraction must be increased by a progressive broadening of eligibility criteria. However, having a heterogeneous treatment group can make analyses challenging. In our study, we identified several subgroups of adolescents typically excluded from trials (eg, those with substance use disorders or anxiety disorders). Conducting trials in these subpopulations may help inform clinical practice. In addition, developing integrated forms of pharmacotherapy and psychotherapy that target commonly cooccurring psychiatric disorders may yield more informative results for mental health care professionals and research funding agencies.114,115
The current study has several limitations. First, although the NCS-A items closely resemble the exclusion criteria, they do not precisely match them. For example, the 12-month time frame used in the NCS-A when assessing the presence of “alcohol or drug abuse and/or dependence within the last 6 months” could have led to an overestimation of exclusion rates. In addition, because most of the trials that used the criterion “significant suicide risk” did not detail it (ie, attempt, plan, ideation, etc), we operationalized “significant suicide risk” using NCS-A items of reporting a previous suicide attempt. Finally, most of the clinical trials included in this analysis did not detail the specific medical conditions used as exclusion criteria. Other conventions might have yielded different exclusion estimates. Second, 3 exclusion criteria were not available in the NCS-A, which likely led to an underestimation of the overall exclusion rates. Finally, the NCS-A participants were 13 to 18 years of age. Because children with MDD tend to have fewer psychiatric and general medical comorbidities than adolescents with MDD,116 exclusion rates of clinical trials for MDD may be lower in children than adolescents.
Conclusions
Because clinical trials of adolescent MDD exclude adolescents with psychiatric and addictive disorders, significant medical conditions, suicide risk, and other characteristics, they have limited external validity. These studies represent important progress in the development of evidence-based treatments for adolescent MDD. However, our results support that careful consideration should be given to balancing eligibility criteria and adequate internal validity with applicability in routine clinical care while ensuring patient safety.
- CIDI
Composite International Diagnostic Interview
- DSM-IV
Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition
- MDD
major depressive disorder
- MDE
major depressive episode
- NCS-A, National Comorbidity Survey
Adolescent Supplement
- RCT
randomized controlled trial
- SAQ
self-administered questionnaire
The views and opinions expressed in this report are those of the authors and should not be construed to represent the views of any of the sponsoring organizations, agencies, or the US government.
Drs Blanco and Merikangas conceptualized and designed the study and drafted the initial manuscript; Dr Hoertel conceptualized and designed the study, collected all eligibility criteria from the clinical trials, and drafted the initial manuscript; Dr Franco collected all eligibility criteria from the clinical trials and reviewed and revised the manuscript; Ms He conducted the initial analyses; Drs Olfson, López, González-Pinto, and Limosin critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: Supported by the Intramural Research Program of the National Institute of Mental Health (ZIAMH002808), National Institutes of Health grants MH076051 and MH082773 (Drs Blanco and Olfson), and the New York State Psychiatric Institute (Drs Blanco, Hoertel, and Olfson). Funded by the National Institutes of Health (NIH).
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2017-3098.
References
Competing Interests
POTENTIAL CONFLICT OF INTEREST: Dr Blanco owns stock options in Eli Lilly and Company and Sanofi; Dr González-Pinto has received grants and served as consultant, advisor, or continuing medical education speaker for the following entities: Almirall, AstraZeneca, Cephalon, Eli Lilly, Bristol-Myers Squibb, GlaxoSmithKline, Janssen-Cilag, Jazz, Johnson & Johnson, Lundbeck, Merck, Otsuka, Pfizer, Servier, Sanofi-Aventis, Schering-Plough, Solvay, Centre for Biomedical Research in Mental Health Network, Carlos III Institute, Basque Government, Stanley Medical Research Institute, and Wyeth; Dr Limosin is a member of the speakers and/or advisory boards for Janssen, Euthérapie, Lundbeck, and Roche; the other authors have indicated they have no potential conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
