BACKGROUND

The utility of CRAFFT (Car, Relax, Alone, Forget, Friends, Trouble) in identifying current and future problematic substance use and substance use disorders (SUDs) in pediatric emergency department (PED) patients is unknown. We conducted a secondary analysis of a study in 16 PEDs to determine the concurrent and predictive validity of CRAFFT with respect to SUD.

METHODS:

At baseline, 4753 participants aged 12 to 17 years completed an assessment battery (CRAFFT and other measures of alcohol, drug use, and risk behaviors). A subsample was readministered the battery at 1-, 2-, and 3-year follow-up to investigate future SUDs.

RESULTS:

Of 2175 participants assigned to follow-up, 1493 (68.6%) completed 1-year, 1451 (66.7%) completed 2-year, and 1265 (58.1%) completed the 3-year follow-up. A baseline CRAFFT value of ≥2 was significantly associated with problematic substance use or mild or moderate to severe SUD diagnosis on the Diagnostic Interview Schedule for Children at baseline (P < .001). The results persisted after 1, 2, and 3 years (P < .001). The best combined sensitivity and specificity was achieved with a baseline CRAFFT value of ≥1 as a cutoff for predicting problematic substance use and a Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition diagnosis of mild SUD at 1, 2, and 3 years. The baseline CRAFFT score that best predicted a moderate to severe SUD at 1 year was ≥2; but at 2 and 3 years, the cutoff score was ≥1.

CONCLUSIONS:

CRAFFT has good concurrent validity for problematic substance use and SUD in PED patients and is useful in predicting SUDs at up to 3 years follow-up but with limited sensitivity.

What’s Known on This Subject:

The Car, Relax, Alone, Forget, Friends, Trouble screen is used to screen for problematic substance use and substance use disorders (SUDs) among youth in multiple settings, but its concurrent and predictive validity among youth who visit the pediatric emergency department is not known.

What This Study Adds:

The Car, Relax, Alone, Forget, Friends, Trouble screen can be used to identify patients in the pediatric emergency department with current problematic substance use and SUDs. It has value in predicting SUDs, but its sensitivity is limited.

Use of alcohol and other drugs (AOD) is common among US adolescents and increases with age.1 Among 12- to 17-year-old youth in the United States in 2017, the past-year prevalence of a substance use disorder (SUD) (alcohol or illicit drugs) was 4%, and the prevalence was 1.8% for an alcohol use disorder (AUD).2 Alcohol and drug screening, brief intervention, and referral to treatment3 in youth is recommended by the American Academy of Pediatrics,4 although it is not yet clear whether screening, brief intervention, and referral to treatment is an effective approach to risky alcohol use among adolescent patients in acute care settings.5,6 Nonetheless, a screen that can be used to reliably predict current and future misuse of AOD would be valuable.

The 6-item CRAFFT (mnemonic of key letters in the test’s questions: Car; Relax; Alone; Forget; Friends; Trouble) questionnaire is a brief adolescent substance use screening option used in clinic settings.7 Studies have revealed that the sensitivity of the CRAFFT is 76% to 92%, the specificity is 76% to 94%, and the internal consistency is 0.68 to 0.81.8,12 Authors of past studies who have evaluated the psychometric properties of the CRAFFT have used Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria.9,11,13,14 In 2013, the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) merged substance abuse and dependence into a single diagnosis of SUD, dropping the legal involvement item and adding an item about craving.15 By using DSM-5 criteria, the CRAFFT tool has demonstrated excellent sensitivity and specificity at its established cut point of 2 for identifying adolescents in need of further assessment for use of AOD among youth who attended a primary care clinic in the United States.16 

Our objective for the current study was to determine the concurrent and predictive validity of the CRAFFT screen with respect to SUD, as defined by the Diagnostic Interview Schedule for Children (DISC),17 in a pediatric emergency department (PED) setting with a large diverse sample of the US adolescent population. We hypothesized that the baseline CRAFFT screen would accurately predict problematic substance use and SUDs at baseline and at the 1-, 2-, and 3-year follow-up.

A criterion assessment battery was self-administered on a tablet computer by participants 12 to 17 years of age in 16 PEDs with diverse geographic locations (cumulative 0.94 million annual emergency department [ED] visits) affiliated with the Pediatric Emergency Care Applied Research Network (PECARN)18 between May 2013 and June 2015. The original study methodology is described elsewhere.19 Institutional review board approval was obtained by all sites before participant enrollment. Informed parental consent and adolescent assent were obtained. Parents and youth were informed that parents would not have access to their teenagers’ responses and that participant confidentiality would only be breached to protect the safety and welfare of the participant. No responses were used for planning clinical care because the research and clinical staff were unaware of the subjects’ responses. Children with a non–life-threatening injury, illness, or mental health condition and children who were medically, cognitively, and behaviorally stable were included. Exclusion criteria included severe and/or acute emotional distress, cognitive impairment, inability to provide informed assent, no accompanying adult qualified to provide written informed consent, no telephone or residential address, previous enrollment, and inability to read and speak English or Spanish. The assessment battery included the CRAFFT screen (www.crafft.org) along with the DISC,17 a structured, DSM-5–based interview used to determine alcohol use diagnoses over the previous year. To control for order effects, 6 differently ordered assessment battery protocols were randomly assigned in the Web-based survey tool (DatStat, Inc, Seattle, WA). A Spanish version was available for youth who preferred to read in Spanish. Participants were first asked to think about the past-month and past–12-month time frames. They consulted a calendar to remember any important events to help them remember their intake of AOD during the preceding 12 months. If the participants answered “no” to the 3 Part A questions on the CRAFFT, they answered the first Part B question only. If they answered “yes” to any of the Part A questions, they were administered all 6 Part B questions (Supplemental Information). A random subsample of the original sample was re-administered the criterion assessment battery at the 1-, 2-, and 3-year follow-up to examine the predictive validity of the CRAFFT.19 Inclusion at follow-up was not sequential, and participants were included regardless of the number or sequence of follow-ups completed. Therefore, each follow-up group was independent of each other. Follow-up assessments were completed by March 2018. Compared with the overall sample, those who completed the follow-up assessments were more likely to be girls, white, and non-Hispanic; they had a slightly lower mean CRAFFT score. Age, risk category (as determined by the National Institute on Alcohol Abuse and Alcoholism [NIAAA] 2-question screen), and DSM-5 outcomes were similar between those with and without follow-up data (Supplemental Table 4).

Concurrent validity, the degree to which the results of a test are comparable with those of an established gold standard measure of the same construct, was assessed by comparing CRAFFT results to the DSM-5 version of the AUD and SUD module of the DISC17 as the gold standard. The DISC, 1 of the most widely used and studied mental health interviews, has been tested in both clinical and community populations20 ages 9 to 17 years and has been used in a number of ED studies.21,22 The DISC has been shown to have high sensitivity (0.73–1.0) for psychiatric disorders, including SUD.17 Risk classification on the DSM-5 was as follows: problematic use: 1 symptom; mild SUD: 2 to 3 symptoms; and moderate or severe SUD: ≥4 symptoms.

Concurrent validity was examined by using the χ2 statistic to compare a categorization of problematic use and a DISC diagnosis of mild SUD or moderate or severe SUD against a CRAFFT score of ≥2. Additionally, a receiver operating characteristic curve (ROC) analysis was used to investigate possible cut points of the CRAFFT screen score for detecting DISC classifications for the whole cohort and was also stratified by sex. We defined the optimal cut point as the point in which the sum of sensitivity and specificity was maximized. Test characteristics were calculated at each potential cut point, and the area under the curve was used to provide an assessment of the overall accuracy of the CRAFFT screen in predicting DISC classifications. Dichotomous variables were defined as follows: problematic use = problematic use or mild, moderate, or severe DSM-5 SUD versus no problematic use; mild SUD = mild, moderate, or severe SUD versus problematic use or lower; and moderate SUD = moderate or severe SUD versus mild SUD or lower.

With this methodology, sensitivity was used to measure the probability that the CRAFFT screen appropriately categorized youth with problematic use or a DISC diagnosis of SUD, whereas specificity was used to measure the probability that the CRAFFT screen appropriately categorized youth who did not have problematic use or a DISC diagnosis of SUD. To examine predictive validity, the above analyses were repeated by using DISC classification at the 1-, 2-, and 3-year follow-up.

Sample size was based on the ROC analysis of the NIAAA 2-question screen (and, in particular, sensitivity) as the basis for the sample size requirements in the initial study.19 On the basis of an assumption of a target sensitivity of 90% (±2.5%), ∼550 participants who would screen positive for problematic substance use or higher would be required. Existing data suggest that ∼4% of younger children (12–14 years) and 18% of older children (15–17 years) would fall into this group.9,23 If half of the enrolled participants belonged to each of these age groups, we would need ∼5000 participants to observe the required number of positive screen results.

With respect to predictive validity with an estimated sensitivity of 90% (±6), ∼1000 participants would need to be enrolled under the assumption that ∼110 participants would have positive test results. Assuming an attrition of 20% among participants, we would need to enroll 1250 participants in the predictive validity component of the study. All analyses were performed by using SAS version 9.4 (SAS Institute, Inc, Cary, NC).

The analyses included results from 4753 participants who completed all baseline CRAFFT questions and a subsample of 2175 participants who were assigned to follow-up for 3 years. Of these participants, 2143 (98.5%) answered all CRAFFT and DSM-5 substance use questions at baseline. There were no differences, with the exception of ethnicity (P = .04), in the demographic and baseline substance use characteristics of subjects who were chosen for follow-up versus those who were not chosen for follow-up. Of those assigned to follow-up, 1493 (68.6%), 1451 (66.7%), and 1265 (58.1%) participants completed follow-up at 1, 2, and 3 years, respectively (Fig 1). There were more girls than boys (54% vs 46%), and the mean age was 14.4 years (SD 1.7). Forty-six percent of participants identified as white, and 26% identified as African American; 27% identified as Hispanic. Virtually all participants completed the baseline survey in English. Parent educational levels, number of parents in the household, and educational performance of the participants are presented in Table 1.

FIGURE 1

Case ascertainment and follow-up at 1, 2, and 3 years. Inclusion at follow-up was not sequential, and participants were included regardless of the number or sequence of follow-up completed. Therefore, each follow-up group is independent of each other.

FIGURE 1

Case ascertainment and follow-up at 1, 2, and 3 years. Inclusion at follow-up was not sequential, and participants were included regardless of the number or sequence of follow-up completed. Therefore, each follow-up group is independent of each other.

TABLE 1

Characteristics of Participants Who Answered CRAFFT and DSM-5 Alcohol Use Questions at Baseline

Substance Use Categories Defined by DISC CriteriaOverall (N = 2143)
No Use (n = 2086)Problematic Use (n = 26)Mild SUD (n = 18)SUD (n = 13)
Participant biological sex, n (%)
 Male 959 (46.0) 13 (50.0) 9 (50.0) 6 (46.2) 987 (46.1) 
 Female 1127 (54.0) 13 (50.0) 9 (50.0) 7 (53.8) 1156 (53.9) 
Participant age at baseline      
n 2086 26 18 13 2143 
 Mean (SD), y 14.4 (1.66) 16.1 (0.74) 15.9 (1.23) 15.7 (1.25) 14.4 (1.66) 
Participant race, n (%)      
 White 955 (45.8) 13 (50.0) 13 (72.2) 4 (30.8) 985 (46.0) 
 African American 546 (26.2) 6 (23.1) 1 (5.6) 1 (7.7) 554 (25.9) 
 American Indian, Alaskan native, Asian American, native Hawaiian, or other Pacific Islander 96 (4.6) 0 (0.0) 0 (0.0) 4 (30.8) 100 (4.7) 
 >1 race 162 (7.8) 3 (11.5) 2 (11.1) 1 (7.7) 168 (7.8) 
 Unknown or not reported 327 (15.7) 4 (15.4) 2 (11.1) 3 (23.1) 336 (15.7) 
Participant ethnicity, n (%)      
 Hispanic or Latino 563 (27.0) 8 (30.8) 5 (27.8) 6 (46.2) 582 (27.2) 
 Not Hispanic or Latino 1419 (68.0) 17 (65.4) 12 (66.7) 7 (53.8) 1455 (67.9) 
 Unknown or not reported 104 (5.0) 1 (3.8) 1 (5.6) 0 (0.0) 106 (4.9) 
Version (language) of baseline survey, n (%)      
 English 2064 (98.9) 26 (100.0) 18 (100.0) 13 (100.0) 2121 (99.0) 
 Spanish 22 (1.1) 0 (0.0) 0 (0.0) 0 (0.0) 22 (1.0) 
Parent education level, n (%)      
 Less than high school 316 (15.1) 5 (19.2) 1 (5.6) 3 (23.1) 325 (15.2) 
 High school graduate 360 (17.3) 3 (11.5) 2 (11.1) 3 (23.1) 368 (17.2) 
 Some school after high school 436 (20.9) 9 (34.6) 5 (27.8) 1 (7.7) 451 (21.0) 
 Graduated from college or a university 535 (25.6) 4 (15.4) 3 (16.7) 6 (46.2) 548 (25.6) 
 Professional training beyond a 4-y college 299 (14.3) 3 (11.5) 5 (27.8) 0 (0.0) 307 (14.3) 
 Missing 140 (6.7) 2 (7.7) 2 (11.1) 0 (0.0) 144 (6.7) 
No. parents at home, n (%)      
 None 62 (3.0) 1 (3.8) 2 (11.1) 0 (0.0) 65 (3.0) 
 1 774 (37.1) 13 (50.0) 5 (27.8) 7 (53.8) 799 (37.3) 
 2 1244 (59.6) 12 (46.2) 11 (61.1) 6 (46.2) 1273 (59.4) 
 Missing 6 (0.3) 0 (0.0) 0 (0.0) 0 (0.0) 6 (0.3) 
Average grades in school, n (%)      
 Mostly A’s 372 (17.8) 4 (15.4) 3 (16.7) 0 (0.0) 379 (17.7) 
 A’s and B’s 723 (34.7) 8 (30.8) 6 (33.3) 4 (30.8) 741 (34.6) 
 Mostly B’s 226 (10.8) 1 (3.8) 1 (5.6) 2 (15.4) 230 (10.7) 
 B’s and C’s 425 (20.4) 7 (26.9) 4 (22.2) 3 (23.1) 439 (20.5) 
 Mostly C’s and below 263 (12.6) 6 (23.1) 4 (22.2) 4 (30.8) 277 (12.9) 
 Not applicable 17 (0.8) 0 (0.0) 0 (0.0) 0 (0.0) 17 (0.8) 
 “I am not sure” or “I prefer not to answer” 55 (2.6) 0 (0.0) 0 (0.0) 0 (0.0) 55 (2.6) 
 Missing 5 (0.2) 0 (0.0) 0 (0.0) 0 (0.0) 5 (0.2) 
Substance Use Categories Defined by DISC CriteriaOverall (N = 2143)
No Use (n = 2086)Problematic Use (n = 26)Mild SUD (n = 18)SUD (n = 13)
Participant biological sex, n (%)
 Male 959 (46.0) 13 (50.0) 9 (50.0) 6 (46.2) 987 (46.1) 
 Female 1127 (54.0) 13 (50.0) 9 (50.0) 7 (53.8) 1156 (53.9) 
Participant age at baseline      
n 2086 26 18 13 2143 
 Mean (SD), y 14.4 (1.66) 16.1 (0.74) 15.9 (1.23) 15.7 (1.25) 14.4 (1.66) 
Participant race, n (%)      
 White 955 (45.8) 13 (50.0) 13 (72.2) 4 (30.8) 985 (46.0) 
 African American 546 (26.2) 6 (23.1) 1 (5.6) 1 (7.7) 554 (25.9) 
 American Indian, Alaskan native, Asian American, native Hawaiian, or other Pacific Islander 96 (4.6) 0 (0.0) 0 (0.0) 4 (30.8) 100 (4.7) 
 >1 race 162 (7.8) 3 (11.5) 2 (11.1) 1 (7.7) 168 (7.8) 
 Unknown or not reported 327 (15.7) 4 (15.4) 2 (11.1) 3 (23.1) 336 (15.7) 
Participant ethnicity, n (%)      
 Hispanic or Latino 563 (27.0) 8 (30.8) 5 (27.8) 6 (46.2) 582 (27.2) 
 Not Hispanic or Latino 1419 (68.0) 17 (65.4) 12 (66.7) 7 (53.8) 1455 (67.9) 
 Unknown or not reported 104 (5.0) 1 (3.8) 1 (5.6) 0 (0.0) 106 (4.9) 
Version (language) of baseline survey, n (%)      
 English 2064 (98.9) 26 (100.0) 18 (100.0) 13 (100.0) 2121 (99.0) 
 Spanish 22 (1.1) 0 (0.0) 0 (0.0) 0 (0.0) 22 (1.0) 
Parent education level, n (%)      
 Less than high school 316 (15.1) 5 (19.2) 1 (5.6) 3 (23.1) 325 (15.2) 
 High school graduate 360 (17.3) 3 (11.5) 2 (11.1) 3 (23.1) 368 (17.2) 
 Some school after high school 436 (20.9) 9 (34.6) 5 (27.8) 1 (7.7) 451 (21.0) 
 Graduated from college or a university 535 (25.6) 4 (15.4) 3 (16.7) 6 (46.2) 548 (25.6) 
 Professional training beyond a 4-y college 299 (14.3) 3 (11.5) 5 (27.8) 0 (0.0) 307 (14.3) 
 Missing 140 (6.7) 2 (7.7) 2 (11.1) 0 (0.0) 144 (6.7) 
No. parents at home, n (%)      
 None 62 (3.0) 1 (3.8) 2 (11.1) 0 (0.0) 65 (3.0) 
 1 774 (37.1) 13 (50.0) 5 (27.8) 7 (53.8) 799 (37.3) 
 2 1244 (59.6) 12 (46.2) 11 (61.1) 6 (46.2) 1273 (59.4) 
 Missing 6 (0.3) 0 (0.0) 0 (0.0) 0 (0.0) 6 (0.3) 
Average grades in school, n (%)      
 Mostly A’s 372 (17.8) 4 (15.4) 3 (16.7) 0 (0.0) 379 (17.7) 
 A’s and B’s 723 (34.7) 8 (30.8) 6 (33.3) 4 (30.8) 741 (34.6) 
 Mostly B’s 226 (10.8) 1 (3.8) 1 (5.6) 2 (15.4) 230 (10.7) 
 B’s and C’s 425 (20.4) 7 (26.9) 4 (22.2) 3 (23.1) 439 (20.5) 
 Mostly C’s and below 263 (12.6) 6 (23.1) 4 (22.2) 4 (30.8) 277 (12.9) 
 Not applicable 17 (0.8) 0 (0.0) 0 (0.0) 0 (0.0) 17 (0.8) 
 “I am not sure” or “I prefer not to answer” 55 (2.6) 0 (0.0) 0 (0.0) 0 (0.0) 55 (2.6) 
 Missing 5 (0.2) 0 (0.0) 0 (0.0) 0 (0.0) 5 (0.2) 

The baseline CRAFFT items are described in Table 2, and the baseline CRAFFT scores by sex and age group are described in Supplemental Table 5. High school youth contributed to the vast majority of positive answers to the 6 CRAFFT items. The median baseline CRAFFT score for a problematic use classification was 3.0 (interquartile range [IQR] 2.0–4.0). For mild SUD, it was 3.0 (IQR 1.0–4.0). For moderate or severe SUD, it was 5.0 (IQR 4.0–6.0). There was only 1 middle school student diagnosed with a mild SUD; another middle school student was diagnosed with a moderate or severe SUD at baseline. Supplemental Table 6 shows the participants’ DISC categories at baseline and follow-up.

TABLE 2

Participants’ Positive Responses to Baseline CRAFFT Items by Sex and Age Group

Baseline CRAFFT Items
1 (Car)2 (Relax)3 (Alone)4 (Forget)5 (Family)6 (Trouble)
Total, n (%) 348 (16.0) 144 (6.6) 123 (5.7) 100 (4.6) 68 (3.1) 86 (4.0) 
Participant biological sex, n (%)       
 Male 142 (40.8) 68 (47.2) 54 (43.9) 46 (46.0) 39 (57.4) 39 (45.3) 
 Female 206 (59.2) 76 (52.8) 69 (56.1) 54 (54.0) 29 (42.6) 47 (54.7) 
Age group, n (%)       
 Middle school 72 (20.7) 12 (8.3) 8 (6.5) 7 (7.0) 9 (13.2) 5 (5.8) 
 High school 276 (79.3) 132 (91.7) 115 (93.5) 93 (93.0) 59 (86.8) 81 (94.2) 
Baseline CRAFFT Items
1 (Car)2 (Relax)3 (Alone)4 (Forget)5 (Family)6 (Trouble)
Total, n (%) 348 (16.0) 144 (6.6) 123 (5.7) 100 (4.6) 68 (3.1) 86 (4.0) 
Participant biological sex, n (%)       
 Male 142 (40.8) 68 (47.2) 54 (43.9) 46 (46.0) 39 (57.4) 39 (45.3) 
 Female 206 (59.2) 76 (52.8) 69 (56.1) 54 (54.0) 29 (42.6) 47 (54.7) 
Age group, n (%)       
 Middle school 72 (20.7) 12 (8.3) 8 (6.5) 7 (7.0) 9 (13.2) 5 (5.8) 
 High school 276 (79.3) 132 (91.7) 115 (93.5) 93 (93.0) 59 (86.8) 81 (94.2) 

Table 3 displays DISC-derived DSM-5 classification of problematic substance use, mild SUD, or moderate or severe SUD versus a CRAFFT risk assessment of <2 or ≥2 at baseline and follow-up at 1, 2, and 3 years. A baseline CRAFFT value of ≥2 was significantly associated with all substance use categories at baseline and at the 1-, 2-, and 3-year follow-up.

TABLE 3

Distribution of Substance Use Categories by DISC Criteria at Baseline and Follow-up Versus CRAFFT at Baseline

Baseline CRAFFT <2, n/N (%)Baseline CRAFFT ≥2, n/N (%)P
Baseline    
 Problematic use 7/1967 (0.4) 50/176 (28.4) <.001a 
 Mild SUD 5/1967 (0.3) 26/176 (14.8) <.001a 
 Moderate or severe SUD 0/1967 (0.0) 13/176 (7.4) <.001a 
1-y follow-up    
 Problematic use 25/1411 (1.8) 29/117 (24.8) <.001a 
 Mild SUD 11/1411 (0.8) 17/117 (14.5) <.001a 
 Moderate or severe SUD 3/1411 (0.2) 10/117 (8.5) <.001a 
2-y follow-up    
 Problematic use 55/1384 (4.0) 33/112 (29.5) <.001a 
 Mild SUD 20/1384 (1.4) 20/112 (17.9) <.001a 
 Moderate or severe SUD 4/1384 (0.3) 7/112 (6.3) <.001a 
3-y follow-up    
 Problematic use 66/1216 (5.4) 27/83 (32.5) <.001a 
 Mild SUD 37/1216 (3.0) 18/83 (21.7) <.001a 
 Moderate or severe SUD 10/1216 (0.8) 8/83 (9.6) <.001a 
Baseline CRAFFT <2, n/N (%)Baseline CRAFFT ≥2, n/N (%)P
Baseline    
 Problematic use 7/1967 (0.4) 50/176 (28.4) <.001a 
 Mild SUD 5/1967 (0.3) 26/176 (14.8) <.001a 
 Moderate or severe SUD 0/1967 (0.0) 13/176 (7.4) <.001a 
1-y follow-up    
 Problematic use 25/1411 (1.8) 29/117 (24.8) <.001a 
 Mild SUD 11/1411 (0.8) 17/117 (14.5) <.001a 
 Moderate or severe SUD 3/1411 (0.2) 10/117 (8.5) <.001a 
2-y follow-up    
 Problematic use 55/1384 (4.0) 33/112 (29.5) <.001a 
 Mild SUD 20/1384 (1.4) 20/112 (17.9) <.001a 
 Moderate or severe SUD 4/1384 (0.3) 7/112 (6.3) <.001a 
3-y follow-up    
 Problematic use 66/1216 (5.4) 27/83 (32.5) <.001a 
 Mild SUD 37/1216 (3.0) 18/83 (21.7) <.001a 
 Moderate or severe SUD 10/1216 (0.8) 8/83 (9.6) <.001a 
a

Pearson χ2 test of independence.

An optimal cutoff of ≥2 for a baseline CRAFFT score predicted DISC-derived problematic substance use, mild SUD, and moderate or severe SUD, respectively, for the whole cohort (sensitivity and specificity values are presented in Figs 24) and also by sex at baseline (Supplemental Figs 5–10).

FIGURE 2

ROC for the CRAFFT when predicting DSM-5 diagnosis of problematic substance use. AUC, area under the curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

FIGURE 2

ROC for the CRAFFT when predicting DSM-5 diagnosis of problematic substance use. AUC, area under the curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

FIGURE 3

ROC for the CRAFFT when predicting DSM-5 diagnosis of mild SUD. AUC, area under the curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

FIGURE 3

ROC for the CRAFFT when predicting DSM-5 diagnosis of mild SUD. AUC, area under the curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

FIGURE 4

ROC for the CRAFFT when predicting DSM-5 diagnosis of moderate or severe SUD. AUC, area under the curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

FIGURE 4

ROC for the CRAFFT when predicting DSM-5 diagnosis of moderate or severe SUD. AUC, area under the curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

Figures 2 and 3 demonstrate that an optimal cutoff of ≥1 for a baseline CRAFFT score predicted DISC-derived problematic substance use and mild SUD at 1-, 2-, and 3-year follow-up for the whole cohort. Sensitivity and specificity values are also reported. A cutoff of ≥1 for a baseline CRAFFT score was optimal in predicting DISC-derived problematic substance use and mild SUD at 1-, 2-, and 3-year follow-up on the basis of sex (Supplemental Figs 5, 6, 8, and 9).

An optimal cutoff of ≥2 for a baseline CRAFFT score predicted a moderate or severe, DISC-derived SUD diagnosis at 1 year for the whole cohort (Fig 4) and by sex (Supplemental Figs 7 and 10). Additionally, a cutoff of ≥1 predicted a moderate or severe DISC-derived SUD diagnosis at the 2- and 3-year follow-up for the whole cohort. Sensitivity and specificity are also shown in Fig 4. The optimal cutoff for a baseline CRAFFT score in predicting a moderate or severe, DISC-derived SUD diagnosis by sex at the 2- and 3-year follow-up varied between ≥1 and ≥2 (Supplemental Figs 7 and 10). The ROC analyses were conducted by using just the high school students, and the optimal cutoff points remained the same as those for the entire sample.

In this study, we observed that at a cut point of ≥2, the CRAFFT demonstrated good sensitivity and specificity with respect to a problematic alcohol use classification as well as a DSM-5 diagnosis of SUD for the whole cohort and by sex. The area under the curve values were 0.93 for problem use, 0.90 for mild SUD, and 0.98 for moderate or severe SUD for the whole cohort, which is comparable with those of other studies of youth in different settings, such as community health centers15 and hospital-based adolescent clinics.7,9 

We used the optimal cut point as the point in which the sum of sensitivity and specificity were maximized. If another cut point is used, clinicians need to determine tolerance for false-positives. Misclassification of a youth as a nonuser when, in fact, he or she has problematic use could inadvertently reinforce the current level of use, when correct identification with brief advice to reduce use by the PED clinical staff might instead have a positive effect. Therefore, we believe that an instrument that can be used to screen for use of AOD should have a high sensitivity because of the potential to decrease future morbidity.

A lower specificity designation would mean a higher number of false-positives. The cost of a false-positive screen result would indicate a need for an additional interview by the clinician to accurately assess the patient’s use of AOD. A PED would need to decide what level of specificity is acceptable so that it does not negatively impact overall ED patient workflow and efficiency. Using a cut point of 1 would have decreased the specificity while still preserving the sensitivity and negative predictive value. We believe that a cut point of 2 on the CRAFFT would achieve an optimum trade-off between benefit and risk.

The prevalence of problematic use of AOD or higher on the CRAFFT has varied from 8% to 30% across health care settings, including school-based health centers.8 In our study, the prevalence was 2.65%. Thus, the positive predictive value of the CRAFFT in screening youth for AOD in the PED might be lower than that at the other practice locations. The lower prevalence might also be explained by the fact that our cohort had a younger sample compared with those of other studies. Alternatively, we required parental consent in our study, which has been shown to negatively affect youth reporting of substance use.24 

The CRAFFT is advantageous because it can be self-administered by the teenager and delivered on an electronic platform and because the responses can direct the physician to clinically actionable problem categories.25 In addition, it screens for AOD, unlike other instruments (NIAAA 2-question screen; Alcohol Use Disorders Identification Test; Tolerance, Worried, Eye-Opener, Amnesia, K/Cut-down)26 that are only used to assess alcohol use. The disadvantages of the CRAFFT are that youth may not consider illicit drugs to include over-the-counter medications, synthetic substances, herbal preparations, or prescription medications, misuse of which has increased in recent times.27 In this study, we used the original version of the CRAFFT screen. However, the CRAFFT has since been updated (CRAFFT version 2.1) with opening questions on past–12-month frequency of use of AOD to improve sensitivity to detect substance use.28 

Our findings on the concurrent validity of the CRAFFT differ from those of a systematic review performed by Newton et al26 who compared 10 AOD-use instruments with a DSM-IV reference standard for AUD, SUD, or cannabis use disorder. On the basis of their findings, they recommended that ED clinicians should use 2 diagnostic questions that are based on DSM-IV criteria for screening youth alcohol misuse and a single DISC question for previous-year cannabis use for detecting cannabis misuse. They observed that the CRAFFT and other screens had modest accuracy for detecting use of AOD in youth.26 

We also observed that the CRAFFT screen at baseline had modest sensitivity and specificity in predicting problematic use or a DSM-5 diagnosis of SUD at 1, 2, and 3 years for the whole cohort and by sex. A cut point of 1 on a baseline CRAFFT was able to predict problematic use and mild SUD at up to 3 years, with a higher specificity being offset by reduced sensitivity for each succeeding year. A cut point of ≥2 on a baseline CRAFFT was able to predict a moderate or severe SUD at 1 year, whereas a cut point of ≥1 on a baseline CRAFFT predicted moderate or severe SUD at 2 and 3 years, again with higher specificity being countered by reduced sensitivity for each succeeding year. The strong evidence for concurrent validity and predictive validity at 1 year suggests the potential usefulness of annual recurrent screening. The prevalence of past-year illicit drug use in US adolescents increases from 13% in eighth-graders to 40% in 12th-graders; for past-year alcohol use, the prevalence is 18% among eighth-graders, increasing to 56% among 12th-graders.26 Yet, the trends for these drugs have remained relatively stationary for the past few years.29 Furthermore, the prevalence of daily use of alcohol and marijuana (as a marker of excessive drug use) remains relatively low.29 This may explain why the drug usage rates and risk level for substance use among our patients were relatively stable and remained low through the course of our study. Another possibility is that some patients who were identified as at risk may have received counseling and or treatment after the ED visit. However, we do not have evidence to support or disprove this speculation.

Our group (Linakis et al30) previously investigated the predictive validity of the NIAAA 2-question screen in identifying alcohol misuse and AUD in this same sample, whereas in this study, we examined substance use and SUDs. In both studies, the predictive validity of the respective instruments (the CRAFFT and the 2-question NIAAA screen) diminished at the 2- and 3-year follow-up, suggesting that to increase screening accuracy, it is necessary to screen youth every year for SUD and AUD. Furthermore, the positive predictive value of the CRAFFT and the NIAAA screen for SUD and AUD was low (8.5% of youth with positive screen results on the CRAFFT developed moderate or severe SUD after 1 year; 7% of adolescents with positive screen results on the NIAAA 2-question screen developed AUD 1 year later). Nonetheless, a brief intervention for youth with positive screen results may still be useful in reducing short-term risk for problematic substance use, even when most youth will not develop SUD or AUD in the future.31 

Finally, there were few middle school children who screened positive for problematic use of AOD in our study. Whether this low rate is a true reflection of use of AOD in the sample or of reluctance to report use in this age group, especially when parents are present, is a matter of speculation. Thus, screening middle school children may be less useful than screening high school students. However, early drinking and illicit drug use in adolescence is associated with SUDs involving AOD later in life.32,35 Consequently, it is important that middle school children with positive screen results on the CRAFFT receive further evaluation and potentially a brief intervention and follow-up, if indicated.

There were some notable study strengths. The study sample was large, and we used a diagnostic measure, the DISC, as the external validation measure. Additionally, we were able to assess the predictive validity of the CRAFFT up to 3 years, something that has not been reported to date. Our study also has several limitations. First, although we had a large sample size representing a diverse population, it was drawn from youth who were treated in a PED, and the results may not be applicable to the general population. Further study of children at risk, such as those transitioning from middle school to high school, is needed. Second, because of ED care interruptions, the accuracy of reporting may have been affected for some patients. Third, although electronic self-administration of the questions may have encouraged more honest reporting by the participants, we are not sure how those answers would differ if the participants were interviewed face-to-face. A study in which the authors investigated the relationship between different ways of administering the CRAFFT questionnaire (pen and paper, computer based, or verbal report) revealed that the highest likelihood of being honest was when the questionnaire was administered with pen and paper, whereas the next best form of administration was computer based.36 Fourth, there were only 2 middle school children who had a baseline CRAFFT score of ≥1 and SUD at the 1- or 2-year follow-up, so we cannot comment on the predictive validity of the CRAFFT in this age group. Fifth, we may have missed patients who were too sick to answer the survey questions. Sixth, the follow-up rates varied from 60% to 71% over the 3 years of follow-up, and participants who did not follow-up were more likely to be boys, African American, and Hispanic and had a slightly higher mean baseline CRAFFT score.

The CRAFFT has good concurrent validity for problematic substance use and SUD in patients in the PED, with value in predicting SUDs at up to a 3-year follow-up but with limited sensitivity. Future studies are needed to determine how the screen can be optimally used in the PED as well as how to develop and improve developmentally appropriate interventions in those identified as at risk.

Drs Shenoi and Linakis contributed to the design of the study, formulated the manuscript concept, and critically reviewed and edited the manuscript; Drs Bromberg, Chun, and Mello contributed to the design of the study and critically reviewed and edited the manuscript; Dr Casper and Ms Richards contributed to the design of the study, supervised the analyses, and reviewed and revised the manuscript; Dr Spirito contributed to the study design, formulated the manuscript concept, and drafted the initial manuscript; and all authors approved the final manuscript as submitted.

FUNDING: Supported in part by National Institute on Alcohol Abuse and Alcoholism grant 1R01AA021900 to Drs Spirito and Linakis. The project was conducted through the Pediatric Emergency Care Applied Research Network, which is supported in part by the Health Resources and Services Administration, the Maternal and Child Health Bureau, and the Emergency Medical Services for Children Network Development Demonstration Program under cooperative agreements U03MC00008, U03MC00001, U03MC00003, U03MC00006, U03MC00007, U03MC22684, and U03MC22685. This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, the Health Resources and Services Administration, the US Department of Health and Human Services, or the US Government. Funded by the National Institutes of Health (NIH).

Our efforts would not have been possible without the commitment of the investigators and research coordinators from the 16 participating PECARN sites. We thank the staff at these sites for their dedication and assistance throughout the study. We also thank the participants and their parents for participating in this study.

     
  • AOD

    alcohol and other drugs

  •  
  • AUD

    alcohol use disorder

  •  
  • CRAFFT

    Car, Relax, Alone, Forget, Friends, Trouble

  •  
  • DISC

    Diagnostic Interview Schedule for Children

  •  
  • DSM-5

    Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition

  •  
  • DSM-IV

    Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition

  •  
  • ED

    emergency department

  •  
  • IQR

    interquartile range

  •  
  • NIAAA

    National Institute on Alcohol Abuse and Alcoholism

  •  
  • PECARN

    Pediatric Emergency Care Applied Research Network

  •  
  • PED

    pediatric emergency department

  •  
  • ROC

    receiver operating characteristic

  •  
  • SUD

    substance use disorder

1
Miech
RA
,
Johnston
LD
,
O’Malley
PM
,
Bachman
JG
,
Schulenberg
JE
,
Patrick
ME
.
Monitoring the Future National Survey Results on Drug Use, 1975–2017, Volume I, Secondary School Students
.
Ann Arbor, MI
:
Institute for Social Research, The University of Michigan
;
2018
. Available at: http://monitoringthefuture.org/pubs.html#monographs. Accessed June 24, 2018
2
Substance Abuse and Mental Health Services Administration
.
Key Substance use and mental health indicators in the United States: results From the 2017 national survey on drug use and health
. Available at: https://www.samhsa.gov/data/sites/default/files/cbhsq-reports/NSDUHFFR2017/NSDUHFFR2017.htm. Accessed December 19, 2018
3
Harris
SK
,
Louis-Jacques
J
,
Knight
JR
.
Screening and brief intervention for alcohol and other abuse.
Adolesc Med State Art Rev
.
2014
;
25
(
1
):
126
156
[PubMed]
4
Levy
SJ
,
Kokotailo
PK
;
Committee on Substance Abuse
.
Substance use screening, brief intervention, and referral to treatment for pediatricians.
Pediatrics
.
2011
;
128
(
5
). Available at: www.pediatrics.org/cgi/content/full/128/5/e1330
5
Yuma-Guerrero
PJ
,
Lawson
KA
,
Velasquez
MM
,
von Sternberg
K
,
Maxson
T
,
Garcia
N
.
Screening, brief intervention, and referral for alcohol use in adolescents: a systematic review.
Pediatrics
.
2012
;
130
(
1
):
115
122
[PubMed]
6
Tanner-Smith
EE
,
Lipsey
MW
.
Brief alcohol interventions for adolescents and young adults: a systematic review and meta-analysis.
J Subst Abuse Treat
.
2015
;
51
:
1
18
[PubMed]
7
Knight
JR
,
Shrier
LA
,
Bravender
TD
,
Farrell
M
,
Vander Bilt
J
,
Shaffer
HJ
.
A new brief screen for adolescent substance abuse.
Arch Pediatr Adolesc Med
.
1999
;
153
(
6
):
591
596
[PubMed]
8
Knight
JR
,
Harris
SK
,
Sherritt
L
, et al
.
Prevalence of positive substance abuse screen results among adolescent primary care patients.
Arch Pediatr Adolesc Med
.
2007
;
161
(
11
):
1035
1041
[PubMed]
9
Knight
JR
,
Sherritt
L
,
Shrier
LA
,
Harris
SK
,
Chang
G
.
Validity of the CRAFFT substance abuse screening test among adolescent clinic patients.
Arch Pediatr Adolesc Med
.
2002
;
156
(
6
):
607
614
[PubMed]
10
Skogen
JC
,
Bøe
T
,
Knudsen
AK
,
Hysing
M
.
Psychometric properties and concurrent validity of the CRAFFT among Norwegian adolescents. Ung@hordaland, a population-based study.
Addict Behav
.
2013
;
38
(
10
):
2500
2505
[PubMed]
11
Cummins
LH
,
Chan
KK
,
Burns
KM
,
Blume
AW
,
Larimer
M
,
Marlatt
GA
.
Validity of the CRAFFT in American-Indian and Alaska-Native adolescents: screening for drug and alcohol risk.
J Stud Alcohol
.
2003
;
64
(
5
):
727
732
[PubMed]
12
Levy
S
,
Sherritt
L
,
Harris
SK
, et al
.
Test-retest reliability of adolescents’ self-report of substance use.
Alcohol Clin Exp Res
.
2004
;
28
(
8
):
1236
1241
[PubMed]
13
Subramaniam
M
,
Cheok
C
,
Verma
S
,
Wong
J
,
Chong
SA
.
Validity of a brief screening instrument-CRAFFT in a multiethnic Asian population.
Addict Behav
.
2010
;
35
(
12
):
1102
1104
[PubMed]
14
Dhalla
S
,
Zumbo
BD
,
Poole
G
.
A review of the psychometric properties of the CRAFFT instrument: 1999-2010.
Curr Drug Abuse Rev
.
2011
;
4
(
1
):
57
64
[PubMed]
15
American Psychiatric Association
. Substance Use Disorders. 
Diagnostic and Statistical Manual of Mental Disorders
. 5th ed.
Washington, DC
:
American Psychiatric Association
;
2013
:
483
485
16
Mitchell
SG
,
Kelly
SM
,
Gryczynski
J
, et al
.
The CRAFFT cut-points and DSM-5 criteria for alcohol and other drugs: a reevaluation and reexamination.
Subst Abus
.
2014
;
35
(
4
):
376
380
[PubMed]
17
Fisher
PW
,
Shaffer
D
,
Piacentini
JC
, et al
.
Sensitivity of the Diagnostic Interview Schedule for Children, 2nd edition (DISC-2.1) for specific diagnoses of children and adolescents.
J Am Acad Child Adolesc Psychiatry
.
1993
;
32
(
3
):
666
673
.
18
Pediatric Emergency Care Applied Research Network
.
The Pediatric Emergency Care Applied Research Network (PECARN): rationale, development, and first steps.
Acad Emerg Med
.
2003
;
10
(
6
):
661
668
[PubMed]
19
Bromberg
JR
,
Spirito
A
,
Chun
T
, et al;
Pediatric Emergency Care Applied Research Network
.
Methodology and demographics of a brief adolescent alcohol screen validation study [published online ahead of print July 3, 2017].
Pediatr Emerg Care
. doi:
[PubMed]
20
Shaffer
D
,
Fisher
P
,
Lucas
CP
,
Dulcan
MK
,
Schwab-Stone
ME
.
NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses.
J Am Acad Child Adolesc Psychiatry
.
2000
;
39
(
1
):
28
38
[PubMed]
21
Chung
T
,
Colby
SM
,
Barnett
NP
,
Monti
PM
.
Alcohol use disorders identification test: factor structure in an adolescent emergency department sample.
Alcohol Clin Exp Res
.
2002
;
26
(
2
):
223
231
[PubMed]
22
Chung
T
,
Colby
SM
,
O’Leary
TA
,
Barnett
NP
,
Monti
PM
.
Screening for cannabis use disorders in an adolescent emergency department sample.
Drug Alcohol Depend
.
2003
;
70
(
2
):
177
186
[PubMed]
23
Chung
T
,
Colby
SM
,
Barnett
NP
,
Rohsenow
DJ
,
Spirito
A
,
Monti
PM
.
Screening adolescents for problem drinking: performance of brief screens against DSM-IV alcohol diagnoses.
J Stud Alcohol
.
2000
;
61
(
4
):
579
587
[PubMed]
24
Rojas
NL
,
Sherrit
L
,
Harris
S
,
Knight
JR
.
The role of parental consent in adolescent substance use research.
J Adolesc Health
.
2008
;
42
(
2
):
192
197
[PubMed]
25
Subramaniam
GA
,
Volkow
ND
.
Substance misuse among adolescents: to screen or not to screen [published correction appears in JAMA Pediatr. 2014;168(10):971]?
JAMA Pediatr
.
2014
;
168
(
9
):
798
799
[PubMed]
26
Newton
AS
,
Gokiert
R
,
Mabood
N
, et al
.
Instruments to detect alcohol and other drug misuse in the emergency department: a systematic review.
Pediatrics
.
2011
;
128
(
1
). Available at: www.pediatrics.org/cgi/content/full/128/1/e180
[PubMed]
27
Levy
S
,
Weiss
R
,
Sherritt
L
, et al
.
An electronic screen for triaging adolescent substance use by risk levels.
JAMA Pediatr
.
2014
;
168
(
9
):
822
828
[PubMed]
28
Kelly
SM
,
Gryczynski
J
,
Mitchell
SG
,
Kirk
A
,
O’Grady
KE
,
Schwartz
RP
.
Validity of brief screening instrument for adolescent tobacco, alcohol, and drug use.
Pediatrics
.
2014
;
133
(
5
):
819
826
[PubMed]
29
Johnston
LD
,
Miech
RA
,
O’Malley
PM
,
Bachman
JG
,
Schulenberg
JE
,
Patrick
ME
.
Monitoring the Future National Survey Results on Drug Use: 1975-2017: Overview, Key Findings on Adolescent Drug Use
.
Ann Arbor, MI
:
Institute for Social Research, The University of Michigan
;
2018
30
Linakis
JG
,
Bromberg
JR
,
Casper
TC
, et al;
Pediatric Emergency Care Applied Research Network
.
Predictive validity of a 2-question alcohol screen at 1-, 2- and 3-year follow-up.
Pediatrics
.
2019
;
143
(
3
):
e20182001
[PubMed]
31
Hadland
SE
,
Knight
JR
,
Harris
SK
.
Alcohol use disorder: a pediatric-onset condition needing early detection and intervention.
Pediatrics
.
2019
;
143
(
3
):
e20183654
[PubMed]
32
McCabe
SE
,
West
BT
,
Morales
M
,
Cranford
JA
,
Boyd
CJ
.
Does early onset of non-medical use of prescription drugs predict subsequent prescription drug abuse and dependence? Results from a national study.
Addiction
.
2007
;
102
(
12
):
1920
1930
[PubMed]
33
Lynskey
MT
,
Heath
AC
,
Bucholz
KK
, et al
.
Escalation of drug use in early-onset cannabis users vs co-twin controls.
JAMA
.
2003
;
289
(
4
):
427
433
[PubMed]
34
Hingson
RW
,
Heeren
T
,
Winter
MR
.
Age at drinking onset and alcohol dependence: age at onset, duration, and severity.
Arch Pediatr Adolesc Med
.
2006
;
160
(
7
):
739
746
[PubMed]
35
Miech
R
,
Johnston
L
,
O’Malley
PM
,
Keyes
KM
,
Heard
K
.
Prescription opioids in adolescence and future opioid misuse.
Pediatrics
.
2015
;
136
(
5
). Available at: www.pediatrics.org/cgi/content/full/136/5/e1169
[PubMed]
36
Knight
JR
,
Harris
SK
,
Sherritt
L
, et al
.
Adolescents’ preference for substance abuse screening in primary care practice.
Subst Abus
.
2007
;
28
(
4
):
107
117
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data