Persistent disparities exist in early identification of autism spectrum disorder (ASD) among children from low-income families who are racial and/or ethnic minorities and where English is not the primary language. Parental literacy and level of maternal education may contribute to disparities. The Developmental Check-In (DCI) is a visually based ASD screening tool created to reduce literacy demands and to be easily administered and scored across settings. In a previous study, the DCI showed acceptable discriminative ability between ASD versus non-ASD in a young, underserved sample at high-risk for ASD. In this study, we tested the DCI among an unselected, general sample of young underserved children.
Six hundred twenty-four children ages 24 to 60 months were recruited through Head Start and Early Head Start. Parents completed the DCI, Modified Checklist for Autism in Toddlers, Revised with Follow-Up, and Social Communication Questionnaire. Children scoring positive on any measure received evaluation for ASD. Those screening negative on both Modified Checklist for Autism in Toddlers, Revised with Follow-Up and Social Communication Questionnaire were considered non-ASD.
Parents were primarily Hispanic, reported high school education or less, and had public or no insurance. The DCI demonstrated good discriminative power (area under the curve = 0.80), performing well across all age groups, genders, levels of maternal education, primary language, and included ethnic and racial groups. Item-level analyses indicated that 24 of 26 DCI items discriminated ASD from non-ASD.
The DCI is a promising ASD screening tool for young, underserved children and may be of particular value in screening for ASD for those with low literacy levels or with limited English proficiency.
Early detection of autism spectrum disorder (ASD) helps link children to intervention that improves functional outcomes. Existing ASD screening tools demonstrate questionable accuracy among young children from low-income, ethnic and racial minority families, which may contribute to disparities in service access.
The Developmental Check-In, a primarily pictorial ASD screening tool, demonstrates promise in identifying ASD risk among young underserved children in the general population. Screening with the Developmental Check-In may potentially reduce disparities in identification of ASD related to ethnicity and racial background.
Early detection and diagnosis of autism spectrum disorder (ASD) is needed to link children to early intervention, which can lead to improved outcomes.1,2 Children with ASD from poor, racial, or ethnic minority groups are identified later, on average, than other children3–7 and often are misdiagnosed.7,8 When they receive diagnoses at young ages, they have more severe symptoms than white children, suggesting that those less severely impaired often are missed.9–11 These disparities are compounded among Latino individuals with limited English proficiency.12
Although the American Academy of Pediatrics recommends using a standardized ASD screening tool at 18 and 24 months,13,14 existing tools may not accurately identify autism in underserved populations.15 For example, the Modified Checklist for Autism in Toddlers, Revised (M-CHAT-R)16 overidentifies children of low-income, racial minority parents or parents with less education.17–19 Even with the recommended follow-up interview designed to reduce false positives, positive predictive value (PPV) is lower in children of color and those from lower-income households.20 Although Dai et al21 found no racial differences in PPV for the Modified Checklist for Autism in Toddlers, Revised, with Follow-Up (M-CHAT-R/F) interview, PPV was only 0.40 to 0.42. Another common ASD screening tool, the Social Communication Questionnaire (SCQ),22 has decreased sensitivity among children with Black or multiracial mothers or those with lower income or education.23,24
Literacy demands of ASD screening tools may influence their accuracy. Parent response on these tools often is inconsistent with parent descriptions given during follow-up interviews, particularly for those with limited English proficiency or formal education, who express confusion about the meaning of written items.15 Pictorial tools may reduce literacy demands, although few have been created, with most designed for use among non–North American populations25,26 or focusing only on select features of ASD.27
Although the average age of autism diagnosis in the United States is 4 years,28 many screening tools are validated on samples of children <3. Among minority and poor children, in whom symptoms often are missed or misinterpreted, having screening tools that are effective for preschool-aged children may be particularly important.
In response to these concerns, we created the Developmental Check-In (DCI), a visually based ASD screening tool. We conducted a preliminary validation study with 376 children ages 24 to 60 months from racial or ethnic minority, low-income, or limited English proficiency families.15 Of the sample, 288 were considered at-risk because they had been referred for ASD evaluation. An additional 88 children from community day care centers were also included. The DCI demonstrated utility in discriminating ASD from non-ASD (area under the curve [AUC] = 0.75), and 26 of the 28 DCI items predicted ASD status. The current study extends validation of the DCI in an unselected sample of young underserved children.
Methods
The Rutgers Biomedical Health Sciences Institutional Review Board approved this study.
Procedure
We recruited children ages 24 to 60 months from Early Head Start and Head Start programs in 4 low-income communities in New Jersey. Staff distributed invitation packets to all parents, consisting of the DCI, M-CHAT-R, SCQ, short demographic form, and consent form. Both the M-CHAT-R and SCQ were included because of their common use in clinical care and research. Parents reviewed and signed the consent form to participate. If parents consented, Head Start staff distributed the DCI, M-CHAT-R, and SCQ to teachers. Parents who completed the screening tools were entered into a drawing for a $100 gift card.
Research staff collected completed packets from sites weekly, scored screeners, and contacted parents to complete missing data. If a child screened at medium risk on the M-CHAT-R (completed by parent or teacher), research staff called (repeatedly) to conduct the follow-up interview in the parent’s primary language (English or Spanish). Those who continued to screen positive after the interview were considered positive. Clinical evaluations were offered to any child who screened positive on the parent- or teacher-completed M-CHAT-R/F, SCQ, or DCI. This allowed for evaluation of children who would have not been considered at-risk based only on results of the parent-completed M-CHAT-R/F and SCQ. Teacher screening results were used to increase the pool of children offered clinical evaluations but were not otherwise used in analyses. Although the DCI was scored and used to identify children eligible for clinical evaluation, we did not use it in any way to determine ASD status.
ASD Screening Measures
DCI
Development of the DCI is described elsewhere.15 Created in English and Spanish, it consists of 26 photographs grouped by domain: communication (4 items), play (6 items), social (8 items), and behavior (8 items). Each photograph has a brief descriptor. Cutoff scores based on age of child suggested by earlier work15 were used: cutoff was ≥7 for children 24 to 35 months, ≥4 for children 36 to 47 months, and ≥3 for children 48 to 60 months. See Fig 1 for a sample item. The same cutoff scores were used with parent and teacher-completed versions.
M-CHAT-R/F
The M-CHAT-R16,29 is a 20-item, parent-report ASD screening tool used to examine children’s development. It has been validated for children ages 16 to 48 months.30 A score of 0 to 2 is considered low risk, 3 to 7 is considered medium risk, and ≥8 is considered high risk. A follow-up interview is given for scores between 3 and 7. In a low-risk sample of toddlers, the MCHAT-R/F AUC was 0.907, and PPV was 0.45 to 0.51.29
SCQ
The SCQ22 is a 40-item parent report ASD screening measure, originally designed for children ≥48 months, that has demonstrated acceptable rates of sensitivity and specificity.22 Researchers in previous studies extended the age range down to 24 months, with AUCs ranging from 0.62 to 0.90.23,24,30–34 In this study, we used the current form of the SCQ and a cutoff score of 15, as recommended by the SCQ manual.22
Participants
Screening packets were distributed for 1648 children; 747 parents returned a packet. After excluding subjects with missing data, 637 subjects remained in the sample. In Figure 2, we depict participant inclusion.
Determination of ASD Status
Developmental Evaluation Measures
ADOS-2
The ADOS-2, is a semistructured, play-based assessment for ASD with standardized administration and scoring schema. Sensitivity for modules 1 and 2 ranges from 0.77 to 0.98, and specificity ranges from 0.19 to 0.94 depending on language level, age, and diagnosis.35 The measure of interest used in analyses was overall algorithm classification (autism and autism spectrum were combined as “ASD”). This algorithm classification also helped inform clinical diagnosis.
MSEL
MSEL is a comprehensive assessment of development normed for children ages birth to 5 years, 8 months that was administered to measure cognitive and language functioning. For this study, standard scores for all scales except gross motor were the measures of interest used to inform clinical diagnosis.
Clinical Evaluation Procedure
We offered clinical evaluations at Head Start locations for children screening positive on any parent or teacher MCHAT-R/F, SCQ, or DCI. Decisions about ASD status were based on results of clinical evaluation (ASD or non-ASD) or on low-risk results on both the M-CHAT-R/F and SCQ (presumptive non-ASD), as described below.
ASD status was determined on the basis of the child’s medical history, behavioral observations, and ADOS-2, which informed the clinical judgment of the experienced, licensed evaluator who was research reliable on the ADOS-2 and blind to screening results. Those completing the evaluation received a $50 gift card. Within 2 to 3 weeks of the research appointment, a full evaluation report with recommendations for follow-up intervention and/or further testing was provided regardless of ASD status. Children who did not meet ASD criteria, regardless of other diagnoses, were assigned a status of non-ASD.
To reduce risk of false-positives, we took a conservative approach to assigning ASD status and excluded from analysis children who screened positive on multiple screening tools but whose parents declined clinical evaluation (n = 13).
Children who qualified for evaluation by screening positive on the DCI or any teacher-completed screening tool, but who screened negative on the parent-completed MCHAT-R/F and SCQ, were assigned presumptive non-ASD status if their parents declined clinical evaluation (n = 43). This strategy is more data-driven than other studies in which researchers assigned ASD status on the basis of presence or absence of diagnosis in a health chart26 or parent-reported ASD diagnosis38,39 and follows other validation procedures in which multiple methods were used to reduce risk of false-negatives.29 This resulted in a sample of 624 children for data analysis.
Data Analysis
We assessed the performance of the DCI in discriminating between children with ASD and without ASD using the same methods described in our previous DCI validation study.15 Total DCI score and subscores for speech, play, social engagement, and behavior were compared between groups by using the Mann–Whitney U test. Given the skewed distribution of scores, we used nonparametric tests. Using logistic regression, we performed item analysis to examine discriminant ability of each item. We estimated the odds of scoring at-risk on each item for children with ASD compared with children without ASD.
We performed a receiver operator characteristic (ROC) AUC analysis to examine discriminant ability of the DCI in distinguishing ASD from non-ASD. We estimated optimal cutoff points for the DCI using the Youden and Liu indices.40–42 We report specificity and sensitivity of the DCI at the optimal cutoff points. An AUC of 0.5 indicates no discrimination above chance, and an AUC of 1.0 indicates perfect discrimination. In general, an AUC of 0.9 to 1.0 indicates excellent, AUC of 0.8 to 0.9 indicates good, AUC of 0.7 to 0.8 indicates fair, and AUC of 0.6 to 0.7 indicates poor discriminative ability. Discrimination is assumed to be useful if AUC ≥0.75.43,44
Pepe et al proposed guidelines45–47 on the ROC-regression methods to account for confounders and effect modifiers. Following these guidelines, we introduced covariate adjustment by regression model in ROC analysis to account for potential confounding.46,48,49 We explored associations between all measured covariates, diagnostic status, and DCI total score. If any significant associations were observed, we adjusted estimated ROC curves for these covariates. We reported the AUC for both unadjusted and adjusted ROC curves. To determine if covariate-specific ROC analysis was needed, we examined the effect of the covariates on the estimated ROC curve.45–47 For covariates identified as potential confounders, we estimated covariate-specific cutoff points.50,51 We also compared discriminative ability of the DCI among different demographic groups.
All procedures performed in this study involving human participants were in accordance with the ethical standards of the institution and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Results
In Table 1, we depict demographic characteristics of the sample. Average age was 49 months. Most children (66%) were Hispanic and from families in which English was not the primary language spoken at home (65%). Most parents reported high school or less education (65%). Most families (86%) were Medicaid enrolled or had no insurance.
. | ASD (n = 52) . | Non-ASD (n = 572) . | P . |
---|---|---|---|
Child | |||
Mean age in mo (SD) | 46 (6.71) | 49 (7.28) | .001 |
Age range in mo | |||
24–35 | 6 | 4 | .40 |
36–47 | 50 | 38 | .10 |
48–60 | 44 | 58 | .05 |
Sex | |||
Male | 87 | 48 | <.001 |
Race and ethnicity | |||
White, not Hispanic | 8 | 7 | .81 |
Black, not Hispanic | 13 | 24 | .08 |
Hispanic | 73 | 66 | .28 |
Other race, not Hispanic | 6 | 3 | .16 |
Parent | |||
Primary language | |||
Not English | 67 | 65 | .80 |
Education level | |||
High school or less education | 67 | 65 | .73 |
Insurance coverage | |||
Medicaid | 88 | 86 | .65 |
No health insurance | 8 | 7 | .88 |
Private insurance | 4 | 7 | .59 |
Respondent | |||
Mother | 94 | 86 | .11 |
. | ASD (n = 52) . | Non-ASD (n = 572) . | P . |
---|---|---|---|
Child | |||
Mean age in mo (SD) | 46 (6.71) | 49 (7.28) | .001 |
Age range in mo | |||
24–35 | 6 | 4 | .40 |
36–47 | 50 | 38 | .10 |
48–60 | 44 | 58 | .05 |
Sex | |||
Male | 87 | 48 | <.001 |
Race and ethnicity | |||
White, not Hispanic | 8 | 7 | .81 |
Black, not Hispanic | 13 | 24 | .08 |
Hispanic | 73 | 66 | .28 |
Other race, not Hispanic | 6 | 3 | .16 |
Parent | |||
Primary language | |||
Not English | 67 | 65 | .80 |
Education level | |||
High school or less education | 67 | 65 | .73 |
Insurance coverage | |||
Medicaid | 88 | 86 | .65 |
No health insurance | 8 | 7 | .88 |
Private insurance | 4 | 7 | .59 |
Respondent | |||
Mother | 94 | 86 | .11 |
Sample mean and SD for age in months, percentage in each demographic category, and P values for the test of difference in variables between ASD and non-ASD groups.
As shown in Table 1, children with ASD were younger (46 months versus 49 months) and more likely to be male than children without ASD. There were no statistically significant differences between ASD and non-ASD children in race and ethnicity, primary language at home, parent education, health insurance, or survey respondent.
Children with ASD had significantly higher scores, on average, in each category of the DCI than children without ASD (eg, total DCI score 6.67 for ASD versus 1.94 for non-ASD, P < .001) (Table 2). Discriminative performance of each DCI item was compared by ASD status (Table 3). Parents of children subsequently diagnosed with ASD were significantly more likely to endorse all but 2 of the 26 DCI items. “Says 1 word” and “responds to name” did not discriminate ASD from non-ASD.
. | ASD (n = 52) . | Non-ASD (n = 572) . | Pa . |
---|---|---|---|
Total DCI score (total 28 items) (SD) | 6.67 (4.95) | 1.94 (2.32) | <.001 |
DCI, speech score (toal 4 items) (SD) | 0.62 (0.97) | 0.19 (0.57) | <.001 |
DCI, play score (total 6 items) (SD) | 1.73 (1.92) | 0.40 (0.99) | <.001 |
DCI, social engagement score (total 8 items) (SD) | 1.23 (1.79) | 0.17 (0.52) | <.001 |
DCI, behavior score (total 8 items) (SD) | 3.09 (1.99) | 1.19 (1.51) | <.001 |
. | ASD (n = 52) . | Non-ASD (n = 572) . | Pa . |
---|---|---|---|
Total DCI score (total 28 items) (SD) | 6.67 (4.95) | 1.94 (2.32) | <.001 |
DCI, speech score (toal 4 items) (SD) | 0.62 (0.97) | 0.19 (0.57) | <.001 |
DCI, play score (total 6 items) (SD) | 1.73 (1.92) | 0.40 (0.99) | <.001 |
DCI, social engagement score (total 8 items) (SD) | 1.23 (1.79) | 0.17 (0.52) | <.001 |
DCI, behavior score (total 8 items) (SD) | 3.09 (1.99) | 1.19 (1.51) | <.001 |
For Mann–Whitney U Test statistic for comparison of instrument scores among ASD and non-ASD children.
. | ASD (n = 52), % . | Non-ASD (n = 572), % . | Odds Ratio . | P . |
---|---|---|---|---|
Items about speech | ||||
Asks for help | 0.15 | 0.03 | 7.24 | <.001 |
Says 1 word | 0.09 | 0.07 | 1.53 | .39 |
Says 2 words | 0.13 | 0.06 | 2.62 | .003 |
Says 3 words | 0.23 | 0.04 | 6.85 | <.001 |
Items about play | ||||
Plays peek-a-boo | 0.25 | 0.04 | 8.33 | <.001 |
Feeds toy | 0.33 | 0.12 | 3.60 | <.001 |
Imitates | 0.23 | 0.03 | 11.95 | <.001 |
Plays pretend | 0.31 | 0.08 | 4.96 | <.001 |
Plays make believe | 0.25 | 0.06 | 6.62 | <.001 |
Plays pretend with children | 0.37 | 0.08 | 6.74 | <.001 |
Items about social engagement | ||||
Smiles back | 0.10 | 0.002 | 60.74 | <.001 |
Makes eye contact | 0.15 | 0.01 | 14.67 | <.001 |
Points | 0.06 | 0.01 | 4.31 | .04 |
Shows toys | 0.08 | 0.00 | 1.00 | |
Comforts others | 0.13 | 0.02 | 8.74 | <.001 |
Shares interest | 0.21 | 0.02 | 16.78 | <.001 |
Shares toy | 0.31 | 0.09 | 4.44 | <.001 |
Plays with children | 0.19 | 0.02 | 12.14 | <.001 |
Items about behavior | ||||
Walks on toes often | 0.37 | 0.19 | 2.40 | .01 |
Bothered by noise | 0.44 | 0.20 | 3.22 | <.001 |
Tantrums often | 0.65 | 0.27 | 5.08 | <.001 |
Avoids other children | 0.35 | 0.09 | 5.40 | <.001 |
Avoids eye contact | 0.35 | 0.08 | 6.35 | <.001 |
Spins wheels over and over | 0.50 | 0.20 | 3.97 | <.001 |
Flaps hands a lot | 0.40 | 0.12 | 5.02 | <.001 |
Responds to name | 0.04 | 0.04 | 0.95 | .90 |
. | ASD (n = 52), % . | Non-ASD (n = 572), % . | Odds Ratio . | P . |
---|---|---|---|---|
Items about speech | ||||
Asks for help | 0.15 | 0.03 | 7.24 | <.001 |
Says 1 word | 0.09 | 0.07 | 1.53 | .39 |
Says 2 words | 0.13 | 0.06 | 2.62 | .003 |
Says 3 words | 0.23 | 0.04 | 6.85 | <.001 |
Items about play | ||||
Plays peek-a-boo | 0.25 | 0.04 | 8.33 | <.001 |
Feeds toy | 0.33 | 0.12 | 3.60 | <.001 |
Imitates | 0.23 | 0.03 | 11.95 | <.001 |
Plays pretend | 0.31 | 0.08 | 4.96 | <.001 |
Plays make believe | 0.25 | 0.06 | 6.62 | <.001 |
Plays pretend with children | 0.37 | 0.08 | 6.74 | <.001 |
Items about social engagement | ||||
Smiles back | 0.10 | 0.002 | 60.74 | <.001 |
Makes eye contact | 0.15 | 0.01 | 14.67 | <.001 |
Points | 0.06 | 0.01 | 4.31 | .04 |
Shows toys | 0.08 | 0.00 | 1.00 | |
Comforts others | 0.13 | 0.02 | 8.74 | <.001 |
Shares interest | 0.21 | 0.02 | 16.78 | <.001 |
Shares toy | 0.31 | 0.09 | 4.44 | <.001 |
Plays with children | 0.19 | 0.02 | 12.14 | <.001 |
Items about behavior | ||||
Walks on toes often | 0.37 | 0.19 | 2.40 | .01 |
Bothered by noise | 0.44 | 0.20 | 3.22 | <.001 |
Tantrums often | 0.65 | 0.27 | 5.08 | <.001 |
Avoids other children | 0.35 | 0.09 | 5.40 | <.001 |
Avoids eye contact | 0.35 | 0.08 | 6.35 | <.001 |
Spins wheels over and over | 0.50 | 0.20 | 3.97 | <.001 |
Flaps hands a lot | 0.40 | 0.12 | 5.02 | <.001 |
Responds to name | 0.04 | 0.04 | 0.95 | .90 |
Stacks blocks, lines up toys removed in phase 2.
Unadjusted ROC analysis of the discriminative power of the DCI yielded an AUC of 0.84 (95% confidence interval [CI]: 0.78–0.89). In Table 4, we provide a detailed report of sensitivity and specificity of the DCI at each potential cutoff point. The optimal cutoff point across all ages, as estimated by Liu and Youden indices, was 3.5.
Cutpoint . | Sensitivity, % . | Specificity, % . | Correctly Classified, % . | LR+ . | LR− . |
---|---|---|---|---|---|
(≥0) | 100.0 | 0.0 | 8.3 | 1 | — |
(≥1) | 98.1 | 32.9 | 38.3 | 1.46 | 0.06 |
(≥2) | 88.5 | 52.5 | 55.5 | 1.86 | 0.22 |
(≥3) | 80.8 | 73.1 | 73.7 | 3.00 | 0.26 |
(≥4) | 71.2 | 83.4 | 82.4 | 4.28 | 0.35 |
(≥5) | 57.7 | 86.9 | 84.5 | 4.40 | 0.49 |
(≥6) | 48.1 | 91.1 | 87.5 | 5.39 | 0.57 |
(≥7) | 44.2 | 94.4 | 90.2 | 7.91 | 0.59 |
(≥8) | 32.7 | 97.4 | 92.0 | 12.47 | 0.69 |
(≥9) | 26.9 | 98.1 | 92.2 | 14.00 | 0.75 |
(≥10) | 23.1 | 99.1 | 92.8 | 26.40 | 0.78 |
(≥11) | 19.2 | 99.3 | 92.6 | 27.50 | 0.81 |
(≥12) | 15.4 | 99.3 | 92.3 | 22.00 | 0.85 |
(≥13) | 11.5 | 99.5 | 92.2 | 22.00 | 0.89 |
(≥15) | 11.5 | 99.7 | 92.3 | 33.00 | 0.89 |
(≥16) | 9.6 | 99.8 | 92.3 | 55.00 | 0.91 |
(≥18) | 7.7 | 99.8 | 92.2 | 44.00 | 0.92 |
(≥19) | 1.9 | 100.0 | 91.8 | — | 0.98 |
(>19) | 0.0 | 100.0 | 91.7 | — | 1 |
Cutpoint . | Sensitivity, % . | Specificity, % . | Correctly Classified, % . | LR+ . | LR− . |
---|---|---|---|---|---|
(≥0) | 100.0 | 0.0 | 8.3 | 1 | — |
(≥1) | 98.1 | 32.9 | 38.3 | 1.46 | 0.06 |
(≥2) | 88.5 | 52.5 | 55.5 | 1.86 | 0.22 |
(≥3) | 80.8 | 73.1 | 73.7 | 3.00 | 0.26 |
(≥4) | 71.2 | 83.4 | 82.4 | 4.28 | 0.35 |
(≥5) | 57.7 | 86.9 | 84.5 | 4.40 | 0.49 |
(≥6) | 48.1 | 91.1 | 87.5 | 5.39 | 0.57 |
(≥7) | 44.2 | 94.4 | 90.2 | 7.91 | 0.59 |
(≥8) | 32.7 | 97.4 | 92.0 | 12.47 | 0.69 |
(≥9) | 26.9 | 98.1 | 92.2 | 14.00 | 0.75 |
(≥10) | 23.1 | 99.1 | 92.8 | 26.40 | 0.78 |
(≥11) | 19.2 | 99.3 | 92.6 | 27.50 | 0.81 |
(≥12) | 15.4 | 99.3 | 92.3 | 22.00 | 0.85 |
(≥13) | 11.5 | 99.5 | 92.2 | 22.00 | 0.89 |
(≥15) | 11.5 | 99.7 | 92.3 | 33.00 | 0.89 |
(≥16) | 9.6 | 99.8 | 92.3 | 55.00 | 0.91 |
(≥18) | 7.7 | 99.8 | 92.2 | 44.00 | 0.92 |
(≥19) | 1.9 | 100.0 | 91.8 | — | 0.98 |
(>19) | 0.0 | 100.0 | 91.7 | — | 1 |
Number of observations: 624, ROC area: 0.84, SE: 0.029, 95% CI: 0.78–0.89, optimal cutoff point: 3.5. LR +, positive likelihood ratio; LR−, negative likelihood ratio; —, not applicable.
In Table 5, we report ROC estimation results adjusted for covariates, and in Fig 3, we depict the corresponding ROC curve. The estimated AUC is 0.80 (95% CI: 0.71–0.88), slightly lower than the unadjusted AUC. The significant coefficients for sex and parental education suggest that boys and children with low parental education have higher DCI scores, controlling for other covariates. Estimated cutoff points were girls: 2.5, boys: 3.5, low parent education: 3.5, and higher parent education: 2.5.
Total DCI score . | Coefficient . | P . |
---|---|---|
Child | ||
Age | −0.02 | .16 |
Male | 0.71 | <.001 |
Race and ethnicity | ||
Black, not Hispanic | 0.78 | .15 |
Hispanic | 0.28 | .56 |
Other race, not Hispanic | 0.87 | .25 |
Parent | ||
Primary language | ||
English not primary language | 0.52 | .13 |
Education Level | ||
High school or less education | 0.66 | .01 |
Insurance coverage | ||
Medicaid coverage | 0.05 | .92 |
No health insurance coverage | 1.05 | .09 |
Respondent | ||
Mother | −0.16 | .66 |
Total DCI score . | Coefficient . | P . |
---|---|---|
Child | ||
Age | −0.02 | .16 |
Male | 0.71 | <.001 |
Race and ethnicity | ||
Black, not Hispanic | 0.78 | .15 |
Hispanic | 0.28 | .56 |
Other race, not Hispanic | 0.87 | .25 |
Parent | ||
Primary language | ||
English not primary language | 0.52 | .13 |
Education Level | ||
High school or less education | 0.66 | .01 |
Insurance coverage | ||
Medicaid coverage | 0.05 | .92 |
No health insurance coverage | 1.05 | .09 |
Respondent | ||
Mother | −0.16 | .66 |
Site indicators are included in the estimating equation. Reference categories are white-non-Hispanic, English primary language at home, parent with post–high school education, private insurance coverage, respondent is not mother. Observed coefficient: 0.80; bootstrap SE; 0.0422, 95% CI: 0.714–0.879.
The DCI discriminated ASD from non-ASD better among children with private insurance (AUC = 1.00) than among those with Medicaid (AUC = 0.83, 95% CI: 0.77–0.89) or with no insurance (AUC = 0.83, 95% CI: 0.69–0.97) (Supplemental Table 6). This result should be interpreted cautiously because the sample with private insurance was small (n = 39). There were no statistically significant differences in the discriminative power of the DCI based on ethnicity, parent education, or language spoken at home.
Discussion
The DCI demonstrates good ability (AUC = 0.80) to discriminate ASD from non-ASD across all age groups in this unselected sample of young children from families who were primarily Hispanic, spoke Spanish as a primary language, were insured through Medicaid or not insured, and had a maternal education of high school or below. Although the gap in diagnoses of autism has been narrowing between Black and white young children, Hispanic children are still diagnosed with ASD at a lower rate.3 The DCI holds promise for addressing disparity in age of diagnosis among Hispanic children.7 The DCI has good discriminative power for children between 36 and 47 months of age, a group for which most common autism screening tools are not validated. It is noteworthy that in this sample, the proportion identified with ASD was 8%, much higher than the estimated ASD prevalence by the Centers for Disease Control and Prevention.28
The DCI had an unadjusted AUC of 0.84. This is lower than the AUC reported in the M-CHAT-R/F validation study29 (AUC = 0.90) and higher than most AUC values reported in SCQ validation studies (ranges from 0.62 to 0.90).34 However, M-CHAT-R/F and SCQ validation studies typically report lower tool performance among low socioeconomic status, parent education, and minority race and ethnicity groups. This study was solely composed of an underserved sample, suggesting that the DCI holds promise for identifying ASD in groups in which other screening tools may be less predictive. Direct comparison of performance between the DCI, MCHAT-R/F, and SCQ is not possible in this full sample because clinical evaluation and subsequent diagnosis were not obtained for all subjects. Comparison of the 3 tools in the subsample who received clinical evaluation is beyond the focus of the current study.
It was not feasible to conduct full evaluations of every screened negative case to confirm non-ASD status. To reduce false-negatives, we required subjects to screen negative on 2 different validated ASD screening tools (MCHAT-R/F and SCQ) to be presumed non-ASD. To increase potential to find true-positives, at-risk score on any of the 3 screening tools (M-CHAT-R/F, SCQ, or DCI) resulted in offering a clinical evaluation. The design of this study provided multiple pathways for case finding, which is a more-robust strategy than in other ASD screening tool validation studies.18,21,29
All but 2 DCI items discriminated ASD from non-ASD: “says 1 word” and “responds to name.” It is surprising that “responds to name” does not discriminate because it is often considered a key indicator of social communication. Informal query with parents suggests that many endorse this item if their child ever responds to name, rather than basing their response on the child’s typical performance. Providers may want to assess this skill directly rather than relying on parent report.
The DCI performed at least as well in this unselected sample as with a high-risk sample.15 Similarities and differences between the at-risk and unselected samples are noted. In both samples, “smiles back” and “shares interest” most discriminated ASD from non-ASD. However, in the unselected sample, “imitates,” “makes eye contact,” and “plays with children” also discriminated well. “Points” was the strongest discriminator item in the high-risk sample (odds ratio = 31.7, P < .001) but not in this unselected sample (odds ratio = 4.31, P = .04). These findings underscore the importance of assessing multiple categories of behavior and social communication when screening, even when demographic characteristics are similar.
Researchers in future studies should clinically evaluate all children, regardless of screening status, a design that would allow direct comparison of the DCI to the MCHAT-R/F and SCQ. Researchers should assess parental literacy, increase sampling of children <24 months, and test DCI performance in other samples. It would also be worthwhile to identify optimal methods of administering the DCI (eg, electronically, paper, sent home, in waiting room).
Although we presented empirical cutoff scores that maximize sensitivity and specificity in our sample, selection of cutoffs in practice should be informed by clinical aim. Thus, lower cutoffs may be chosen to ensure that no children at-risk for ASD are missed, even if specificity is reduced. Different cutoffs may also be important among different demographic groups.
The DCI is intended to be a broader ASD screener than other pictorial screeners.27 As a visually based ASD screener, the DCI may be of particular value in screening for ASD among low literacy or limited English proficiency groups in the United States. It is not known how the DCI would perform outside the United States.
This study has several limitations. Although the DCI has low literacy demands, parental literacy was not assessed. Because participation was voluntary, there may be some selection bias. It is not known if any of the children in this study were related to each other or had a family member already diagnosed with ASD, which might have had implications for a child’s ASD risk status or a parent’s knowledge of ASD and screening responses. It is also possible that this sample does not fully represent all underserved young children. Although presumptive non-ASD status was assigned on the basis of low-risk screening results on both the M-CHAT-R/F and SCQ, a full clinical evaluation was not conducted for this group. Some children who screened negative on both measures may have had ASD. A larger-scale study may have yielded sufficient power to more fully determine impact of ethnicity, parent education, or language spoken at home on DCI performance.
Despite these limitations, there are important implications related to these findings. Results of initial testing of the DCI are promising. With the DCI, we offer an ASD screening tool that appears to work well among underserved families and addresses limitations of other, validated autism screeners. As such, the DCI holds potential to help reduce disparities in identification of ASD.
Acknowledgments
We thank Marilyn Lopez and Natalia Gonzalez for their involvement in data collection and review of the study design. Thank you also to Dr Rhiannon Luyster for assistance in conceptualizing the study design.
Dr Harris conceptualized and designed the study, collected data, and drafted the initial manuscript; Dr Coffield conceptualized and designed the study, supervised data collection, and drafted the initial manuscript; Dr Janvier conceptualized and designed the study, conceived of and designed the instrument, and collected data; Dr Mandell conceptualized and designed the study and reviewed data analysis; Dr Cidav conducted the data analyses and drafted the data analysis section of the manuscript; and all authors reviewed and revised the manuscript and approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: Funded in part by a grant from the New Jersey Department of Health, Governor’s Council for Medical Research and Treatment of Autism, CAUT13APS025.
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2020-033803.
- ADOS-2
Autism Diagnostic Observation Schedule-Second Edition
- ASD
autism spectrum disorder
- AUC
area under the curve
- CI
confidence interval
- DCI
Developmental Check-In
- M-CHAT-R
Modified Checklist for Autism in Toddlers, Revised
- M-CHAT-R/F
Modified Checklist for Autism in Toddlers, Revised with Follow-Up
- MSEL
Mullen Scales of Early Learning
- PPV
positive predictive value
- ROC
receiver operator characteristic
- SCQ
Social Communication Questionnaire
References
Competing Interests
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Comments