Screening Tools for Autism Spectrum Disorder in Primary Care: A Systematic Evidence Review

On the basis of a preliminary literature review, we developed a list of 4 key questions (KQs) around autism screening. KQ 1 examined the methods and tools used in PC and PC-like settings to screen for children in 3 age ranges (0–2, 3–5, and 6–12 years). KQ 2 reviewed the psychometric performance characteristics associated with these screening methods. KQ 3 reviewed whether there were adequate data to examine a possible association between early screening and improved outcomes. Finally, KQ 4 reviewed evidence for harm of screening. See Supplemental Tables 7 through 10 for details of KQs.

Study inclusion and exclusion criteria are listed in Tables 1 and 2. We searched Medline through OVID, PsycINFO, Educational Resources Informational Clearinghouse, and Cumulative Index of Nursing and Allied Health Literature for English-language articles published from January 2000 to June 2018. The start date of 2000 coincides with the publication of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) and the wide-scale availability of gold-standard autism assessment instruments. Search strategies and terms included a combination of medical subject headings and keywords relevant to screening for ASD and the KQs (see Supplemental Tables 7 through 10). After the initial literature search, reference lists of recent narrative and systematic reviews were reviewed to identify missed articles.

Multiple reviewers independently screened the titles and abstracts for inclusion and exclusion criteria. We identified 5255 unique citations addressing screening. After abstract review, we excluded 4846 publications, leaving 409 articles for full text review. See Fig 1 for a flow diagram of process. Two reviewers examined each article for inclusion and exclusion, with disagreements resolved by consensus of 4 authors (S.E.L., A.W., J.F., Z.W.). Articles that did not address the KQs, were conducted outside of countries rated as “very high human development” by the United Nations Development Program, included high-risk or referral populations, or with a sample size of <100 subjects were excluded.

Team members entered information for each KQ in the 27 included articles into a structured abstraction form. A second team member independently reviewed the abstraction. We abstracted or calculated sensitivity, specificity, positive and negative predictive values (NPVs), and identification rates wherever possible. Then, using a systematic quality rating tool developed through key stakeholder appraisal of current tools and processes, 2 reviewers independently rated the quality of each study as good, fair, or poor. The tool established clear benchmarks for quality on the basis of sample size, reference standard, use in general population, and inclusion of participants with negative screening results (see Table 3). Discrepancies in quality rating were resolved through team consensus.

Among the 27 unique studies included in this review, 7 were good quality (Table 4), 18 were fair quality (Table 5), and 2 were poor quality (Table 6). Articles primarily addressed the known screening tests and their psychometric properties (KQ 1 and KQ 2), with notable exception of limited data surrounding comparative psychometrics based on risk factors, child age, and child and family characteristics; little evidence for improved outcomes (KQ 3); and no results regarding harm (KQ 4).

This specific review included studies of screening strategies for young children (<13 years of age) in PC and PC-like settings. No studies were identified meeting inclusion criteria for children in the 6 to 12 age range.

The Checklist for Autism in Toddlers (CHAT) was developed to identify ASD at 18 months via parent report and practitioner observation of early red flags (eg, lack of pointing, pretend play).²³  Two studies were included, of good and fair quality, respectively.^23,32 

In the good-quality study, researchers managed a birth cohort of children screened in PC at 18 months.²³  ASD diagnostic data were linked to CHAT scores and clinical diagnosis for psychometric performance characteristics. Use of the CHAT resulted in substantial underidentification of ASD (ie, low sensitivity and modest positive predictive value [PPV]) in a low-risk population. In the fair-quality study of the CHAT, it was not possible to extract true sensitivity, specificity, or NPV. A PPV of 0.43 was calculated by author J.F.; however, this number should be interpreted with caution because of the small sample size and large loss to follow-up.³² 

The Modified Checklist for Autism in Toddlers (M-CHAT) is a parent-completed questionnaire to identify children 16 to 30 months at risk for a diagnosis of ASD.³⁰  In several population-based studies, researchers have examined its psychometric characteristics, and revisions of the form and content have decreased the number of questions from the original 23 (M-CHAT) to 20, with additional items as follow-up questions for screen failures (Modified Checklist for Autism in Toddlers, Revised with Follow-Up [M-CHAT-R/F]).^26,41  This instrument has been translated into several languages and studied in multiple countries.^{37,39,41,44,46,47}  Nineteen studies of the M-CHAT were included, with 3 being good quality, 14 being fair quality, and 2 being poor quality (Tables 4 through 6).

In the largest M-CHAT study,⁴⁰  presenting cumulative data from a group or series of investigations, children were screened with the M-CHAT at well-child visits. Of the positive M-CHAT screening results, 74.6% completed the Modified Checklist for Autism in Toddlers–Follow-Up (M-CHAT/F), and 21% remained screen positive. Subsequently, 60.7% of those who continued to screen positive after follow-up completed a diagnostic evaluation. Over half (53.8%) received an ASD diagnosis, and 97.7% were identified as having a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnosis or developmental concerns. Because not all children screened received a diagnostic evaluation, sensitivity and specificity could not be calculated. The authors attempted to identify missed cases through multiple methods (eg, provider concerns, administration of other screeners), but the rules and data related to this procedure were not clear enough to calculate psychometric characteristics. In the same capacity, given limitations regarding these data (eg, nonrandom sampling, unclear rules for pediatrician flags and screening), there are insufficient data for understanding true sensitivity, specificity, and NPV, and therefore the study received a fair rating.

In a second good-quality study, the researchers examined the reliability and validity of the M-CHAT and M-CHAT-R/F in a large population of toddlers at well-child visits.²⁶  Of M-CHAT positive screening results, 82% were followed-up with the M-CHAT-R/F, and 36.8% remained screen positive. Of those who screened positive for both stages of the M-CHAT-R/F, 63% had a diagnostic evaluation. Almost half of this group (48%) received an ASD diagnosis, 47% were classified as having other concerns, and 5% were classified as typically developing. PPV was higher when taking into account all actionable developmental concerns (0.95 vs 0.48). To test potential missed cases, one site conducted a stratified sampling of children who screened positive on M-CHAT-R/F who passed the follow-up interview. Some children who screened negative on M-CHAT-R/F were also identified as at risk by physician concerns and other screeners, and additional children were identified with ASD.⁴⁰  Given the limitations regarding these data (eg, nonrandom sampling, unclear rules for pediatrician flags, and alternate scoring), there are still concerns about understanding true sensitivity, specificity, and NPV. Additional good-quality studies report PPVs for the M-CHAT between 0.14 and 0.5 and M-CHAT/F between 0.48 and 0.67.^27,28 

Seven investigators reported using translations of the M-CHAT or M-CHAT/F.^{10,28,37,39,41,44,46}  One study in France was of good quality, with moderate sensitivity (0.67), high specificity (0.94 and 0.99 with follow-up), low to moderate PPV (0.14 and 0.60 with follow-up), and high NPV (0.99).²⁸  The other studies were of fair or poor quality.^{10,37,39,41,44,46}  Given substantial modifications in use and translation, it is not clear how these instruments might relate to other US population studies.

Although the quality of the English studies of the M-CHAT and M-CHAT/F limits interpretation of population-level psychometric properties, data are available in which it is suggested that screening identified children before documented parent and provider concerns. In one fair-quality study, <20% of children screening at risk were flagged by surveillance.³⁵  An additional fair-quality study in the United States screened toddlers at all scheduled medical appointments (well-child, sick, follow-up, injection visit) over 6 months at a large community-based pediatric practice.³⁸  They used a combination of the M-CHAT and the Infant Toddler Checklist (ITC) and managed failures in either instrument. Although no PPV can be calculated for the M-CHAT independently, the combined ITC and/or M-CHAT PPV is estimated at 0.27. In this study, less than two-thirds (60%) of cases were identified before parent concerns, and 50% were identified before pediatrician concerns.³⁸ 

The ITC is a communication and broad developmental screener that is part of a larger assessment system, the Communication and Symbolic Behavior Scale (CSBS) Developmental Profile.²⁴  The ITC was not designed as a stand-alone screener for ASD but has been studied in relation to identification of ASD. The ITC is a 24-item questionnaire of specific social communication milestones, with an overall question regarding presence of concerns. Cutoff scores are available from normative samples for children <24 months of age. Three studies were included, 2 good quality^24,25  and 1 fair quality.³⁸ 

In the first good-quality study, children (6–24 months) were recruited from PC settings.²⁴  Children identified by screening positive or from parental concerns were further assessed with the CSBS sampling technique (ie, brief interactive assessment coded by researchers). Data were linked to an ASD prevalence database to identify ASD risk through questionnaire results, CSBS assessment data, and state administrative data. In the study, the authors reported a sensitivity of 0.933 and a calculated PPV of 0.69 to 0.77. There was substantial variability in repeated screens and timing of positive screens, which limits the understanding of the ITC as a single point and time screener. In addition, the population questionnaire surveillance technique resulted in a low yield rate, which may affect understanding of true sensitivity of the measure. In the second good-quality study, researchers examined the effectiveness of the ITC as an ASD screener at 12-month well-child visits conducted by 137 pediatricians.²⁵  A small subset of children who screened negative was referred to as a nonrandom “typically developing” control sample. A total of 73% of this group was ultimately designated with normal development. Of screen failures, 17% were diagnosed with ASD, 30% had language delay, 5% had developmental delay, 20% had another diagnosis, and 25% were false-positive results. PPV was substantially increased when inclusive of all developmental diagnoses to 0.75. In the fair-quality study, researchers screened toddlers with the ITC and the M-CHAT, as reviewed above.³⁸ 

The Psychological Development Questionnaire–1 (PDQ-1) is a 10-item questionnaire that primarily reflects caregiver evaluation of their child’s level of communication and social orientation. It was developed in 2018 and therefore has only been evaluated in one known study by the measure authors.²⁹  This study was of good quality and included a large and diverse sample.²⁹  Particularly among children 31 to 36 months, the PDQ-1 exhibited very high psychometric properties, with high sensitivity (0.71–1.0), high specificity (0.99–1.0), PPV of 0.79–1.0, and NPV of 0.99–1.0.

In additional studies in the United States included in this review, researchers used the Social Communication Questionnaire and the Pervasive Developmental Disorders Screening Test II as potential screeners, but only 1 fair-quality study of each was documented, which limited our ability to understand potential value,^8,43  particularly in children >31 months.⁸  Two other instruments, the Early Screening of Autistic Traits (ESAT) and the Young Autism and Other Developmental Disorders Checkup Tool, were used in other countries for unselected population screening in children <18 months of age.^31,36  Because of limited relevance to common PC practices (eg, involving extensive procedural evaluation), they are not discussed in this review.

No studies directly addressed the benefits of referral for early treatment after screening within the study. Available data are limited to either screening in relation to initiation of service^{24,25,35,38,40,48}  or studies of intervention with no direct link to screening and identification.^4,49 

No included studies addressed harm of screening.

Results of this systematic review documents that widely available ASD screeners, when incorporated into universal screening of unselected pediatric populations between 16 and 40 months of age, can accurately identify many children with ASD. The most extensively studied tool (ie, M-CHAT family of screeners) has demonstrated PPVs of 0.06 to 0.60 in the highest-quality studies.^28,40  However, this variability highlights that the screener differs in its effectiveness to identify ASD in children across ages, settings, and contexts, with psychometrics varying substantially on the basis of use of follow-up interview for validation. Importantly, when other developmental concerns are incorporated into assessment of PPV, false-positives are infrequent. Only one included study examined psychometric properties or characteristics of universal screening for children >40 months.⁸ 

Despite these modest to high PPVs, methods employed to study ASD screening at present remain insufficient to understand additional psychometric properties. Specifically, few of the studies provided adequate data of random or cohort samples to determine the frequency of missed cases. In addition, the attrition seen in study validation is substantial across almost all employed tools. In future studies, researchers should focus on longitudinal population-based samples to address issues of NPV as well as provide clearer data regarding the link of screening to improved outcomes.

Comparing estimated prevalence rates of ASD in community population studies (ie, 1 in 59¹  in 8-year-olds and 1 in 75 in 4-year-olds⁵⁰ ) is a gross validation method for evaluation of population identification. Calculated rates for studies included in this review appeared modest at 1 in 149 and 1 in 300.^26,30,40  These numbers are below the conservative estimated prevalence bands based on epidemiological studies.^1,5,50  It is important to note that almost all of these screening studies have (1) involved research opt-in follow-up procedures, which may skew the sample; (2) experienced significant attrition; and (3) excluded patients previously referred for concerns based on developmental surveillance. In this capacity, the PPVs reported may represent conservative estimates of the ability of screening tools and broader surveillance processes to identify children with ASD. Such selection and evaluation within this systematic review are intentional because it approximates screening in the absence of concerns in the PC office, where parents (or providers) may not have identified concerns to discuss during routine care. However, in most cases, screening in the absence of concerns represents only a portion of our understanding of the process of early identification of ASD. If children are referred earlier to services for general developmental or specific ASD concerns and are subsequently excluded from this type of evaluation of screening results, one must be cautious in interpreting base rates of identification. It is possible that systematic specific ASD screening of young children when combined with identification of concerns via surveillance and general developmental screening before designated screening time points may identify a much larger percentage of children. Unfortunately, few studies have included methodologies for systematic evaluation of deployment of hybrid surveillance screening procedures. This is unfortunate because real-world pediatric care involves not just simple point-in-time screening but a variety of deployed surveillance and repeated screening practices over time. This suggests the need for pragmatic clinical trials of such strategies to optimize our ability to identify young children as well as develop coherent strategies for monitoring for ASD symptoms at later ages, given the near absence of data to this end.

Other well-studied tools in comparison with the MCHAT-R/F have demonstrated modest psychometric characteristics when used as single-point screeners of well-child populations. The ITC can be used to identify many children with actionable developmental concerns, before parental and clinician concern between 12 and 24 months of age, but using it in isolation may label ≥10% of the total low-risk population at risk for ASD or other communication disorders.²⁴  Other screening measures included in this review currently do not have replicated results suggestive of superior performance, although new instruments and approaches may be able to improve psychometric functioning.²⁹  There are some emerging data from non-US–based programs and adaptations regarding the potential for combined screening and surveillance and prospective strategies involving community health visitation and trained providers to identify ASD at early ages.^49,51  However, these tools have not yet been studied in PC settings and practices in the United States.

In several included articles, it is highlighted that disparities in access to use of screeners and diagnostic resources exist across racial, ethnic, linguistic, and socioeconomic lines.^8–10  Even among those who have been screened, children of families with lower education or who identify as racial minorities were more likely to screen as a false-positive.⁴²  Additionally, those with lower educational attainment had lower odds of completing the follow-up interview compared with those with more education.⁴²  Disparities may be reduced by having early child care providers screen for ASD in their classrooms⁸  or having the screen administered by an interviewer instead of being self-administered.⁹ 

Although there have been many studies of ASD screeners for young children in clinical and convenience samples, there are few studies of universal screening practices in PC settings, and there is a dearth of knowledge about screening beyond the early toddler and preschool years. We explicitly excluded studies of screeners used in preidentified risk or diagnostic samples because screening these groups may represent a different population than children identified during universal screening in general pediatric care. Including children who have been referred may give information about factors associated with an early diagnosis, which is not the aim of this review. Consequently, excluding these studies from our review resulted in a small number of included studies. Findings of this report should be interpreted with significant caution in that they represent studies not of “universal screening” in the broadest form but rather “universal screening” excluding children who have been previously referred for risk or previously identified with neurodevelopment concerns. In reality, data inclusive of children identified by surveillance and screening procedures would ultimately provide a more-complete picture of how effective these processes may be.

Other major limitations are lack of data and methods to examine screening processes and tools and little information on how to capture potential false-negatives during the screening process. Few included studies beyond the original CHAT investigation adequately follow a large enough random sample of children to be able to comment on NPV, specificity, or sensitivity in population form.²³  The available included studies also had high rates of attrition, although our assessment of quality did not evaluate studies on the basis of attrition. There is little information available about children who fail to complete referrals for diagnostic evaluations. Data from these studies lack how psychometric performance characteristics of screeners in PC settings may vary on the basis of child and family risk factors. Similarly, included studies of the psychometric functioning of early screeners were almost universally accomplished with the aid of specific research supports and processes, with limited data about how feasible all stages of the screeners may be in pragmatic implementation (ie, required 2-stage implementation of screener and algorithmic interview). Finally, in the data, it was suggested that children whose screening results placed them in the at-risk category and who were later diagnosed with ASD received higher levels of service in comparison with children without such screening results.⁵²  However, systematic prospective studies have not yet been conducted that examine service access and the impact of service delivery postdiagnosis for children screened at risk.

To date, most available data regarding screening for risk for a diagnosis of ASD involve preschool children. Children with ASD who are identified after the age of 4 years may reflect differences in development of social skills in the context of adequate language development and/or milder behavioral issues. There is a critical need for development of enhanced tools and systematic study of older children.

As ASD screening takes place in a broader surveillance process wherein pediatric providers monitor many aspects of growth, development, and health, and it is urgent to develop streamlined ways to integrate these screeners that can be evaluated in a methodologically rigorous manner (ie, pragmatic trials). Specifically, data inclusive of children identified by surveillance and screening procedures would provide a more-complete picture of how effective these processes may be in shaping child outcomes. Similarly, studies of comparative implementation of different screening and surveillance practices and tools (eg, repeated screening procedures, combined use of measures and observers, direct comparison of available tools, validation strategies) would be valuable. Despite resource challenges, research adequately evaluating participants who screened negative would provide valuable data. Similarly, data directly evaluating traditional intermediate and longer-term health outcomes, as well as pertinent outcomes for the ASD population (eg, quality of child and family life, adaptive and social integration measures), are lacking. There are potential financial and human costs and harm to individuals, families, and systems related to optimal deployment of screening tools and processes; however, little empirical evidence of these factors exists. Moreover, given the fundamental heterogeneity of ASD of developmental and behavioral presentation, comprehensive study of the characteristics of children identified at certain ages with specific tools is necessary to the development of processes designed to pick up heterogeneous profiles at later ages.

We found significant evidence that formal ASD screening of children at young ages (16–40 months) in general pediatric practices has PPVs for identifying children with ASD of ∼50%.^{23,24,26–29,40}  When considering the ability of this screening to identify other developmental conditions, the PPV rises to ∼95%.^26,40  Little information is available on the degree to which screening tools miss positive cases. In addition, although there are separate studies of potential benefits of screening in initiation of services and of potential intervention benefit, no studies directly relating screening to clinical outcomes and/or harm exist. Given the pragmatic, ethical, and political challenges associated with establishing this linkage, clinical and policy groups will likely have to continue to guide screening practices in absence of this evidence. Essentially, such recommendations and practice behaviors must act on data documenting (1) that screening unselected young children will pick up many, but not all, children and (2) that early treatment can improve certain outcomes for children, with substantial variation in terms of what treatments are available and what benefits will be seen.

In this capacity, practical implications of these findings are simultaneously supportive of (1) potential use of validated ASD screening tools at young ages in PC in unselected populations, acknowledging known psychometric and service limits of available use, and (2) the urgent need for rigorous development and evaluation of enhanced strategies for more effectively identifying even-larger groups of young and school-aged children with ASD across varied systems of care.

ASD

autism spectrum disorder

CHAT

Checklist for Autism in Toddlers

CSBS

Communication and Symbolic Behavior Scales

DSM-IV

Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition

DSM-IV-TR

Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision

ESAT

Early Screening of Autistic Traits

ITC

Infant Toddler Checklist

KQ

key question

M-CHAT

Modified Checklist for Autism in Toddlers

M-CHAT/F

Modified Checklist for Autism in Toddlers–Follow-Up

M-CHAT-R/F

Modified Checklist for Autism in Toddlers, Revised with Follow-Up

NPV

negative predictive value

PC

primary care

PDQ-1

Psychological Development Questionnaire–1

PPV

positive predictive value

USPSTF

US Preventive Services Task Force