To examine screening practices for autism spectrum disorder (ASD), subsequent referrals, and diagnostic outcomes within a large network of primary pediatric care practices.
Rates of ASD screening with the Modified Checklist for Autism in Toddlers (M-CHAT) at 18- and 24-month well-child visits were examined among 290 primary care providers within 54 pediatric practices between June 2014 and June 2016. Demographic, referral, and diagnostic data were abstracted from the medical records for all children who failed the M-CHAT (ie, score of ≥3) at either or both visits.
Rates of M-CHAT screening were 93% at 18 months and 82% at 24 months. Among 23 514 screens, scores of 648 (3%) were ≥3 (386 at 18 months, 262 at 24 months) among 530 unique children who failed 1 or both screenings. Among screen-failed cases, 18% received a diagnosis of ASD and 59% received ≥1 non-ASD neurodevelopmental disorder diagnosis within the follow-up period. Only 31% of children were referred to a specialist for additional evaluation.
High rates of ASD-specific screening do not necessarily translate to increases in subsequent referrals for ASD evaluation or ASD diagnoses. Low rates of referrals and/or lack of follow-through on referrals appear to contribute to delays in children’s receipt of ASD diagnoses. Additional education of primary care providers regarding the referral process after a failed ASD screening is warranted.
Research reveals that autism-specific screening can be used to identify children at risk for autism spectrum disorder (ASD) and facilitate early diagnosis of ASD. However, in practice, there is an average delay of 2.7 years between first parental concerns and diagnosis.
High rates of ASD screening do not automatically translate to earlier diagnoses of ASD. Low rates of referrals and/or lack of follow-through on referrals contribute to delays in identifying ASD, highlighting an educational need within primary pediatric care.
In 2007, the American Academy of Pediatrics published a clinical report on the identification and evaluation of children with autism spectrum disorder (ASD) that included recommendations for surveillance and screening,1 with particular attention to screening at the 18- and 24-month well-child visits. Since this guideline was published, rates of ASD screening by clinicians have reportedly ranged from a low of 8% to as high as 60%.2 Although the US Preventive Services Task Force determined that current evidence is insufficient to assess the value of early detection of ASD,3 data support the idea that children who receive behavioral intervention at younger ages have better developmental outcomes than older children.4–9 Often, these interventions are inaccessible without an ASD diagnosis,10 which stresses the importance of detection at the earliest possible age. In the US Preventive Services Task Force’s own report, autism-specific screening was responsible for identifying 50%–60% of ASD cases before any concerns were raised by parents or providers,3,11 suggesting that ASD screening may facilitate early detection that leads to early intervention for many children.
The American Academy of Pediatrics clinical report recommends that all children who have positive screen results for possible ASD be referred simultaneously for a comprehensive ASD evaluation, an audiology evaluation, and early intervention services. The timing of these referrals is key because whether they are being made at the time of the positive screening result or at a subsequent visit can directly affect age of diagnosis as well as receipt of appropriate therapies and interventions. Although children can be reliably diagnosed at as early as 18 months of age, the average age of ASD diagnosis in the United States is after 4 years,12 and reports reveal an average delay of 2.7 years between the time parents first report concerns and eventual ASD diagnosis.13,14 This means that many children may be missing out on viable, time-sensitive interventions that could affect developmental outcomes. Conversely, widespread implementation of universal screening can lower the age of ASD diagnosis by 2 years compared with recent Centers for Disease Control and Prevention surveillance findings, which increases time available for early intervention.15
Differences have also been reported in rates of screening and age at evaluation among racial and ethnic groups. Authors of a recent study using a cross-sectional analysis of the 2016 National Survey of Children’s Health reported that non-Hispanic white children were 10% more likely to receive screening than non-Hispanic African American or Hispanic children.16 Compared with non-Hispanic white children, a lower percentage of non-Hispanic African American and Hispanic children underwent an ASD evaluation before 36 months of age.17 This further compounds the impact of later detection on developmental outcomes among children of color.
The Modified Checklist for Autism in Toddlers (M-CHAT)18 is 1 of the most commonly used tools for ASD screening. In a study involving >18 000 toddlers who received a revised version of the M-CHAT that included a follow-up interview, 98% of those with positive screen results were determined to have significant developmental concerns and 54% were later diagnosed with ASD.19 It is important to note that 60% of the toddlers who had positive screen results in this study were subsequently evaluated for ASD by a clinical psychologist or developmental specialist using a rigorous battery of standardized instruments, which may not accurately mimic practices that occur outside the research context. For example, some of the diagnostic instruments may not be used in a clinical setting (eg, Autism Diagnostic Interview, Revised), accessing specialists may be more difficult in particular communities, and family or provider factors may impact follow-through on a specialty referral, all of which may affect whether and when children with positive screening results present for diagnostic evaluations and/or receive ASD diagnoses. Because of these differences, it is valuable to examine ASD screening practices and subsequent outcomes in real-world primary pediatric care settings to understand where efforts should be focused in bridging the gap between first concerns and eventual ASD diagnosis. With this in mind, our objectives with this study were to (1) conduct a retrospective, longitudinal medical record review to determine rates of ASD screening performed by primary care providers (PCPs) at 18- and 24-month well-child visits; (2) calculate the frequency of neurodevelopmental diagnoses made by PCPs or specialists, including the rates of ASD diagnoses, up to 4 years post-ASD screening; (3) describe referral patterns for children who fail ASD screening; and (4) compare rates of specialist referrals, completion of referrals, and age at ASD diagnosis across racial and ethnic groups and between insurance types.
The electronic medical records (EMRs) of all children who received M-CHAT scores of ≥3 (ie, a failing score) at their 18- and/or 24-month well-child visits from July 2014 through June of 2016 were reviewed. Visits were conducted within a hospital-owned network of 290 pediatricians within 54 primary care pediatric practices. Practices within the network accepted both commercial insurance and public insurance (Medicaid, Children's Health Insurance Program). Use of the M-CHAT was part of the standard screening procedure at both the 18- and 24-month visits for all practices. Screening was included as part of the recredentialing process for pediatricians within the network. All practices employed the original M-CHAT without the follow-up interview, with administration and scoring integrated into the shared EMR system (Epic) used throughout the network. After administration of the M-CHAT, scores were immediately available for the pediatrician to review at the time of the visit.
Options for specialist evaluation available to referring physicians included 2 tertiary care autism centers (1 internal and 1 external to the primary care network) where there are developmental behavioral pediatricians and neuropsychologists as well as internal and external neurologists, psychologists, and psychiatrists. These options were available for both privately insured and publicly insured patients.
Information regarding M-CHAT scores, ASD diagnoses made by the PCP and/or specialist, referrals to early childhood intervention or private therapeutic interventions, and referrals for subspecialty consultation occurring at the time of M-CHAT administration were abstracted from the EMR. Data regarding referral for specialist evaluation was abstracted by reviewing all orders entered into the EMR (both internal and external referrals) as well as documentation within the clinic note that a referral was discussed and specific contact information for specialists was provided to the family. Additionally, a referral to a specialist was documented by review of phone messages or e-mails from parents within the EMR to the provider requesting a referral. All documented referrals were made by the primary care pediatrician. There were no case managers or family navigators within the network to assist in the process. There was no formal referral tracking system within the EMR to determine if or when the referral and evaluation were completed.
Data regarding ASD diagnoses were obtained by review of diagnoses entered for all encounters and diagnoses added to the patient’s problem list by the primary care pediatrician within the EMR. Review of internal autism specialist documentation of diagnoses within the EMR as well as external specialist documentation uploaded into the EMR was abstracted. ASD diagnoses documented within a clinic note by the primary care pediatrician but not part of the encounter diagnoses were also included. Data regarding pediatrician-made, non-ASD diagnoses were abstracted for both the 18- and 24- month visits. Data regarding non-ASD diagnoses made by an autism specialist after PCP referral were abstracted. Demographic information (ie, race, ethnicity, and insurance type) was also abstracted.
The 18-month visits were conducted between July of 2014 and December of 2015. The 24-month visits were conducted from January of 2015 through June of 2016. Medical records were continually reviewed through May of 2018, which afforded a 23- to 46-month follow-up period to capture new ASD diagnoses and ages at diagnoses. All records were reviewed and re-reviewed by the lead author (S.A.M.), who is a developmental behavioral pediatrician. A random selection of ∼10% of the medical records were independently verified by 1 of the coauthors (L.N.B.), a neuropsychologist, with subsequent comparisons between these abstracted data by a colleague unaffiliated with the study. One discrepancy was identified through this process, reviewed together by S.A.M. and L.N.B., and corrected. This study was approved by the Baylor College of Medicine Institutional Review Board.
Descriptive statistics were used to summarize demographic variables, identify rates of screening at 18- and 24-month well-child visits, determine frequencies of types of referrals, and characterize the distribution of M-CHAT scores. Untransformed scores on the M-CHAT from the 18- and 24-month visits were treated as continuous variables. ANOVAs or χ2 tests, as appropriate, were used to determine if differences existed in rates of referrals, completions of referrals, and/or age at ASD diagnosis across racial and ethnic groups and between insurance types (commercial versus private). Comparisons were also made between groups with and without documented ASD diagnoses. All statistical analyses were performed by using SPSS version 24 (IBM SPSS Statistics, IBM Corporation).
Eligible children were those who had an 18- or 24-month well-child visit. A total of 93% of eligible children received an M-CHAT at 18 months (12 531 of 13 417) and 82% at 24 months (10 983 of 13 328). A total of 648 scores (3%) were ≥3 (386 at 18 months and 262 at 24 months) among 532 unique children who failed 1 or both screenings. Two children died during the follow-up period and were removed from subsequent analyses. Additionally, there were 39 children (7%) who were lost to follow-up. Loss to follow-up was defined as no well-child visit after 3 years of age. Of these 39 children, 31 were never referred and 8 were referred at or after the 18- or 24-month visit. All children lost to follow-up were removed for inferential analyses. Most patients had evidence of follow-up through 4 years of age. Across all failed screens, the average M-CHAT score at 18 months was 6.1 (SD = 3.8; range = 3–26) and at 24 months was 6.6 (SD = 4.4; range = 3–25). In Table 1, demographic information for this collective group with failed M-CHAT scores at either or both the 18- or 24-month visit is provided.
A total of 122 referrals for additional evaluation with a specialist were made at or before the 18- or 24-month visit (with an average age of 20.8 months), and 69 referrals were made for additional evaluation after the 24-month visit (with an average age of 36 months at referral); 26 children were referred at both times. This equates to 165 unique children (31%) among the entire screen-failed sample (n = 530) receiving specialist referrals. In Fig 1 the type and frequency of specialist referrals are detailed. Of referrals made at or before the 18- or 24-month visits (early referrals), 68 children (59%) completed the autism specialist evaluation and 42 received an ASD diagnosis at an average age of 27 months. Of referrals made after the 24-month visit (late referrals), including those made twice, 39 children (57%) completed the specialist evaluation, and 31 received an ASD diagnosis at an average age of 43 months. Of note, 2 children who received a non-ASD diagnosis on initial evaluation with a specialist went on to receive an ASD diagnosis at a later time. There was a >12-month difference in the age of ASD diagnosis for these children, depending on whether they were referred before (mean = 30.6; SD = 7.8) or after (mean = 43.4; SD = 8.8) their 24-month visit (P < .001). There were significant differences in the sex distribution between those diagnosed with ASD and those with no documented ASD diagnosis (P < .001; see Table 1). Additionally, both 18-month and 24-month M-CHAT scores were significantly higher for those diagnosed with ASD (18-month: mean = 7.3 [4.9]; range = 0–18; 24-month: mean = 7.7 [5.1]; range = 0–24) compared with those without an ASD diagnosis (18-month: mean = 4.5 [3.7]; range = 0–26; P = < .001; 24-month: mean = 3.3 [3.9]; range = 0–25; P = < .001), respectively.
Of those children who completed the specialist evaluation but did not receive an ASD diagnosis, the most common non-ASD diagnoses included speech and language disorder, global developmental delay, and behavioral disorder. There were no significant relationships between race and ethnicity and (1) time of referral (early, late, more than once; P = .45) or (2) completion of early (P = .82) or late (P = .08) referrals. Likewise, there were no significant relationships between insurance type and (1) time of referral (early, late, more than once; P = .78) or (2) completion of early (P = .14) or late (P = .16) referrals.
Among the 530 unique children who failed 1 or both M-CHAT screenings, 314 (59%) received a non-ASD neurodevelopmental diagnosis at the 18- and/or 24-month visit from their primary care pediatrician. These diagnoses included delayed speech and language development and global developmental delay. A total of 107 children (20%) were referred to early childhood intervention, 86 (16%) were referred for private therapy, and 66 (12%) were already receiving such services at the time of their failed screening. An audiology referral was made for 193 children (36%). Primary care pediatricians made the ASD diagnosis themselves for 6 of the children (1%).
There were 17 additional children whose ASD diagnosis was documented by the primary care pediatrician but who were either already being managed by a specialist or were referred for additional evaluation by someone other than their primary care pediatrician. In total, among unique children who failed the M-CHAT at either or both visits, 96 (18%) received a diagnosis of ASD within the 23 to 46 month follow-up period at an average age of 33.8 months (SD = 9.7; range = 18–56 months).
Review of documentation by the primary care pediatrician at the 18- and 24-month visits revealed that nearly half (n = 241 of 530) of the children did not have any comments pertaining to the M-CHAT. Documentation of an abnormal M-CHAT was observed for 108 children (20%); however, only 50% of these were referred. Additional documentation for nonreferral included waiting to discuss at the follow-up visit, parent declining the referral, and the failed M-CHAT being attributed to another neurodevelopmental condition or the presence of a speech delay. Less than 3% (n = 13) of primary care pediatricians documented discussing or re-asking some of the M-CHAT questions.
In this study, we examined ASD screening and referral practices within a large network of primary care pediatricians. High rates of screening using the M-CHAT were observed at both the 18- and 24-month well-child visits (93% and 82%, respectively), with 3% of children failing ≥1 screening. Delays in ASD diagnosis are frequently attributed to lack of timely screening, yet rates of screening using a standardized developmental measure have increased nationally from 23% to up to 60%.20 Despite even higher rates of screening in our study sample, 69% of patients were never referred for ASD evaluation with a specialist, and many patients were not referred until after 3 years of age, thereby contributing to older age at diagnosis.
Previous studies used to assess the predictive value of the M-CHAT have been conducted in controlled settings in which every child was offered a comprehensive ASD assessment after a failed screen and a majority of children completed this evaluation.19 To our knowledge, this study is the first to examine ASD screening practices in a large community setting, where placement of referrals, follow-through with evaluations, and rigor of specialist assessments are more difficult to ensure. These different study designs yielded sharply contrasting results for ASD diagnostic outcomes, with 54% of screen-fail children being diagnosed with ASD by Chlebowski et al20 versus 18% in the current study. This low rate of M-CHAT screen-fail children who ultimately received a diagnosis of ASD is consistent with previous studies of other developmental screening instruments used in actual primary care pediatric practice.21 However, in making comparisons to results from Chlebowski et al20 , it is important to note that children who received a comprehensive evaluation were all assessed by the same multidisciplinary team of ASD experts using the same standardized tools, which likely contributed to the identification of more children with ASD.
It is possible that a lack of specialist referrals, a failure to follow through on referrals, or some combination of the 2 (and potentially other factors) contributed to the lower rate of ASD diagnoses in the current study. In 10 separate implementation trials of general developmental/behavioral screening published between 1996 and 2014, rates of referral to a specialist were reported to range anywhere from 10% to 86% after a positive screen result.22 Even when referrals are placed to pediatric specialists, 30% of referrals are never completed in a community health center setting.23 In our study, 31% (n = 165 of 530) of patients were ever referred to a specialist, with 65% (n = 107 of 165) of these being completed.
Many children in our study who failed the ASD screening received a non-ASD developmental diagnosis (59%), and a large proportion of these children (34%) were referred for state-funded early intervention or private therapies. This suggests that primary care pediatricians are making non-ASD developmental diagnoses and are frequently referring for recommended interventions but few are referring for autism-specific evaluation with a specialist at the time of a failed screen. Evidence-based treatments for ASD, including early intensive behavioral interventions, are most beneficial in improving language and educational placement when initiated at preschool age and continued for 2 to 3 years.24 However, many children may not be able to access these services without an ASD diagnosis.10 Moreover, high demand for these services means that even children with an ASD diagnosis may be placed on a waitlist for services, particularly for early intensive behavioral interventions.25 This further underscores the need for timely referrals and subsequent evaluations for ASD to expedite the commencement of early intensive behavioral interventions.
The finding that providers are frequently referring for early intervention is consistent with a previous report revealing that, at the time of their ASD evaluation, 89% of children were already receiving some type of services (eg, speech therapy or occupational therapy).26 This is in sharp contrast to an earlier study, which reported that only 10% of children between the ages of 9 and 24 months evidencing delays in development were receiving services.27 The importance of referring to early intervention and private therapies should not be understated, but without additional assessment and diagnosis (either by the primary care pediatrician or a specialist), targeted interventions such as early intensive behavioral interventions may be difficult or impossible to access.10
M-CHAT scores at both the 18-month and 24-month visits were significantly higher for those patients with a documented ASD diagnosis compared with those patients without an ASD diagnosis. Likewise, children who failed the M-CHAT at both the 18- and 24-month visits were more likely to receive an ASD diagnosis. It seems possible that children with higher scores had a more severe and/or clearer presentation that prompted more immediate referrals for an ASD evaluation, particularly if observed across consecutive screen fails. A search of the literature did not reveal similar observations; however, authors of 1 study found that the number of areas of concern on the M-CHAT (eg, social, repetitive, restrictive) was associated with both M-CHAT score and ASD diagnostic scores on the Autism Diagnostic Observation Schedule and the Childhood Autism Rating Scale-2.28 Again, this could suggest that endorsement of a variety of ASD symptoms lends to greater concern for ASD that prompts immediate referral for specialist evaluation.
The median age of ASD diagnosis is 4 years,29 yet ASD symptoms are most often noted in the second year of life. Several groups have examined factors associated with delays in ASD diagnosis, some of which have focused on demographic factors or child-specific factors (eg, race, ethnicity, parent education, child cognitive functioning, and ASD severity) and others which have focused on systemic factors.30 In our study, no differences in referral placement, completion of referral, or age at diagnosis were observed between racial and ethnic groups or by insurance type (commercial versus public). Although screening has been associated with earlier ASD diagnoses, difficulties in finding specialist providers, the distance required to access providers, and delays in diagnosis have been associated with both changing diagnoses and being told that the child did not have autism.30 Our findings are similar to the extent that specialists who evaluate for ASD were not often accessed because of a lack of referrals by primary care pediatricians and/or decreased rates of referral completion, and the low numbers of ASD diagnoses suggests that the primary care pediatricians were not evaluating for or making ASD diagnoses themselves. Overall, this appears to be more consistent with a wait-and-see approach, which is strongly discouraged in the ASD literature.31
One option to address low rates of referrals is to use the more current version of the M-CHAT with follow-up interview. Using the follow-up interview has reportedly increased the positive predictive value of the M-CHAT from 0.11 to 0.65,32 suggesting that briefly reviewing particular behaviors in more depth may help to clarify the picture and inform the provider’s referral actions. Another option is to implement a best practice alert within the EMR that notifies the pediatrician of screen-failed scores during the visit and advises appropriate courses of action. This would serve as a call to action after a failed M-CHAT screen and require providers to respond to the alert to afford tracking of their actions (eg, conducted follow-up interview, evaluated for ASD, referred for ASD specialist evaluation).
Strengths of this study include (1) examination of ASD screening practices within a large network of 290 primary care pediatricians at >50 sites with high rates of screening compared with previous reports, and (2) the fact that all screen-fail children were followed to determine diagnostic outcomes 2 to 4 years later. However, some important limitations should be noted. Although our commercial (60%) versus public (40%) insurance breakdown was consistent with data from a recent report from the Centers for Disease Control and Prevention,33 our sample had a larger percentage of Hispanic children compared with the national average, which may affect generalizability in communities with a different racial and ethnic breakdown. Data were collected in a retrospective fashion on the basis of review of EMRs. As such, reasons for nonreferral were based on primary care pediatrician documentation, and we did not directly solicit pediatricians’ rationales for nonreferrals. Likewise, additional communication during the visit between pediatrician and parent may not have been documented within the EMR, potentially leading to an underrepresentation of diagnoses obtained elsewhere. It is also important to recognize for those children who received a non-ASD diagnosis that an ASD diagnosis is difficult to entirely rule out given the high rates of comorbid conditions that can present with ASD. Similarly, as each birth cohort ages, the proportion of individuals in that cohort who are diagnosed with ASD increases,34 so it is possible that many of these children will go on to receive ASD diagnoses at some later age. In fact, the varied follow-up period within the current study (23–46 months) likely afforded capture of some of these diagnoses for children who were under longer surveillance. Continued follow-up with the entire screen-failed sample could help determine how often and when later ASD diagnoses are made. Finally, the M-CHAT follow-up interview was not administered to clarify failed items because the follow-up interview is not currently used within this network of pediatric practices. Although the positive predictive value of the M-CHAT is enhanced when using the interview,32 it is unknown whether primary care pediatricians would be more likely to refer a child who continued to fail the M-CHAT after follow-up interview for specialist evaluation, but this is an important area for future investigation. Additional primary care pediatrician training on how to explain both the results of a failed screen to a family and the importance of additional evaluation should be considered because these directly impact follow-through after referrals are placed.
Improving early identification of ASD requires appropriate developmental surveillance at all well-child visits that incorporates both parent concerns and direct observation of the child and documentation of this in the medical record along with autism-specific screening at specified ages. However, efforts to improve early diagnosis and treatment of children with ASD go beyond the administration of an autism screen. Our study shows that, for many children, increased rates of screening do not automatically translate to earlier diagnoses. There is an apparent absence of action after failed ASD screening as well as difficulties with completion of specialty referrals that contribute to delays in children’s receipt of ASD diagnoses. Additional education of PCPs regarding the referral process after a failed ASD screening is warranted. Likewise, if there is diagnostic certainty after surveillance and screening, primary care pediatricians should be empowered to make ASD diagnoses without a wait for specialty referral. However, in situations of diagnostic uncertainty, referral to an autism specialist must more promptly follow surveillance and screening to promote timely diagnosis and receipt of evidence-based interventions. Additional education of PCPs with regard to both ASD diagnostic and referral practices is necessary to address these needs.
Dr Monteiro conceptualized and designed the study, collected data, and drafted the initial manuscript; Dr Dempsey conducted statistical analyses and contributed to drafting the initial manuscript; Dr Berry advised on study design and assisted with medical record verification; Dr Voigt assisted in the conceptualization and design of the study; Dr Goin-Kochel advised on study design and contributed to drafting the initial manuscript; and all authors reviewed and revised the manuscript.
FUNDING: Funded by the William Stamps Farish Foundation. Dr Goin-Kochel is also partially supported through the Intellectual and Developmental Disabilities Research Center (1U54 HD083092) at the Baylor University College of Medicine.
POTENTIAL CONFLICT OF INTEREST: Dr Goin-Kochel is contracted with Yamo Pharmaceuticals to consult on clinical trial research design; the other authors have indicated they have no potential conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.