A Statewide Tiered System for Screening and Diagnosis of Autism Spectrum Disorder

At the time of initiation of the EAE Hub system, Indiana lagged behind the national average in the number of children receiving standard developmental screening; had a higher number of children at risk for developmental, behavioral, or social delays;³¹  and had many counties designated as Medically Underserved Areas³²  (see Supplemental Table 4). Reliable state-level data on the average age of ASD diagnosis in Indiana do not exist. However, an internal needs assessment indicated that ASD and developmental delay were the 2 most prevalent diagnoses served in the neurodevelopmental outpatient clinics of the state’s largest pediatric hospital and that most diagnoses were made after children entered the public school system. Furthermore, this assessment revealed that, similar to nationally reported wait times of 6 to 12 months,^33,34  Indiana children were waiting an average of 9 to 12 months from referral to evaluation.

The guiding framework of the EAE Hub system is composed of 3 tiers of service: (1) children receive standard developmental surveillance and screening and ASD screening at primary care well visits; (2) children, ages 18 to 48 months, identified as at risk for ASD are referred to a local EAE Hub for ASD evaluation and counseling on next step recommendations; and (3) children with complex or equivocal symptom presentation are referred for comprehensive ASD evaluation at a specialty diagnostic center. A framework of quality improvement, coordination of care, community engagement, and planned comanagement with the referring PCP overlays the system. The primary EAE Hub team included an executive director (ie, academic pediatrician), project coordinator, and practice liaisons. Notably, the team included 2 parents of children with neurodevelopmental disabilities (including ASD), promoting the importance of family-professional-community partnership in this effort. The development and scale-up of the EAE Hub system was funded by a combination of federal and state grants, philanthropy, and individual contracts with EAE Hub institutions (see Supplemental Information for further information).

Developmental screening technical assistance to pediatric and family medicine primary care practices was sequentially targeted around geographic regions as each EAE Hub was developed. A practice liaison and pediatrician visited practices to provide education on (1) standardized developmental and ASD screening procedures following American Academy of Pediatrics policy;^16,17  (2) training on the Ages and Stages Questionnaires, Third Edition (ASQ-3)³⁵  and Modified Checklist for Autism in Toddlers, Revised with Follow-up (MCHAT-R/F)³⁶  as well as kits at no cost; and (3) referral procedures for the local EAE Hub and community services and resources. Follow-up technical assistance occurred as needed. Geographically focused outreach to community organizations including early intervention agencies, school corporations, advocacy groups, and regional representatives of state agencies was conducted to provide education on the EAE Hub system and develop partnerships to support children and families.

The EAE Hub leadership team received individualized and intensive training from the developers of the Screening Tools and Referral Training-Evaluation and Diagnosis (START-ED) model.²⁵  The objectives of START-ED are to provide primary care pediatricians with a functional and streamlined framework and assessment tools for the accurate diagnosis of young children with ASD. Included in the training were both didactic education on ASD evaluation and certification in administration and scoring of the STAT, selected because of its utility in the assessment of toddlers in the primary care setting. This training was used to prepare the EAE Hub leadership team to adapt the START-ED model for the development of the EAE Hub training curriculum and clinical pathway.

The first EAE Hub site was piloted at an academic health center–affiliated pediatric primary care clinic, allowing for refinement of the model and training curricula. Additional EAE Hub sites were selected on the basis of a 2-step process including (1) an analysis of population distribution to target geographic regions and (2) selection of pediatric primary care practices in targeted regions with known pediatric champions who were actively engaged in early childhood initiatives. Given the general assumption that pediatricians have more formal expertise and experience in atypical child behavior and development, other types of primary care practices (eg, family medicine) were not recruited as EAE Hub sites.

The goal was for each EAE Hub to be a clinically and administratively self-sustaining site within the system. EAE Hub sites ranged from large health systems to private pediatric practices, with commitment from their governing leadership to providing this service in their communities. Individual EAE Hubs negotiated evaluation capacity, payment and revenue, office space and support staff needs and related issues with their home organization. Although there was no formal top-down oversight by the EAE Hub leadership team, consultation and ongoing support was provided to sites through individualized technical assistance and a monthly learning collaborative webinar. The collaborative focused on didactic training, case presentations, and practice-based quality improvement discussions. An annual meeting was held to review quality improvement data, share practice updates, assess system needs and goals, and foster relationships to support sustainability.

Each EAE Hub signed a memorandum of understanding to document formal collaboration and agreement to (1) develop a clinical team, including a pediatrician or nurse practitioner (NP) and nurse or medical assistant, ideally with the inclusion of an administrative leader and care coordinator to support follow-up care; (2) participate in EAE Hub training; (3) implement the standard EAE Hub clinical pathway (see Table 1); (4) collect quality indicator data; and (5) participate in the monthly learning collaborative and annual meeting.

Each EAE Hub, including clinicians and staff, participated in an on-site multiday intensive training on ASD evaluation. Training was provided by academic faculty in general pediatrics, developmental pediatrics, child psychology and/or psychiatry, and quality improvement science. Included in the didactic curriculum were education on developmental screening, structured developmental history and interviewing techniques (including the assessment of Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition [DSM-5] ASD symptoms), medical and psychological differential diagnosis and common comorbid concerns, communication skills for delivery of diagnosis, and current evidence regarding ASD interventions. Training on billing and coding and practice quality improvement was provided to EAE Hub clinicians and pertinent practice staff. Regional community agencies were invited to share local resources, and families of children with neurodevelopmental disabilities joined the training as “faculty for the day” to share their experiences with diagnosis and navigating systems and services. Clinical practicum training included in vivo practice and supervision on all steps of the clinical pathway for up to 6 evaluations of children with (1) typical development, (2) confirmed diagnosis of ASD, and (3) referral concern for ASD. Training faculty provided learners with written feedback, including ratings of performance during observed practicum sessions. Measures of trainee satisfaction were used to guide revisions of the curriculum over time.

The EAE Hub model was developed following the principles of the START-ED model,²⁵  whereby clinicians are provided with training on a standard clinical evaluation protocol and assessment tools for diagnosis of ASD in toddlers. In contrast with standard comprehensive ASD evaluation (ie, which often includes labor-, training-, and cost-intensive diagnostic tools such as Autism Diagnostic Observation Schedule, Second Edition,³⁷  and Autism Diagnostic Interview-Revised³⁸ ), the EAE Hub clinical pathway specifies a brief evaluation protocol designed to be completed in a 90-minute primary care office visit. Evaluation components include a review of ASQ-3 and MCHAT-R/F, diagnostic interview to solicit DSM-5 ASD symptoms and medical history, physical examination, and administration of the STAT (see Table 1). The STAT, a level 2 screening tool originally developed for use in children ages 24 to 35 months, has been shown to have good psychometric properties³⁹  (ie, sensitivity = 1.0; specificity = 0.85; positive predictive value = 0.86; negative predicative value = 0.92). Additional research has revealed promising utility for an extended age range of 14 to 47 months.^26,40,41  At EAE Hub system initiation, an age range of 18 to 42 months was targeted. However, over time, the age range was expanded up to 48 months on the basis of available STAT guidelines (eg, including the use of alternative age-based scoring procedures^39–42 ) as well as clinician feedback regarding comfort and desire to serve a broader group of children for which the standard clinical pathway was appropriate.

EAE Hubs collected and submitted de-identified data for each evaluation via standardized paper-based visit summary sheets or direct entry into an online database. To minimize data collection burden on EAE Hubs and ensure Health Insurance Portability and Accountability Act of 1996 compliance, individual demographic information was not collected (see Supplemental Information and Supplemental Table 4 for county- and state-level demographic information). Data were stored in a secure database and analysis was completed with IBM SPSS Statistics, version 26 (IBM SPSS Statistics, IBM Corporation).

From 2012 to 2018, the EAE team provided technical assistance on developmental screening to 193 primary care practices composed of 798 clinicians (ie, 82% physicians; 17% NPs) and their staff. Outreach efforts also included presentations to 136 community organizations, including early intervention agencies (n = 31), schools (n = 38), autism intervention agencies (n = 7), and local community organizations (n = 60). Medical presentations were delivered at 73 events (see Fig 1A). Education and outreach efforts were conducted in 76% of Indiana counties (see Fig 2).

Twelve EAE Hubs were developed in pediatric primary care practices across the state of Indiana (see Fig 2), representing 8 health systems. EAE Hub training was delivered to 90 individuals, including 30 clinicians (ie, 20 physicians, 10 NPs) and 53 support staff (ie, administrators, medical assistants, billing specialists, and social workers). Over 6 years, there was a 92% Hub retention rate; 1 EAE Hub exited the collaborative because of personnel turnover. Engaging the EAE Hub teams in partnership, intensive training, and monthly continuing education has supported excellent retention of Hub sites and sustainability of the system.

Over 6 years, a total 2076 children were evaluated across the EAE Hub system. Of 706 PCPs making referrals, 36% had received developmental screening technical assistance from the EAE Hub team, suggesting that educational efforts spread beyond those who received direct technical assistance. EAE Hub evaluations increased over time as Hubs became established in communities and more sites were added across the system (see Fig 1B). By 2018, ∼72% of expected ASD diagnoses in their respective regions and 15% of expected ASD diagnoses statewide were made at EAE Hubs (see Supplemental Information and Supplemental Fig 3).

Thirty-three percent of the children evaluated received a diagnosis of ASD (see Table 2). In 8% of cases, evaluation results were equivocal, and the EAE Hub clinician was unable to make a definitive determination of ASD diagnostic status. Although the EAE Hub model mandated referral to a specialty diagnostic center for equivocal cases, limitations in our capacity for follow-up data collection precluded confirmation that these children received further evaluation, and, thus, definitive diagnostic status for this group of children is unknown.

Of all children evaluated (ie, regardless of ASD status), 72% met diagnostic criteria for global developmental delay (GDD), defined as delays in ≥2 developmental domains on the basis of ASQ-3 and/or clinical judgment. Sixteen percent of children met neither criteria for ASD nor GDD; 89% of these children were identified as having one or more developmental, behavioral, or medical concern warranting follow-up or intervention (see Supplemental Table 5). Together, these findings suggest that even those not diagnosed with ASD were likely to benefit from evaluation.

Across the EAE Hubs, the mean age at evaluation was 30 months (see Table 3), significantly less than the national average of 4 to 5 years of age⁸  and consistent with existing reports of community-based diagnostic models.^24–27  Compared to historical 9- to 12-month wait times estimated across tertiary outpatient clinics, the median latency from referral to EAE Hub evaluation (ie, wait time) was 62 days. This finding of improved access through implementation of ASD evaluation in the primary care setting has been found across several smaller studies.^26–29  Decreasing wait times for evaluation services provided in children’s local communities has the important potential of increasing access to early intervention and supportive services.

To determine if frequency of ASD diagnostic outcome, age at evaluation, and wait time differed across EAE Hub sites, a series of exploratory analyses were conducted. For these analyses, 11 of 12 EAE Hubs were included; 1 EAE Hub developed in 2018 was excluded from analysis given the small number of evaluations conducted (n = 4). The frequency of ASD diagnosis varied significantly (P < .001) across site (see Table 2). There was also a significant difference in age at evaluation (P < .001) and wait time (P < .001) across sites (see Table 3). Wait time at the EAE Hub within the Indiana University School of Medicine’s academic outpatient clinic (ie, Hub 6) was significantly longer than the average at all other EAE Hubs combined (P < .001).

Findings regarding differences in the number of children diagnosed with ASD, age at evaluation, and wait time across EAE Hub sites are notable. Previous research has documented the impact of health care provider knowledge and behavior on referrals for ASD evaluation.^43–47  It is likely that referring PCPs may have variable experience with the heterogeneous ASD phenotype as well as different thresholds of concern that prompt referral, both contributing to differences in age and diagnostic profiles of children evaluated in the EAE Hubs. Additionally, some referring PCPs may use the EAE Hub system more broadly (ie, for children without clear ASD symptoms but with other developmental and/or behavioral concerns), thus skewing the number of children diagnosed with ASD at some sites. Hub-specific factors such as catchment area population size and site capacity are likely to account for variable wait times.

Although the development of the EAE Hub system represents a significant advance in improving access to timely ASD evaluation, such statewide efforts have many challenges, and interpretation of our outcomes must be considered in the context of several limitations. First, although our clinical pathway was developed from a well-accepted evaluation protocol²⁵  and involved intensive practice-based training and performance feedback from experts, there was no independent ASD evaluation from which to evaluate diagnostic accuracy or determine diagnosis for those with equivocal diagnosis. We must also understand how child and family sociocultural factors affect access and outcomes through collection of demographic data. Together, these efforts will be critical in further evaluating the validity of such a statewide system. In addition, although more young children are now receiving ASD screening and evaluation in their local communities, we did not have reliable statewide baseline measures from which to evaluate system impact. We also cannot draw conclusions regarding the rate of referral for evaluation for those children who screen positive for ASD or determine if an earlier diagnosis results in earlier entry into intervention and how this may impact child outcome.

To our knowledge, this is the largest published report on the development and implementation of statewide system for early ASD screening and diagnosis to date. We offer lessons learned and key system successes, challenges, and future directions for other regions facing similar neurodevelopmental access issues that may wish to adopt and expand the EAE Hub model.

A key ingredient for EAE Hub system success was committed interdisciplinary planning and ongoing leadership. The Department of Pediatrics prioritized this pediatric public health need and dedicated time, resources, and faculty expertise to this effort. An internal needs assessment drove system development planning, and an interdisciplinary team of subspecialists, family advocates, and general pediatricians met weekly (at 7 am) to carefully construct and debate a statewide approach to improving access to ASD evaluation. Faculty committed effort above and beyond their existing clinical and academic duties to participate in broad leadership workgroups that determined the scope, process, and funding for system development.

From the beginning, we aimed to cultivate strong partnerships with EAE Hub clinicians and their organizations, and these relationships have been the foundation of sustainability. Over 6 years of implementation, one of our most significant insights has been the importance of identifying a pediatric champion (typically a medical doctor in a leadership role) at each Hub site. This champion served to coordinate site-specific EAE Hub services and advocate for the importance of the system at the organization and community level. In addition, these champions were invaluable in fueling connections between the EAE Hub, our central leadership team, and local community advocates and organizations. Having relational connections in and across communities is necessary to most effectively support children and their families in accessing needed services.

A crucial ingredient to system engagement and sustainability was fostering collaboration through the learning collaborative. Through these webinars, our central leadership team focused on nurturing shared pride and ownership of the system among all EAE Hub teams. Sharing quality improvement data underscored the significant impact that each team and our collective system made. The webinars also allowed for regular problem solving of issues such as challenging clinical cases, insurance reimbursement, and service navigation. The central leadership team was able to keep a pulse on system quality and management issues that required follow-up.

Providing training in ASD evaluation and ongoing maintenance of skills to a large group of PCPs requires significant investment. The selection of faculty with requisite expertise and funding of their time and travel to the EAE Hubs to provide on-site training proved challenging. Over time, EAE Hub clinicians and staff, including those involved in supporting service, billing, and quality improvement efforts, retired from or left their institutions. Turnover in personnel created disruption in system operations and capacity as well as demands for training new team members. Periodic formal continuing education and reevaluation of diagnostic accuracy and fidelity to the EAE Hub model is critical for quality assurance. Given time and funding constraints, we were not able to invest in these important efforts from system inception, although we suggest that others who undertake adaptation of this model strive to build and fund this infrastructure from the start.

The development of initial system outcome measures was focused around goals of decreasing evaluation wait times and lowering the age of ASD diagnosis. Yet there are critical downstream impacts that must be measured to further understand the significance of this statewide system. For example, understanding whether early diagnosis leads to cascading effects (including earlier enrollment in intervention, improved child outcomes, reduced burden on the educational system, and lowered lifetime costs) will be critical to further system dissemination and funding. Collaboration with statewide agencies (including Birth-to-Three programs, public school systems, intervention agencies, and health care financing organizations) is one method for systematically collecting these types of data.

There has been inconsistent insurance reimbursement for ASD evaluation services, which strains individual EAE Hub organizations. In addition, payment for 90-minute EAE Hub evaluations is often lower than what would be provided for a higher number of routine office visits. Clinician productivity and reimbursement requirements vary by organization type, and, although primary care clinics set within larger health networks may be able to bear some financial burden in support of addressing a critical pediatric need, this is unlikely to be the case for smaller practices.

An additional hurdle has been navigating significant changes in what insurers deem a valid ASD evaluation. For example, some insurers are now mandating inclusion of specific assessment tools (ie, Autism Diagnostic Observation Schedule, Second Edition) to authorize ASD intervention services. Yet the use of these tools requires expert diagnosticians and, thus, contributes to problems with access. Our central leadership team has worked directly with Medicaid and other insurers to provide education about the EAE Hub system and developed standard documentation regarding the ASD clinical pathway, including evidence for how the model is aligned with recommended standards for evaluation. Standardization of system processes, including adherence to specified formats for evaluation reports and insurance appeals, as well as deepening partnerships with insurers is likely to benefit this continued effort.

In the current health care climate, the time and capacity of primary care clinicians and their teams are continuously stretched. For most EAE Hub clinicians, efforts related to system participation (including service delivery) account for <10% of their practice, and, as such, they must balance demands from their many competing roles. Participation in this statewide effort without any direct funding (eg, for additional support staff or indirect costs associated with office space, patient billing, clinician and staff training time, and data collection efforts) creates a significant burden on individual EAE Hub clinicians and their organizations. One potential avenue to reduce burden may be to develop a shared infrastructure of support with a state department of health or similar agency. This partnership could potentially allow for the use of funding to support the work of individual EAE Hubs and build capacity for the collection of comprehensive longitudinal outcome data to evaluate system impact. These efforts must be priorities for ensuring sustainability, advancing rigorous system evaluation, and improving pediatric population health.

Developing a tiered system of developmental screening and early ASD evaluation is feasible in a geographic region facing significant health care access problems. Through targeted delivery of developmental screening technical assistance, community outreach, medical education, and intensive practice-based training, large numbers of young children at risk for ASD can be identified and evaluated in the local primary care setting. Although further rigorous testing of the EAE Hub system is warranted, our findings suggest that this model has potential for further expansion and dissemination to other states facing similar neurodevelopmental health care system burdens. Future directions must include evaluation of diagnostic accuracy of the system, an effort that is in progress, as well as measurement of provider and family satisfaction, child intervention enrollment and outcomes, and cost of implementation.

We thank Angela Paxton, Mary Delaney, Mary Jo Paladino and Maureen McAteer, DO, for their invaluable contributions to this project. We are grateful to the clinicians and supporting staff at each of the EAE Hubs for their collaboration, service to children and families, and submission of data. Without local pediatric champions who are committed to providing Indiana’s children with this critical community-based service, the implementation of the EAE Hubs would not have been possible. Finally, we thank the Indiana Chapter of the American Academy of Pediatrics for their partnership in this work.

ASD

autism spectrum disorder

ASQ-3

Ages and Stages Questionnaire, Third Edition

DSM-5

Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition

EAE

Early Autism Evaluation

GDD

global developmental delay

MCHAT-R/F

Modified Checklist for Autism in Toddlers, Revised with Follow-up

NP

nurse practitioner

PCP

primary care provider

START-ED

Screening Tools and Referral Training-Evaluation and Diagnosis

STAT

Screening Tool for Autism in Toddlers and Young Children