Effectiveness of Direct Admission Compared to Admission Through the Emergency Department: A Stepped-Wedge Cluster-Randomized Trial

We conducted a stepped-wedge cluster-randomized trial, randomizing 58 primary care and 11 urgent care clinics across 3 health systems to begin the DA program at 1 of 4 time points during the study period, February 1, 2020 to April 30, 2023 (Fig 1). In addition to analytic benefits, this design was chosen to provide all participating clinics access to the DA program (which was consistently requested by clinic leadership and foundational to their willingness to participate in the study) while also allowing time to sequentially educate clinicians and conduct a concurrent evaluation of program implementation and uptake.^27,28  Participating health systems were selected based on: (1) very low direct admission rates at baseline, with DAs generally accepted only for neonatal hyperbilirubinemia and/or failure to thrive, (2) geographic and structural diversity, and (3) institutional support to increase DA rates and participate in the study. A subset of clinics within each hospital’s catchment area were selected for participation based on their historical number of patients referred for hospitalization (Prioritizing higher volume clinics), geographic proximity to the hospital (Prioritizing clinics within 20 miles of participating hospitals), and commitment of clinic leadership to trial the DA program.

Implementation sites included 2 high-volume freestanding children’s hospitals (Hospitals A and C) and 1 community hospital with an annual volume of 745 pediatric stays (Hospital B). At Hospital A, 34 participating clinics represented 2 independently-owned and 2 hospital-owned practice groups that were part of a quality of care alliance with the hospital; at Hospital B, 17 participating clinics represented 3 independently-owned practice groups; and at Hospital C, 18 participating clinics were owned by the health system. Before study initiation, practice groups agreed to clinic-level randomization, stratified such that one-quarter of clinics within each health system began the DA program at each time point. Randomization was performed before trial start by the central research team. The Dartmouth College Committee for the Protection of Human Subjects approved this study and all sites ceded review to Dartmouth. The trial was registered (ClinicalTrials.gov NCT04192799) and reporting follows the CONSORT extension for stepped-wedge cluster-randomized trials.²⁹ 

The DA program included 5 core elements, implemented as a quality improvement program using existing resources and infrastructure: (1) education and tools for referring and accepting clinicians; (2) direct communication between referring and accepting clinicians via a central transfer center; (3) verbal and written instructions for families regarding the DA process; (4) rapid evaluation of the patient upon hospital arrival; and (5) timely initiation of clinical care. Clinician education was facilitated by implementation sites in the month preceding each clinic’s DA program start date. Program participation did not require clinicians to make DA referrals but provided tools and education to support this admission approach.

Children < 18 years of age who received care from a participating clinic were included in the study if they had an admission diagnosis of gastroenteritis, dehydration, SSTI, UTI or pyelonephritis (hereafter UTI), pneumonia, influenza, or other viral infection. These diagnoses comprise approximately 25% of unplanned pediatric admissions and were identified a priori as appropriate diagnoses to consider for DA.^30,31  Children with planned admissions, those admitted to an ICU, and those transferred from other hospitals were excluded. Study eligibility was determined based on daily electronic medical record (EMR) review by research teams at each site; all children with an eligible admission diagnosis and a primary care clinician at a participating clinic or a referral from a participating urgent care clinic were included. Children admitted before their clinic’s DA program start date were excluded from the patient-level analysis but were included in a program-level analysis as described below.

Our primary exposure variable was admission source: DA or ED admission. Clinical and demographic characteristics assessed included child age, gender, race, ethnicity, preferred language, primary payer (Medicaid, commercial, other), admission diagnosis, co-occurring chronic medical conditions, abnormal admission vital signs (Supplemental Table 5), and abnormal initial nursing assessment, defined as ≥1 abnormal assessment variables (capillary refill time > 3 seconds, abnormal color, and/or abnormal behavior). Race and ethnicity were collected using hospitals’ standard operating practices, extracted from the EMR, and included given past research showing racial and ethnic disparities in DA rates.^9,26  Co-occurring chronic medical conditions were identified using the Pediatric Medical Complexity Algorithm.^32,33 

The primary study outcome was timeliness of clinical care, prioritized because past studies have shown that timeliness is highly valued by caregivers, and clinicians describe potential delays in care as one of the main reasons they hesitate to refer or accept DAs.^11,34  This was operationalized using 3 measures: (1) time of initial clinical assessment (first time when ≥3 vital signs were documented; if missing, time of initial nursing assessment); (2) time of initial diagnostic testing, including laboratories and/or imaging and all tests performed per “standing orders” in EDs, and (3) time of initial therapeutic management, including fluids provided via nasogastric tube, gastrostomy tube, or intravenously; antibiotics; pain medications (excluding acetaminophen or ibuprofen alone); and/or other medications. These measures were calculated in minutes and assessed using EMR time stamps from the time of hospital arrival until 6 hours after arrival on the inpatient unit. Several prior studies have demonstrated the accuracy of EMR time stamp data,^35–37  and measure accuracy was assessed using time motion data at each site before study initiation (Supplemental Methods). Data were manually extracted by research coordinators and entered into REDCap. Data integrity was assessed monthly by the central research team.

Prespecified secondary outcomes included caregiver-reported experience of care and post-admission clinical deterioration, defined as transfer to the ICU and/or rapid response call within 6 hours of admission. Family experience of care was assessed on a 0 to 1 scale using the Pediatric Hospitalization Admission Survey of Experience measure, which evaluates 4 domains: patient and family engagement, information sharing, effectiveness of care delivery, and timeliness of care.³⁸  This measure was collected via REDCap electronic survey after receipt of verbal informed consent. The survey was available in 5 languages (Arabic, English, Nepali, Somali, and Spanish). Eligibility criteria included caregiver ability to complete the survey in an available language, presence with their child at the time of admission, and availability 6 to 72 hours after admission to engage with a research coordinator. Surveys were completed during the hospitalization so that responses were not influenced by hospital discharge processes.

Exploratory outcomes, recommended for inclusion by our study’s advisory board, included: (1) number of diagnostic tests completed during the admission process (Supplemental Table 6), (2) proportion of children for whom laboratory tests were collected at >1 time points during the admission process, (3) length of stay (in hours), (4) high-turnover hospitalization, defined as discharge ≤ 24 hours after admission,^39,40  and (5) readmission within 30 days of discharge.

The study protocol was published before study initiation, including details regarding sample size and statistical power.⁴¹  The onset of the coronavirus disease 2019 (COVID-19) pandemic resulted in several protocol changes including: (1) cessation of in-person survey collection during the early pandemic stage, at which time remote collection procedures were developed and Institutional Review Board-approved. Data collection subsequently followed each site’s COVID-19 policies; (2) exclusion of study participants infected with COVID-19; and (3) extension of the study duration to reach the prespecified sample size. A priori, each study block was projected to be 6 months long. Given slower than anticipated accrual, the third and fourth blocks were extended to 9 months in duration; funding agency approval was subsequently received to extend the fourth block by an additional 9 months at hospitals B and C. During the study period, 4 clinics closed unexpectedly; 2 that closed before DA program start were replaced with other clinics to maintain balance across groups. In June 2022, the pediatric unit at Hospital B closed and participant enrollment ceased at that site.

As the primary analysis, we used an observational approach to compare outcomes in children admitted directly to those admitted via EDs after each clinic’s respective program start date (Fig 1) as specified in our study protocol. This approach was prioritized because it was adequately powered for all proposed analyses and allowed for a pragmatic comparison of DA and ED outcomes in children affiliated with clinics where the DA program was available. We first calculated descriptive statistics to compare baseline characteristics and outcomes between children who were directly admitted to those admitted through EDs, including t tests for differences in means and Wilcoxon rank sum tests for differences in data distributions. Given the non-normal distribution of the timeliness measures, including the non-negativity of the outcome and skewed right tail, we used generalized linear models with a logarithmic link function and γ distribution to compare timeliness outcomes, adjusting for hospital fixed effects and a random effect for practice group. We represented the effect of DA versus ED admission as the ratio of the change in the expected outcome in the DA group compared with the ED group (a natural effect based on the linear predictor of the assumed model) and in terms of the counterfactual average difference in the timeliness measures between the DA and ED groups in minutes (average marginal effects). Overall, family experience of care was modeled using a linear regression model, whereas the binary-valued domain scores and clinical deterioration were modeled using logistic regression. In keeping with the stepped-wedge study design, key predictors in the statistical models were practice group and time period (trial step), in addition to covariates shown in Table 1, excluding vital sign and nursing assessment variables given their potential to be on the causal path (ie, consequences of having undergone DA versus ED admission, see Appendix 1 for model specifications). To test our hypothesis that children with UTI or pyelonephritis, SSTI, and/or co-occurring complex chronic medical conditions would experience the greatest benefits from DA, we evaluated group-level heterogeneity of treatment effects. These analyses involved estimation of analogous models to those described above, augmenting each model with a predictor for the interaction between diagnosis and DA.

In addition to the patient-level analyses described above, we also evaluated the “program-level” or intent-to-treat effect of the DA program, which examined whether children affiliated with clinics that had been randomized to begin the DA program at their time of admission had different outcomes than children from clinics that had not yet initiated the program. The primary exposure variable was the clinic-level time-varying predictor indicating whether the DA program had launched at a given site (Pre-DA program compared with post-DA program). Regression analysis and covariate inclusion otherwise followed the above-described approach (See Appendix 1 for model specifications). All analyses were conducted using R version 4.1.2 and SAS 9.4; testing was two-sided and P values < .05 were considered statistically significant.

In total, 2599 children met eligibility criteria for the study, including 171 (6.6%) who were admitted directly and 2428 (93.4%) who were admitted through EDs. Of these, 145 DAs and 1852 ED admissions occurred after each clinic’s respective DA program start date and were included in the patient-level analysis (Supplemental Fig 2).

There were several significant differences in the distribution of baseline characteristics between children admitted directly and those admitted via EDs (Table 1). Children admitted directly were less frequently insured by Medicaid (n = 68, 46.9% versus n = 1256, 67.8% of ED admissions), less likely to be Black and/or African American (n = 28, 19.3% versus n = 665, 35.9% of ED admissions), less likely to have co-occurring complex medical conditions (n = 8, 5.5% versus n = 477, 25.8% of ED admissions), more likely to be admitted with SSTIs (n = 58, 40.0% versus n = 404, 21.8% of ED admissions), and more likely to be admitted at Hospital C (n = 135, 93.5% versus n = 1260, 68.0% of ED admissions). Mixed-methods analysis exploring variation across hospitals in DA program uptake is described separately.⁴² 

Differences in the outcome distributions between children admitted directly and those admitted through EDs are shown in Table 2. Although mean time to initial clinical assessment was similar between the groups, the median initial clinical assessment time was shorter in children admitted via the ED. The mean time to initial diagnostic testing was the same in both groups at 134.1 minutes, but the median time was shorter in children admitted via the ED. Both the mean and median times to initial therapeutic management were shorter in children admitted directly.

In adjusted regression analysis, the time to initial clinical assessment was estimated to be 12% longer in children admitted directly compared with children admitted through the ED (estimate: 1.12, 95% confidence interval [CI]: 1.06 to 1.19). The corresponding average marginal effect suggests that if all patients had been admitted via DA versus the ED, time to initial clinical assessment would have increased, on average, by 3.1 minutes (95% CI: 1.7 to 4.5, Table 3). There was no significant difference between the groups in adjusted mean time to initial diagnostic testing. In contrast, children admitted directly had, on average, a 26% faster time to initial therapeutic management (estimate: 0.74, 95% CI: 0.65 to 0.85), which translates to an estimated average reduction of 49.6 minutes (95% CI: −68.9 to −30.3) from the expected time if all patients received DA versus ED admission.

Family experience of care data were available for 889 (44.5%) participants in the patient-level analysis; reasons for nonparticipation included inability of the research coordinators to connect with caregivers to request their participation (n = 656) and survey ineligibility (most often because of language barrier or COVID-19-related factors, n = 79). Among eligible caregivers offered the survey, the response rate was 70.4% (889 of 1262). Characteristics of children with and without survey data are shown in Supplemental Table 7, and individual item scores are shown in Table 4. There were no significant differences in overall experience of care reported by caregivers of children admitted directly and those admitted via EDs (Table 3). However, caregivers of children admitted directly reported significantly higher scores in the effectiveness of care delivery domain. Post-admission clinical deterioration was rare, occurring in 0.8% (n = 16) of the study sample with no significant differences between the DA and ED admission groups. In adjusted analyses findings were similar, with caregivers of children admitted directly having twice the odds of top-box effectiveness of care delivery scores (odds ratio = 2.08, 95% CI: 1.20 to 3.60) and no other significant differences in family experience of care or clinical deterioration.

In analysis of exploratory outcomes, we observed that children admitted directly received significantly fewer diagnostic tests and were significantly less likely to receive >1 set of laboratory tests during the admission period (35.9%, n = 52 vs 76.8%, n = 1422 of ED admissions). Length of stay, the proportion of hospitalizations ≤ 24 hours, and 30-day readmission rates did not differ significantly between the groups.

Heterogeneity of treatment effects analysis identified significant interactions between admission source and admitting diagnosis, with directly admitted children with UTI experiencing a disproportionately longer time to initial clinical assessment (P value for interaction term < .001) and directly admitted children with SSTI experiencing a disproportionately shorter time to initial diagnostic testing than children admitted via the ED relative to children with other diagnoses (P value for interaction term = .001, Supplemental Table 8). There were no significant interactions between DA and co-occurring complex chronic medical conditions (Supplemental Table 9).

In the program-level analysis, we observed several significant differences between children affiliated with clinics that had and had not begun the DA program, including differences in race, ethnicity and admission diagnosis (Supplemental Table 10). In both descriptive and regression analyses, time to initial clinical assessment was higher among children affiliated with clinics that been randomized to begin the DA program at their time of hospitalization compared with children affiliated with clinics that had not yet begun the program (Supplemental Tables 11 and 12); regression analysis estimated an increased time of 2.9 minutes (95% CI: 0.2 to 5.6) on average. Time to initial diagnostic testing and therapeutic management, family experience of care, and post-admission clinical deterioration were not significantly different in children admitted from clinics that had, and had not, launched the DA program at the time of their admission (Supplemental Table 12).

In this clinic-randomized study of a newly-developed DA program, there were no significant differences in rates of clinical deterioration or caregiver-reported experience of care in children admitted directly compared with children admitted through EDs after program initiation at their clinic. Our patient-level analysis shows that the adjusted time from hospital arrival until initial clinical assessment was approximately 3 minutes faster for ED admissions, on average; the corresponding estimated mean effect for time until initial therapeutic management was 49 minutes faster for DA. Length of stay and readmission rates were similar, and children admitted directly received considerably fewer diagnostic tests than children admitted through EDs. Taken together, these results suggest that, in comparison with ED admission, DA appears to be both safe and acceptable to families.

In a past survey of pediatric hospitalists, 50% reported that more children should be directly admitted, whereas 50% disagreed, supporting trial equipoise.¹¹  Clinicians cited 3 main concerns when describing hesitancy to accept DAs: unnecessary hospitalization if a patient arrives “less sick” than anticipated, delayed provision of clinical care, and unanticipated clinical deterioration before clinician assessment.¹¹  The results of this study do not validate these concerns. Although the mean time to initial clinical assessment was marginally increased in children who were admitted directly, estimated mean time to initial therapeutic management was considerably faster for directly admitted children. Further, clinical deterioration was rare and not observed with any greater frequency than in the ED admission group, and there was no significant difference in the proportion of hospital stays < 24 hours, a measure historically used to estimate “potential unnecessary” hospitalizations.⁴⁰  These findings also align with past research, including several studies demonstrating the potential of DA programs to reduce the time to cardiac critical care in adults.^19,22,43 

Proponents of DA advocate that this admission approach has the potential to improve family experience of care, improve care coordination, and increase health care value.¹¹  Although overall family-reported experience of care did not differ between caregivers of children admitted directly to those admitted through EDs, caregivers of directly admitted children did report significantly higher scores on the effectiveness of care survey domain, reflecting better scores on survey items related to medical errors and wait times. Although this study did not directly evaluate health care value, the lower number and frequency of diagnostic tests in the DA group was striking. Further research is needed to evaluate reasons for these differences, particularly given growing evidence about the potential harms of overusing diagnostic tests.^44,45 

It is important to acknowledge that the COVID-19 pandemic began 6 weeks after the launch of our study, heralding major health system changes, including staff shortages, changes to hospitals’ screening and personal protective equipment policies, temporary closures of primary care clinics, and transitions to telehealth. Pandemic-associated lower ED volumes may have improved the relative timeliness of ED care, whereas COVID-related primary care clinic changes may have decreased DA access. These health system changes provide further justification for our a priori decision to prioritize the patient-level comparison of DA and ED admission after program launch at each site. In our program-level analysis, we found that average times to initial clinical assessment was modestly longer in clinics that had initiated the DA program. We hypothesize that observed differences are related to secular trends as opposed to the launch of the DA program itself, as DAs comprised a very small fraction of hospitalizations overall, and longer times were also observed after DA program implementation among children admitted via EDs.

It is important to note that the patient-level analysis described herein applied an observational design to evaluate outcomes after DA program implementation. Although this approach enabled direct comparisons between children admitted directly and via EDs and adjustment for important demographic and clinical covariates, causal inference is limited because of potential unmeasured confounding. Although the program-level analysis could theoretically overcome this limitation, low uptake of the DA program challenges interpretation of results. Several populations were under-represented in the DA group, including children who were Medicaid-insured, Black or African American, and those with co-occurring complex chronic conditions. These populations were also under-represented in the survey measures. These findings are consistent with prior retrospective cohort studies and, as shown in our mixed-methods program evaluation, may have been associated with several factors including family preferences, historical and ongoing discrimination, and differential adoption of the DA program by clinicians.^42,46,47  These findings serve as a call to action to adopt a health equity lens in future DA program implementation efforts. It is also important to note that this study was limited to children with prespecified admission diagnoses; although these diagnoses represent a substantial fraction of pediatric hospitalizations, findings may not be generalizable to all admission diagnoses.

In this study across 3 health systems, DA was associated with more rapid therapeutic management in children hospitalized with common medical diagnoses and superior parent-reported effectiveness of care delivery, with no significant differences rates of clinical deterioration. DA programs have the potential to decrease ED crowding, streamline transitions into hospital, and provide timely patient care.

We thank the research team members who extracted data and collected surveys throughout the project: Olivia Bouchard, Casey Halle, Rebecca Harvey, Tess Mitchell, Lauren Pavlechko, and Sarah Podlasiak (Research Coordinators at Nationwide Children’s Hospital), Zachary Henry (Research Coordinator at the University of Pittsburgh School of Medicine), and Kathleen Sanders (Research Coordinator at Providence Regional Medical Center Everett); and members of our national Direct Admission Advisory Panel and the Direct Admission Leadership Teams at each site; these groups met throughout the project and provided invaluable guidance and support.

DA

direct admission

ED

emergency department

EMR

electronic medical record