Direct Admission Program Implementation: A Qualitative Analysis of Variation Across Health Systems

To compare the effectiveness of DA and ED admission, we implemented a DA program at 3 geographically and structurally diverse health systems in which, before program implementation, the ED served as the admission source for almost all hospitalizations. Participating sites had institutional support to begin the program and at baseline, generally only accepted DAs for neonatal hyperbilirubinemia or failure to thrive. Health system characteristics are summarized in Table 1. Detailed protocol information has been published previously; the key points are summarized below.¹² 

The DA program was implemented in February 2020 by using a stepped-wedge design; 69 primary and urgent care clinics were randomly assigned to 1 of 4 time points to initiate DA referrals (Fig 1). Randomization was stratified such that an equal proportion of clinics in each health system began the DA program at each time point; clinician education and the distribution of program materials occurred in the month preceding each clinic’s start date. Each time block was originally anticipated to be 6 months long, but the latter 2 blocks were extended to allow more time to reach prespecified enrollment goals. Hospital A completed participant enrollment in July 2022, as intended, and enrollment was planned to extend further at hospitals B and C. However, in June 2022, the pediatric unit at hospital B closed unexpectedly.

The DA program included 5 core elements: (1) education and tools for referring and accepting clinicians, (2) direct communication between referring and accepting clinicians via a central transfer center, (3) verbal and written instructions for families regarding the DA process, (4) a rapid evaluation of the patient on hospital arrival, and (5) the timely initiation of clinical care. Clinician education included guidance on how to place DA referrals, eligible diagnoses, and clinical characteristics to discuss during referral calls, as well as how to communicate with families regarding DA referrals.

Patients who received care from one of the participating clinics were included in the study if they were <18 years of age, had an admission diagnosis of gastroenteritis, dehydration, skin and soft tissue infection, urinary tract infection or pyelonephritis, pneumonia, influenza, or a viral infection not otherwise specified and were either admitted directly from a participating clinic or via the ED. These diagnoses comprise ∼25% of unplanned pediatric admissions and, before trial initiation, were identified as appropriate for DA.^13,14  Children with planned admissions, those admitted to an ICU, and those transferred from other hospitals were excluded. Study eligibility was determined on the basis of daily electronic medical record (EMR) review, and all eligible children were included in the overall study. Timeliness of care was the primary outcome, and secondary outcomes included postadmission clinical deterioration and caregiver-reported experience of care (results forthcoming).

As described in the protocol, in this study, we applied the RE-AIM framework to evaluate the reach, effectiveness, adoption, and implementation of the DA program.^12,15  To assess program uptake quantitatively, the number and proportion of hospitalizations that originated as DAs were calculated in 2-week intervals and reported as cumulative DA rates for each participating clinic and health system. Semi-structured interviews were conducted with caregivers and clinicians from May 2020 to April 2023. Interviews and analysis were conducted throughout the study period, and our team published 2 papers using a subset of data while the study was ongoing; one was focused on coronavirus disease 2019 (COVID-19),¹⁶  and the second was focused on interprofessional trust.¹⁷ 

We used previously published RE-AIM definitions to inform interview questions tailored for caregivers, inpatient clinicians, and outpatient clinicians (Supplemental Table 1, Supplemental Information).¹⁸  We piloted and revised the guides to optimize response comprehensiveness; the guides were modified iteratively as concepts emerged. Semi-structured interviews were conducted in person with English-speaking caregivers during their child’s hospital stay. Local study team members purposefully sampled potential participants to reflect diversity in admission source, primary payer, race, ethnicity, and time period. Clinicians were interviewed via video conference and were purposefully sampled to reflect varied clinical roles in outpatient and inpatient settings, including nurses (eg, bedside, care management, and transfer center), residents, and attending physicians.

Interviews were continued until thematic saturation was achieved within each health system and stakeholder group and the number of interviews proposed to the funder a priori were completed. Participant sociodemographic characteristics were self-reported; race and ethnicity were included, given past research revealing disparities in access to DA.^10,19  Verbal consent was obtained before each interview, including permission to record audio. Participants received a $50 gift card. Recordings were professionally transcribed and verified for accuracy, and identifiers were removed. The Dartmouth College Committee for the Protection of Human Subjects approved this study and all implementation sites ceded review to Dartmouth. The reporting adheres to the Standards for Reporting Qualitative Research.²⁰ 

A 6-member research team composed of 2 clinician-researchers (JKL and CEM), a parent consultant (CAS), a qualitative research consultant (EJF), a research associate (JT), and a research director (SA) led the thematic analysis using a participatory inductive approach informed by the RE-AIM framework.²¹  Interviews with clinicians and parents were analyzed concurrently. A subset of interviews was double-coded; disagreements were resolved through in-depth discussions and codebook revisions. Subsequently, all interviews were single-coded by using Dedoose 9.0.86 qualitative software, with coding audits by EJF.²² 

After coding completion, the research team reviewed the coded data by site and held a series of meetings to identify site-specific themes. By using axial coding, similar concepts were grouped as themes using RE-AIM domains as a guide. To enhance trustworthiness, we performed member checking with multidisciplinary stakeholder panels at each site. We then conducted thematic analysis sessions with the full authorship group to discuss variation across sites and stakeholder groups, identifying key concepts and determining if themes served predominantly as barriers or facilitators at each site.^21,23  Relationships between themes and domains were assessed to develop a conceptual framework.

A total of 2599 children were included in the study, including 171 (6.6%) who were admitted directly and 2428 (93.4%) who were admitted through the ED (Table 1). Cumulative DA rates varied 10-fold across health systems, from 0.9% at Site A to 9.3% at Site C (Fig 1).

We conducted 137 interviews, including 46 with inpatient-based clinicians, 38 with outpatient-based clinicians, and 53 with parents. Of the parents interviewed, 27 experienced ED admissions and 26 were admitted directly. Major themes within the domains of reach, effectiveness, adoption, and implementation are provided in Fig 2, along with summaries of key concepts, representative quotes, and graphical representations of how the themes varied as barriers and facilitators across sites.

Within this domain, emergent themes included (1) family access to ambulatory care, (2) clinician recommendations regarding where to seek care, and (3) family understanding and experience of the health system.

Ambulatory care access, including factors related to primary care provider (PCP) availability and families’ access to transportation, child care, and work flexibility, was a barrier to DA program reach across sites. An outpatient clinician stated that “Some families don’t have good transportation…A lot of families have other kids they need to care for…Trying to manage that can be a barrier for a lot of families.” [ID#174]

Clinicians’ perceptions of a patient’s appropriateness for DA was an important concept within this domain, and families reported following through with their clinicians’ recommendations regarding where to seek care. At site A, clinicians often directed families to seek care at the ED because they saw “very few cases that would require…inpatient-level admission,” [ID#184] and they valued the role of the ED in making hospital admission recommendations.

Family understanding and experience of the health system were identified as a mixed barrier and facilitator at sites A and C and a barrier at B. Although most parents were unaware of an option for DA, families described incorporating their concerns about their child’s illness severity and perception of PCP resources when determining where to seek care. In the words of one parent: “I felt that he needed more care than my doctor could provide. And also, I felt as though if I took him to my doctor, they would tell me to come [to the ED] anyway.” [ID#239]

Within this domain, emergent themes included: (1) family experience of care, (2) timeliness of care, and (3) patient safety.

Families overwhelmingly reported positive experiences with DA. When describing why they preferred DA, families cited avoiding long ED wait times, exposure to other illnesses, and minimizing points of contact with the health care system. As one parent stated: “We’ve gone through the ED before, and it was horrible…[B]eing able to come right up [to the floor]…calmed us down as parents.” [ID#253] Positive family experience of care was described as “incredibly meaningful” to clinicians across sites, who described this as a major facilitator of the DA program.

The timeliness of care for DAs was a mixed facilitator and barrier at sites A and B, in which clinicians described timely access to intravenous placement as a challenge. Similarly, patient safety was a mixed facilitator and barrier across sites, with a majority of clinicians describing the program as having neither positive nor negative impacts on patient safety. Instead, respondents attributed a lack of adverse events to a “cautious and conservative” approach to DA, describing that the program’s inclusion of diagnoses that infrequently require emergent intervention “helped to maintain” the safety of patients.

Within this domain, emergent themes included (1) clinician familiarity and engagement with the DA program, (2) clinician perspectives about DA efficiency, (3) fear of clinical deterioration, (4) group norming and adoption of change, and (5) relationships and trust between inpatient and outpatient clinicians.

Respondents described that clinicians’ likelihood of referring a patient for DA was influenced by their awareness of the DA program, understanding of the DA process, and knowledge of the accepting hospital. Clinician perspectives about DA efficiency were a barrier at sites A and B, in which inpatient clinicians reported that unclear arrival times of patients could disrupt workflow, and outpatient clinicians described that arranging for DA increased their workload. Although outpatient clinicians at site C agreed that DAs created extra work, they described the benefits of DA as outweighing this issue: “Direct admissions create a small amount of work versus sending to the ED…But I think most of the time, the nurses recognize that it’s better for the family to avoid the ED.” [ID#135]

Fear of clinical deterioration was a barrier to DA across sites. Inpatient clinicians expressed concern about the inpatient unit’s ability to provide higher acuity care to patients who arrived “sicker than the referring provider understood or recognized at the time of their assessment or got sicker on the way [to the hospital].” [ID#164] Outpatient clinicians identified a lack of access to laboratories and other diagnostic resources as contributing to uncertainty regarding the patient’s condition.

Group norming and the health system’s culture of change were integral to program uptake. Central to this was a willingness to change cultural norms around DA, which proved to be an important facilitator at sites B and C. According to one inpatient clinician at site C: “I think that we tend to be pretty supportive of quality improvement projects…we’re accustomed to new things and change. And because of that, I think it was pretty easy to get everyone on board.” [ID#181]

Relationships and trust between inpatient and outpatient clinicians were facilitators of DAs across sites, with previous relationships between accepting and referring clinicians creating “mutual respect and understanding of each other’s clinical skills.” [ID#157] As stated by one clinician, these “subtle nuances are helpful from a trust standpoint.” [ID#184]

Within this domain, emergent themes included (1) hospital capacity and resource availability, (2) community-to-hospital communication systems, (3) inpatient care team coordination, (4) education and written program materials for clinicians, and (5) education and communication with families.

Across sites, hospital capacity and resource availability were barriers to the consistent availability of DA, particularly at site B. Clinicians described staffing shortages and limited bed availability, which were caused or exacerbated by the COVID-19 pandemic, and surges in other respiratory illnesses. In the words of one inpatient clinician: “Several factors, COVID, number one, and the nursing shortage, number two, just completely derailed this project…there was much less potential to accept any DA, project or no project.” [ID#163]

Community-to-hospital communication systems, including the transfer center and the EMR, were also integral in facilitating DAs. At sites A and B, transfer center efficiency was a consistent barrier, with wait times described as ranging from 20 minutes to 2 hours. Separate EMR systems in the inpatient and outpatient settings at site A created a “dislike of DAs most of the time.” [ID#109] Alternatively, at site C, clinicians cited the transfer center’s efficiency coupled with a shared EMR system as critical to their program’s success. Communication and coordination between inpatient clinicians was similarly an important facilitator of DA and proved to be a barrier at site A, a mixed barrier and facilitator at site B, and a facilitator at site C.

Education and written program materials for clinicians were DA facilitators at site C and mixed barriers and facilitators at sites A and B, in which clinicians described wanting more frequent reeducation because of staff turnover caused by COVID-19. Family education and communication was also described as a facilitator at site C, in which the DA process was described by families as “smooth,” “easy,” and “streamlined.”

In addition to variation across sites, there was variation in the perspectives, experiences, and priorities of parents, outpatient-based clinicians, and inpatient-based clinicians (Table 2). For example, inpatient clinicians described prioritizing patient safety, whereas outpatient clinicians described prioritizing family experience of care. In the words of one inpatient clinician: “I would argue safety is more important than the patient experience.” [ID#144] In contrast, outpatient clinicians described wanting to avoid long waits for families and additional points of contact with the health care system. One outpatient clinician expressed frustration when their DA referral was denied and described the family ending up in the ED for “five hours and getting all sorts of labs and tests…it was…the things I was trying to avoid.” [ID#179]

Themes within the RE-AIM domains were closely related and had bidirectional influences, as shown in Fig 3. Across sites, we observed that these domains could be positively or negatively correlated. For example, positive perceptions of program effectiveness influenced greater adoption by clinicians. Conversely, a lack of resources for program implementation decreased patient reach.

In this evaluation of a multisite DA program, we observed a 10-fold variation in DA rates across health systems and thematic differences in program implementation, adoption, effectiveness, and reach. Thematic analysis elucidated important relationships between how the program was implemented and how it was adopted by clinicians, its perceived effectiveness by clinicians and families, and the extent to which patients were reached as intended. Although trust between clinicians and positive feedback from families served as consistent facilitators of program uptake, fewer DAs were observed in health systems with more challenges related to transfer center efficiency, hospital resources, and their system’s culture regarding the adoption of change.

Previous studies have revealed that DA rates are highly variable across hospitals and diagnoses, and in this analysis, we have summarized factors that may contribute to this variation.^10,14,19,24  Although performance sites followed a similar approach to implement this DA program, their DA rates varied substantially. As shown in our previous qualitative study, having shared DA guidelines across outpatient and inpatient settings may serve to build “trust-by-proxy” between stakeholders who may not otherwise have preexisting relationships.¹⁷  Beyond guidelines and consistent with past research, in our current analysis, we have identified other key facilitators of program uptake, including effective referral and communication infrastructure and broad buy-in from stakeholders.^25 –28 

Our qualitative analysis revealed that clinician adoption of the DA program was also strongly influenced by group norming and organizational culture, defined most simply as “how things are around here.”²⁹  Dimensions of organizational culture, including interprofessional collaboration and shared perspectives about the value of programs or interventions, have previously been shown to influence program uptake and guideline adherence.^29,30  Correspondingly, organizational culture and readiness for change are key components of the Consolidated Framework for Implementation Research and the Interactive Systems Framework for Dissemination and Implementation.^31,32  Although organizational culture is challenging to change, studies have revealed that targeted interventions can positively influence clinician behavior and patient outcomes.^30,33  These findings underscore the importance of addressing the individual- and system-level factors that support an intentional shift in the norms and values of “how things are done” within an institution.

One strength of this study is its application of the RE-AIM framework, which was developed to apply and evaluate research in “real world” settings.¹⁵  Although the developers of this framework have recommended that qualitative methods be used to assess why and how results are observed, most applications of the framework have relied exclusively on quantitative analyses.¹⁸  Our conceptual framework builds on the previously published RE-AIM Cascade (which reveals unidirectional steps across RE-AIM domains)³⁴  to illustrate the centrality of patient reach and bidirectional relationships between domains. This framework may be applied to future studies to advance our understanding of how these domains influence the implementation of other programs or interventions.

It is important to acknowledge that this study was initiated just before the onset of the COVID-19 pandemic. During the study period, 2 primary care practices at site A closed permanently, and our community hospital site (site B) closed its pediatric unit in June 2022. These health system changes reflect similar trends observed nationally.^35 –38  The COVID-19 pandemic and post-pandemic staffing challenges had tremendous influences on health care access and quality^16,39,40 ; it is beyond the scope of this article to describe further how barriers and facilitators of DA changed over time. Although efforts were made to enhance rigor and trustworthiness throughout the study, interviewees may have experienced response and recall bias, and analysis may have been influenced by confirmation bias.^41,42  Results may not be transferable to other settings, although our inclusion of participants across 3 health systems in different geographic regions may help to mitigate this limitation. Because of resource limitations, interviews were only conducted in English, thereby excluding the perspectives of non-English-speaking families. Additionally, we acknowledge the imbalance across sites in the number of caregiver interviews; we prioritized interviews with caregivers who experienced DA, which resulted in a relatively larger number of interviews at hospital C. Finally, in this study, we did not evaluate the long-term maintenance of the DA program, given the timing of when interviews and analyses were conducted.

Several elements related to health system resources, trust and relationships between clinicians, and organizational culture influenced the implementation of this multisite DA program. These results may be used by health systems to institute or improve their DA systems, as well as to operationalize the American Academy of Pediatrics’ recently published Policy Statement focused on this admission approach.¹¹ 

We would like to extend our gratitude and appreciation to all the parents and clinicians who generously shared their time and experiences with us. We could not have completed this work without them. We are also indebted to the wonderful research team members who conducted these interviews throughout the project, including Olivia Bouchard, Casey Halle, Rebecca Harvey, Tess Mitchell, Lauren Pavlechko, and Sarah Podlasiak (Research Coordinators at Nationwide Children’s Hospital), Amanda St. Ivany (Research Scientist, Dartmouth Hitchcock Medical Center), Kathleen Sanders (Research Coordinator, Providence Regional Medical Center Everett), and Mary Arakelyan (Research Project Manager, Dartmouth Health) for creating Figure 3.

COVID-19

coronavirus disease 2019

DA

direct admission

ED

emergency department

EMR

electronic medical record

PCP

primary care provider