HEROIC Trials to Answer Pragmatic Questions for Hospitalized Children Free

Consider this scenario: an inpatient pediatric team has just finished rounding. As the team walks toward their workroom together, the attending turns to a new intern and asks if she has any questions. She responds that she was struck by the numerous clinical decisions made during rounds for which there were no randomized controlled trial (RCT) data for reference. How many days of intravenous antibiotics are needed for the 2-week-old baby with a urinary tract infection? Should we prescribe steroids for the teenager hospitalized with e-cigarette or vaping use-associated lung injury? Will daily c-reactive protein testing for the child with a retropharyngeal abscess lead to a safer and earlier hospital discharge? Which antiinflammatory medication(s) works best to treat multisystem inflammatory syndrome in children?

Most fundamental clinical questions surrounding the care of hospitalized children have not been examined by using RCTs, the highest level of evidence. Too often, clinicians caring for hospitalized children must extrapolate from RCT data in adult populations who generally have different clinical characteristics from the pediatric population eligible for the same intervention. Whereas certain rare pediatric diseases (eg, pediatric cancer) are frequently studied by using RCTs, RCT data to guide the care of children hospitalized for the most prevalent diseases (eg, common infections, asthma) are scarce.

The objective of this article is to propose an RCT strategy to efficiently answer fundamental questions for children hospitalized with common diseases. This article is organized into 3 parts: (1) the pressing need for RCTs involving children hospitalized with common diseases, (2) High-Efficiency RandOmIzed Controlled (HEROIC) Trials: a strategy to answer pragmatic questions for hospitalized children with common diseases, and (3) next steps.

Although the annual rate of adult RCTs published is increasing year over year, the rate of pediatric RCTs published per year has not changed since 2005.¹  When pediatric RCTs are conducted, they also involve substantially fewer sites than adult RCTs.²  These realities likely reflect the unique challenges of pediatric trials. All trials are time- and resource-intensive because of processes like managing multisite institutional review board approval, data use agreements, financial management, registering the trial, prespecifying analyses, consenting participants to have their care altered by randomization, performing randomization in an unbiased manner, and maintaining fidelity for the assigned intervention. To execute these processes, sites that participate in RCTs generally require a robust research infrastructure and sizable project funding.

In addition to these universal RCT complexities, pediatric RCTs must contend with challenges that are unique to enrolling children in trials. First, the prevalence of most childhood disease is relatively low, which translates into a smaller pool of potential trial participants. For example, in the United States, there is a more than 20-fold difference in the annual number of hospitalizations for the leading cause of adult hospitalization (septicemia, n = 2 200 000) compared with the leading cause of pediatric hospitalization (bronchiolitis, n = 100 000).³  Low-prevalence diseases require more intensive screening processes, a larger number of recruiting sites, and longer trial durations to allow full participant accrual. Second, there are policy drivers that lead to disparate funding for pediatric research. For example, the low prevalence of pediatric diseases means that federal funding agencies allocate fewer dollars⁴  and industry sponsors have less appetite to fund studies of these diseases.⁵  Third, the complexity of consenting children is increased because children are a vulnerable population and generally require substitute decision-makers for consent.^6,7  Finally, objective outcomes of severe disease, such as mortality, are much less common for children. As a result, pediatric trials must recruit larger samples to have adequate power for these outcomes or rely on surrogate outcomes. These complexities can make RCT participation daunting for investigators and their sites who are otherwise enthusiastic to improve pediatric care through research.

Six million children are hospitalized annually in the United States, at a total cost of nearly $50 billion per year.⁸  For children, hospitalization is a time of increased risk for morbidity and mortality.^9,10  For their parents and caregivers, hospitalization is an emotionally taxing experience that can lead to lasting psychological stress.^11,12  For society, pediatric hospitalizations are not simply costly, they are also a significant source of missed work and lost economic productivity.¹³  As the gold standard study design, RCTs warrant a prominent role in elucidating interventions that can reduce morbidity and mortality for hospitalized children and get them home sooner.

Although the pediatric research community has done a remarkable job studying certain diseases with RCTs (eg, malignancies), the most common diseases for which children are hospitalized are relatively underrepresented in RCTs.¹⁴  For example, respiratory infections like bronchiolitis and pneumonia are the leading cause of hospitalization for children in the United States, but respiratory infections are among the most underrepresented diseases in RCTs relative to their burden.¹⁴  The underrepresentation of common diseases in pediatric RCTs is compounded by the fact that RCTs involving hospitalized children are almost universally conducted at free-standing children’s hospitals. However, the majority (70%) of hospitalized children receive their care outside of children’s hospitals, in general hospitals.¹⁵  The discordance between where children with common diseases receive care and where the research happens results in 2 unintended, but potentially deleterious, consequences for pediatric RCTs involving hospitalized children. First, conclusions reached from RCTs that rely exclusively on children’s hospitals for enrollment may not generalize to the larger population of children who are hospitalized in general hospital settings. Compared with children’s hospitals, general hospitals usually have fewer specialized resources for children (eg, specialty-trained pediatric providers, pediatric intensive care facilities) and hospitalize children who are less medically complex.^15,16  As a result, the effect of interventions studied at children’s hospitals may be different in general hospitals. Secondly, a lack of participation in pediatric RCTs reduces the voice of general hospitals in developing pediatric trial questions. This may be one reason that common diseases for which children are hospitalized are relatively underrepresented in RCTs.

To increase the representativeness of pediatric RCTs and to improve care for children who are hospitalized with the most common diseases, we propose to develop a pragmatic RCT strategy called HEROIC trials. HEROIC trials will engage both children’s and general hospitals in pursuing rapid, low-cost accumulation of study participants with minimal burden for individual sites. Five key strategies that distinguish HEROIC trials are highlighted in Table 1 and discussed below.

The foremost feature that distinguishes HEROIC trials is dispersed low-volume recruitment goals across a large number of hospitals (n = 50–150), as illustrated in Fig 1. This dispersion of recruitment allows for smaller recruitment goals at individual sites, increasing the feasibility of participation for institutions without major research infrastructure. Dispersed site involvement has been used by nonrandomized implementation and observational studies to achieve rapid, low-cost data collection for a small number of hospitalized patients per site. Examples include the Value in Inpatient Pediatrics (VIP) implementation studies of hospitalized children with asthma (12 013 charts reviewed across 68 sites),¹⁷  fever (20 570 charts reviewed across 124 sites),¹⁸  and bronchiolitis (1869 charts reviewed across 21 sites)¹⁹  and a Pediatric Research in Inpatient Settings (PRIS) Network observational study of children hospitalized with bronchiolitis (3612 patient observations across 56 sites).²⁰  The only example of dispersed RCT recruitment involving >50 sites for a common pediatric condition in the hospital setting known to the authors is a Pediatric Emergency Care Applied Research Network trial involving 58 sites.²¹  However, a portion of these sites enrolled only adult patients.

A key component for the success of dispersed site involvement in the above nonrandomized studies was the recruitment of site leads. Site leads were individuals at each site who performed site institutional review board (IRB) submission, led site participant enrollment, and performed chart review for each enrolled participant. Site leads were recruited through the PRIS and VIP Networks and incentivized with authorship, research training, nominal financial support, and/or continuing medical education or maintenance of certification credits. The same recruitment process and incentive structure can be applied to site lead engagement for HEROIC trials.

Sites that participate in HEROIC trials will likely be heterogeneous in terms of both site-level (referral patterns, indications for treatment) and patient-level (race, ethnicity, severity of illness) characteristics. Heterogeneity has different implications for explanatory and pragmatic trials. Explanatory trials focus on the efficacy of an intervention on highly selected participants under idealized conditions. Pragmatic trials, on the other hand, focus on the effectiveness of an intervention among patients and within settings that match everyday practice.²²  Although a great deal of heterogeneity is problematic for explanatory trials, heterogeneity is a significant strength for pragmatic trials, allowing them to better approximate real-world effectiveness that is generalizable. For this reason, HEROIC trials will focus on pragmatic questions. In addition, interventions that are tested in HEROIC trials will be simple and already in widespread clinical use. Simple interventions make it easier for heterogenous sites to consistently and successfully apply the intervention to their patients. A focus on interventions already being used in practice eases the regulatory burden for HEROIC trials (discussed below). Sample pragmatic research questions examining simple interventions in widespread clinical use are provided in Table 2.

IRB approval and data use agreements are important regulatory requirements for multicenter trials, but represent barriers to research participation, particularly for pediatric providers at general hospitals, who often have less experience and support for research. Fortunately, several innovative options for streamlining these requirements have been developed recently. Single IRB, a setup in which 1 site performs IRB review and all other participating sites establish reliance agreements with the reviewing IRB, is now common for multicenter trials and is required for most National Institutes of Health-funded studies.²³  Additionally, interventions studied in HEROIC trials will be minimal risk interventions that are already being used in clinical practice. For such interventions, integrated consent can be appropriate.²⁴  Integrated consent allows a patient’s medical provider to obtain consent within the context of a routine medical encounter, obviating the need for additional research staff to perform consent. The use of electronic consent can further increase the efficiency of the consent process.²⁵ 

Ultimately, a HEROIC trial network capable of executing multiple trials every year should be established. One advantage of scaling HEROIC trials across a network is the opportunity to batch administrative approvals. For example, a data use agreement could cover core data elements for multiple HEROIC trials within the same agreement, reducing the cost and time investment for sites that are participating in multiple trials. Similarly, master reliance agreements can facilitate the rapid establishment of reliance and short timelines for approval. The second advantage of a HEROIC trials network is the ability to be poised to answer emerging questions with speed. A network of site leads with experience in dispersed, high-efficiency recruitment could be quickly engaged to enroll patients who intermittently present with a novel disease such as COVID-19.

Potential strengths of HEROIC trials include the following:

1. Ability to rapidly enroll: engagement of a large number of sites with low-volume recruitment targets for each site means that trial enrollment can be completed in weeks to months for common pediatric diseases.

2. Generalizability: small recruitment targets for each site and minimal research infrastructure requirements will make it easier to include diverse and representative settings, like general hospitals.

3. Low cost: shorter trial duration and small recruitment targets for each site mitigate the need for much of the research staff required within a traditional RCT (eg, research coordinator and site PI salaries).

4. Faster initiation of RCTs: lower operating costs make HEROIC RCTs less dependent on large grants, compared with traditional RCTs. Less reliance on large grant funding decreases the time from trial conception to trial initiation.

5. Readiness to address urgent needs: faster trial initiation and the ability to rapidly enroll participants make HEROIC trials well-poised to tackle time-sensitive research questions, such as questions arising during a pandemic.

The combination of a large number of participating sites and limited financial compensation decreases the extent to which HEROIC trials will be able to prescribe idealized conditions. For example, a HEROIC trial would be ill-suited for a complex protocol that requires timed medication administrations and laboratory draws or close monitoring by research staff. As such, explanatory questions will generally be best served in traditional RCT approaches. Similarly, HEROIC trials are not well-suited for questions that use a placebo or sham control. Placebo and sham control trials require substantial resources and infrastructure that are less compatible with the broad hospital participation and low-cost goals at the core of HEROIC trials. Additionally, most of the pragmatic questions addressed by HEROIC trials will lend themselves to a noninferiority design. Noninferiority designs occasionally result in larger sample sizes compared with superiority trials.²⁶ 

The analysis of multicenter RCTs is often stratified by site (eg, by using a fixed or random effect in regression modeling), which lessens the possibility that differences in treatment effect across sites can lead to spurious estimates of the true overall treatment effect.²⁷  A potential limitation for HEROIC trials is that the small number of participants per site can reduce the ability to understand whether there are differences in the treatment effect across the site.²⁸  HEROIC trials can mitigate this limitation in 2 ways: (1) require a minimum number of participants per site, such that stratified analyses are feasible and/or (2) select interventions for the study that are likely to result in minimal differences in treatment effect across sites. The latter strategy would require interventions that can be uniformly applied across sites and interventions for which there is no strong evidence for interactions with other factors that may vary by site.

A first step toward launching HEROIC trials will be to prioritize study questions that are both pressing and feasible. We are currently leading a series of conferences funded by the Agency for Healthcare Research and Quality (1R13HS028761-01), the goal of which is to identify the most important and feasible RCT questions for children hospitalized with common diseases. Forty multidisciplinary stakeholders for the care of hospitalized children (medical providers, clinical and health services researchers, patients and families, health equity experts, and policymakers) have been recruited to participate in the conference series. Conference attendees will be introduced to the HEROIC strategy, and we anticipate that the HEROIC strategy will be optimal for a portion of the RCT questions that are developed.

From our list of the most important and feasible RCT questions, we will select 1 question to answer with the first HEROIC trial. A HEROIC trial steering committee will work with disease-specific content experts to finalize the study question and the trial leadership team. Recruitment of site leads will occur through networks like PRIS and VIP, following their established blueprint for engaging large and diverse groups of site leads. A centralized electronic database for data entry and randomization (eg, REDCap) will be designed and tested, with access and training provided to all site leads. With input from site leads, the trial leadership team will finalize the trial protocol, adhering to guidelines for protocol development (standard protocol items: recommendations for interventional trials)²⁹  and reporting (consolidated standards of reporting trials).³⁰  Throughout enrollment, regular data quality checks will be performed, with feedback provided to site leads as necessary. The trial leadership team will draft a manuscript of the findings and site leads will provide input and revisions. Our expectation is that site leads will meet the requirements for authorship, either as an author named on the byline or as a group author. Explicit authorship criteria will be provided to site leads before participation. Byline authorship will likely be reserved for site leads who contribute the most to trial conception and data collection.

Our long-term vision is to establish a network of experienced HEROIC trial sites that can simultaneously execute multiple HEROIC trials every year. To support this goal, the first HEROIC trials will have 3 objectives: (1) answer a pressing clinical question for hospitalized children, (2) study and improve the HEROIC trial strategy, and (3) serve as proof of concept to secure grant funding for network infrastructure. The second objective will be achieved by using the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework,³³  which can help identify the barriers and facilitators to the implementation of HEROIC trials across both children’s and general hospitals. By continuously integrating such knowledge to evolve HEROIC methods, we hope to support the sustainability of HEROIC trials as an enduring vehicle to generate evidence for the care of hospitalized children. Ultimately, a reliable source of funding will be necessary to support a HEROIC network infrastructure. Elements of the network infrastructure could include central research coordinators to guide site leads through IRB and data use agreements, salary support for site leads who are involved in multiple trials, a data coordinating center, and/or funding to pursue the linkage of existing data sources (PHIS, HCUP-KID, PEDSnet, etc). Finally, HEROIC trials were designed with the intent of patient-level randomization. The foremost feature of HEROIC trials, dispersed recruitment, is aligned with patient-level recruitment. However, cluster randomization for testing system-level interventions could potentially be supported by the HEROIC trial strategy, too. A large number of HEROIC sites would certainly be a strength for cluster randomization.

Most of our best clinical evidence takes decades to reach patients broadly and improve population-level health. One barrier to faster uptake of clinical evidence is that most trials are conducted at a small number of academic centers, limiting understanding of the effectiveness of the intervention and how it might be implemented outside of those academic centers. The geographic and practice setting (ie, children’s and community hospitals) diversity and the sheer number of hospitals that will be involved in each trial will be assets for the successful dissemination and implementation of HEROIC trial findings. To optimally leverage these assets for dissemination and implementation, HEROIC trials can be conducted as hybrid effectiveness-implementation trials, which seek to answer key implementation questions in concert with the central question about intervention effectiveness.³¹  Implementation questions should be pursued within the context of a framework, such as the consolidated framework for implementation research.³²  Utilizing an implementation framework alongside an ongoing trial involving diverse practice environments should provide a rich understanding of the barriers and facilitators for the dissemination of interventions examined in HEROIC trials.

There are limited RCT data for fundamental clinical questions involving children hospitalized with common diseases. HEROIC trials represent a pragmatic strategy to deliver timely, rigorous, and generalizable evidence to improve care for hospitalized children and their families.