Comparative Efficacy of Bronchiolitis Interventions in Acute Care: A Network Meta-analysis

We conducted a systematic review and NMA adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for NMA (Appendix 1 in Supplemental Information).¹²  We followed a protocol registered with the International Prospective Register of Systematic Reviews on November 31, 2016 (CRD42016048625), and specified all methods, outcomes, and potential analyses a priori. Deviations from the protocol are reported and justified in Appendix 2 in Supplemental Information. This systematic review was exempted from institutional ethics approval and had no patient or public involvement.

A research librarian searched 4 databases: Ovid Medline (1946 to October 18, 2016); Ovid Embase (1974 to week 42 2016); Cochrane Central Register of Controlled Trials via Wiley Cochrane Library (inception to October 19, 2016); and Cumulative Index to Nursing and Allied Health Literature Plus with Full Text via EBSCOhost (1937 to October 19, 2016). The full searches are included in Appendix 3 in Supplemental Information. ClinicalTrials.gov (January 1, 2014, to October 20, 2016) and the World Health Organization International Clinical Trials Registry Platform (January 1, 2014, to October 11, 2016) were also searched. The search was updated on September 1, 2019. To identify unpublished studies and unregistered trials, we also searched the Conference Proceedings Citation Index (January 1, 2014, to September 3, 2019). Throughout the process, we checked reference lists of pertinent studies and reviews and consulted with experts in the field.

Two independent reviewers (S.A.E. and L.A.G.) evaluated the titles and abstracts of identified studies for eligibility. The full texts of all citations flagged as “relevant” or “unclear” were independently reviewed by 2 researchers (S.A.E. and L.A.G.) to determine final inclusion. Disagreements were resolved by consensus or adjudication by a third reviewer (R.M.F.).

Studies were included if they were randomized controlled trials in which researchers recruited patients up to 24 months old diagnosed with bronchiolitis (defined by trial authors, including first and/or recurrent episodes) and compared any bronchodilator, glucocorticoid steroid (inhaled or systemic), hypertonic saline solution, antibiotic, heliox therapy, or high-flow oxygen therapy to any placebo or active comparator (including combined therapies). Studies in which researchers evaluated heliox or oxygen therapy delivered by face masks or nasal cannula were eligible; however, we excluded studies with participants intubated or mechanically ventilated at baseline. We also excluded studies in which researchers assessed longer courses of steroids started during the acute phase of bronchiolitis for the prevention of postbronchiolitic wheezing. Inclusion was not restricted by language or publication status. Because we sought to evaluate multiple interventions (established and novel), we did not restrict inclusion by publication year.

Clinical experts (R.M.F., T.P.K., D.A., A.C.P., and S.B.F.) selected outcomes of interest a priori for clinical relevance. The outcomes were admission rate on day 1 (ARD1) and admission rate on day 7 (ARD7) of initial presentation for outpatient studies and LOS for inpatient studies.

Data were extracted by a single reviewer (L.A.G.) onto a standardized form (available from authors) built in Microsoft Excel (Microsoft, Redmond, WA) and verified by a second reviewer (S.A.E.). Data included study characteristics; inclusion and exclusion criteria; participant demographics; intervention types, doses, modes of administration, timing, and cointerventions; outcome definitions (type, method, and timing of measurement); funding sources and conflicts of interest; and outcome results. When required, data were extracted from graphs and/or plots by the statistician (B.V.). Counts for categorical outcomes and means and SDs for continuous outcomes were extracted, as possible. When mean and/or SD were not reported, continuous outcomes were estimated by using the reported statistics (eg, median, interquartile range, etc).¹³ 

Before data analysis, we defined treatment nodes of interest for the NMA on the basis of current practice and the expertise of clinician authors (Table 1 in Supplemental Information). Additionally, as we included interventions with different routes of administration, we also split controls and placebos into 4 groups: (1) placebo nebulizations (placebo [neb]); (2) systemically delivered (ie, intramuscular, intravenous, or oral delivery) placebos (placebo [sys]); (3) inhaled gas (ie, air) placebos (placebo [air]); and (4) standard care, as defined by the trial authors. Arms of multiarm trials were combined when they were assigned to the same NMA node. After assigning each trial arm to a treatment node, studies contributing to 0 or 1 node of interest were excluded from the NMA.

We performed a random-effects NMA in a frequentist framework using MetaInsight, an online platform powered by the netmeta package in R (version 3.4.3).¹⁴  We calculated odds ratios (ORs) for binary outcomes (ie, ARD1 and ARD7) and mean differences (MDs) for continuous outcomes (ie, LOS), along with their 95% confidence intervals (CIs). We also used P-scores to rank all of the interventions within a network.¹⁵  For 0-events data in binary outcomes, stable continuity corrections of 0.5 were applied.¹³  For indirect comparisons to provide valid effect estimates, NMAs rely on an assumption (ie, clinical transitivity) that participants in all trials contributing to the network are similar in terms of effect modifiers (ie, demographics, disease severity, etc).¹⁶  We used narrow inclusion criteria to limit variability across study populations and qualitatively assessed the distributions of effect modifiers, such as age, disease severity, and wheeze status across treatment comparisons, to confirm that studies were sufficiently similar for valid, indirect inferences.

We assumed a common heterogeneity parameter across all treatment comparisons, and global heterogeneity was assessed by using the I² statistic with the GeMTC R package (version 3.4.3). Statistical inconsistency was evaluated by testing the agreement between direct and indirect evidence when viewed independently. We applied the design-by-treatment interaction model that evaluates inconsistency in the network jointly.¹⁷ 

Network structure was explored by using network diagrams to visualize head-to-head comparisons. In a network diagram, head-to-head comparisons of treatments are revealed through lines connecting individual nodes, with line thickness indicating the number of trials making that comparison. In addition, the area of each node represents the number of patients in which each node treatment was evaluated. In addition to network diagrams, contribution matrices were used to illustrate the contribution of each head-to-head comparison with the network for each outcome.¹⁸ 

Sensitivity analyses were determined a priori and were based on risk of bias (RoB) within individual studies (low versus unclear or high) and fixed- versus random-effects models.

Each included study was assessed by 2 independent reviewers (S.A.E. and L.A.G.) for RoB by using The Cochrane Collaboration risk-of-bias tool, version 1.0.¹⁹  Briefly, each study was evaluated for RoB on 7 domains: (1) random sequence generation, (2) allocation concealment, (3) blinding of participants and personnel, (4) blinding of outcome assessment, (5) incomplete outcome data, (6) selective reporting, and (7) other sources of bias (eg, funding source).¹⁹ For any non-English articles, the translator was consulted during the assessments.

We assessed confidence in the estimates for each outcome with Confidence in Network Meta-Analysis (CINeMA), an adaptation of the Grading of Recommendations Assessment, Development and Evaluation framework specifically developed for NMA.^20,21  Decision rules for confidence ratings were determined a priori on the basis of Salanti 2014 and the CINeMA guidelines (Appendix 4 in Supplemental Information).^16,20  A change in LOS of ±0.5 days or a relative change of ∼20% in ARD1 and ARD7 (ie, OR <0.8 or OR >1.25) were considered clinically important differences to assess the degree of imprecision, heterogeneity, and incoherence.

The funding agency played no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

The search returned 7544 unique records, and 646 full texts were reviewed. Overall, 150 studies enrolling 19 090 patients met the inclusion criteria (Fig 1).

Selected characteristics of the 150 relevant studies are presented in Table 2 in Supplemental Information. Studies were published from 47 counties between 1966 and 2019 (median 2008, interquartile range 2000–2014). There were 25 studies (17%) in which researchers did not report results on the nodes of interest selected for the NMA. Forty-five (30%) studies were conducted in outpatient settings, whereas 103 (69%) studies were in inpatient settings, and 2 studies (1%) were conducted across both settings. Additionally, 37% (n = 57) of studies were restricted to patients ≤12 months of age. In 91 (61%) studies, researchers restricted their sample to patients experiencing their first episode of wheezing. Overall, researchers in 76% of studies enrolled patients with bronchiolitis of moderate severity on the basis of either narrative statements of severity by study authors or clinical scores; 19% did not report a measure or definition of disease severity (Table 2 in Supplemental Information).

RoB by domain for all included studies is presented in Table 3 in Supplemental Information. The RoB across each network is summarized in Fig 2, revealed as the proportion of studies evaluated at each of the 3 levels of RoB (low RoB, some concerns, high RoB) for each of the 7 RoB domains. The overall RoB was low for 20 studies (13%), unclear for 70 studies (47%), and high for 60 studies (40%). Evaluation of clinical transitivity by visually inspecting distributions of potential effect modifiers across studies indicated that included trials were similar in terms of patient age, sex, and clinical acuity, and thus indirect comparisons of treatments across trials were likely to be valid (Appendix 6 in Supplemental Information).

The network plots revealing the connectedness of each outcome network are presented in Fig 3. Summaries of the studies in each network and their connectedness are described in Appendix 5 in Supplemental Information. The nodes for each network (ARD1, ARD7, and LOS) are presented in Tables 4–6 in Supplemental Information, respectively. The forest plots in Fig 4 present treatment-effect sizes compared with a common reference group (nebulized placebo) on the basis of the NMA. The cumulative rankings of treatments based on evidence from the entire network are revealed in Tables 1–3 for ARD1, ARD7 and LOS, respectively. The certainty of evidence for each network estimate is reported in Tables 7–9 in Supplemental Information.

Compared with the nebulized placebo, 2 treatments revealed evidence of effectiveness at reducing ARD1 (Fig 4A): nebulized epinephrine and nebulized hypertonic saline plus salbutamol. Nebulized hypertonic saline plus salbutamol was ranked as the most effective treatment to reduce admission rate and was significantly more effective than nebulized placebo (OR: 0.44, 95% CI: 0.23 to 0.84; low confidence), nebulized salbutamol (OR: 0.55, 95% CI 0.33 to 0.91; moderate confidence), and nebulized salbutamol plus ipratropium bromide (OR: 0.27, 95% CI: 0.08 to 0.87; moderate confidence) (Table 1). Nebulized epinephrine was significantly more effective than nebulized placebo (OR: 0.64, 95% CI: 0.44 to 0.93; low confidence). No other statistically significant differences between treatments were identified. Overall confidence in the evidence for the ARD1 network was low to very low (Table 7 in Supplemental Information).

No treatment significantly decreased ARD7 compared with nebulized placebo (Fig 4B). The first-ranked treatment was nebulized salbutamol plus systemic steroid (OR: 0.68, 95% CI: 0.21 to 2.21; very low confidence). However, nebulized epinephrine plus systemic steroid was the only treatment significantly more effective than any other treatment (Table 2). Overall confidence in the ARD7 network was low to very low (Table 8 in Supplemental Information).

Two treatments revealed evidence of effectiveness at reducing LOS compared to nebulized placebo (Fig 4C): nebulized hypertonic saline plus epinephrine (MD: −0.91 days, 95% CI: −1.42 to −0.39; low confidence) and nebulized hypertonic saline (MD: −0.63 days, 95% CI: −1.02 to −0.25; low confidence). Nebulized hypertonic saline plus epinephrine was ranked first for effectiveness and was statistically significantly more effective than nebulized epinephrine alone, nebulized salbutamol, and nebulized placebo (Table 3). No other statistically significant differences were found between treatments, and overall confidence in the evidence was very low (Table 9 in Supplemental Information).

Results of the sensitivity analyses are presented in the Supplement (Appendices 7–9 in Supplemental Information, respectively. No statistically significant differences between treatments and nebulized placebo were identified in the ARD1 network with low RoB trials. There were no other notable or significant differences for ARD1 and ARD7 compared with the primary analysis, including no significant differences in treatment rankings. For LOS, when looking at low RoB studies only, hypertonic saline alone became the top-ranked treatment (ie, the treatment ranking of nebulized hypertonic saline alone and nebulized hypertonic saline plus epinephrine were switched). In other sensitivity analyses for LOS, the overall treatment rankings remained similar to the primary analysis.

This review attempted to address uncertainty regarding the management of bronchiolitis. Notably, few treatments were significantly more effective than nebulized placebo (ie, 0.9% saline) at improving short-term outcomes. Both nebulized epinephrine and nebulized hypertonic saline plus salbutamol appear to reduce admission rates during the initial emergency department presentation. However, the strength of evidence for this finding was low, largely because of concerns around imprecision. Importantly, no treatments indicated significant benefit compared with nebulized placebo to prevent admission up to 7 days after initial discharge from the emergency department. Finally, although nebulized hypertonic saline combined with epinephrine, as well as hypertonic saline alone, revealed significant reductions in LOS, our confidence in this evidence is low to very low because of both imprecision in the estimate of effect and concerns around RoB within individual studies. We did not find evidence for benefits from other interventions, either alone or combined, in any outcome. Further evaluation of the treatments revealing evidence of effect but with low or very low confidence is required before these treatments can be recommended for use in regular practice.

Few major advances have been made over the last several decades in managing bronchiolitis. Although consensus now acknowledges the key role of supportive treatment in managing bronchiolitis, the rationales for most pharmacologic treatments remain contentious, and most clinical practice guidelines worldwide do not recommend routine use of pharmacologic treatments.²²  Researchers in mechanistic studies provide hypotheses for the benefit (eg, of hypertonic saline and epinephrine) or futility (eg, of corticosteroids and antibiotics) of these interventions, but these rarely align with the direction and magnitude of effect seen in clinical studies. Furthermore, a recent review highlighted varying and sometimes conflicting recommendations,⁹  and pharmacologic treatments remain pervasive in clinical practice.²³  Some recent treatment options (eg, oxygen therapy via high-flow nasal cannula) have revealed signs of possible benefit, but they remain controversial and uptake in practice varies.²⁴  Finally, many treatments have not been compared head-to-head, which limits the assessment of their comparative effectiveness.

To date, only 2 other reviews have attempted to synthesize and evaluate the effectiveness of multiple treatments for bronchiolitis across settings.^11,25  Both previous reviews used distinct eligibility criteria, included different interventions, or focused on specific settings and outcomes, leading to limited overlap between included studies and comparability between reviews. Furthermore, methods used to estimate network effects and rank treatments varied, and not all reviews were used to assess the certainty of the body of evidence. Thus, the interpretations of the clinical relevance of these results have also differed, which may explain why review authors (and guidelines based on these reviews) provided conflicting recommendations. In our review, a key finding is the low quality of evidence across comparisons, which limits our confidence in these effect estimates. Major concerns around imprecision, and in some cases RoB, add a caveat to interpreting positive results and should be the basis for making future treatment recommendations.

A common criticism of clinical research studies in this field has been the heterogeneous definitions applied to bronchiolitis.²⁶  Bronchiolitis may encompass a spectrum of acute wheezing disorders with common clinical traits but distinct underlying pathophysiologies. Subgroups based on demographic, clinical, viral, or other biological characteristics have been proposed to identify putative responder phenotypes or endotypes.²⁷  These possible effect modifiers have implications for NMA, given that assumptions about homogeneity and therefore the generalizability of findings are a limitation of mixed-treatment comparisons. In particular, age, history of wheezing episodes, and viral agent remain controversial indicators of indirectness when combining data from bronchiolitis studies. There is no clear guidance on how to approach the challenge of clinical heterogeneity at the review level, although it has been a source of discussion in the analysis and interpretation of previous reviews.^28,29  Using a pragmatic definition of bronchiolitis, we avoided excluding potentially relevant evidence, accommodating different perspectives in clinical practice. We qualitatively assessed the distribution of some putative effect modifiers (ie, age, disease severity, and wheeze status) across comparisons and concluded that there were no major concerns with transitivity. Although this has been a quintessential conundrum in wheezing disorder research and practice, the lack of a harmonized approach to phenotyping continues to hamper further exploration into these subgroups at both the trial and review levels.

This review followed the current methodologic standards for conducting and reporting on analyses that simultaneously compare multiple interventions.¹³  We searched extensively for relevant literature and included all studies, regardless of language of publication and incorporated, new methods to assess the confidence in our findings (ie, CINeMA). We are confident that this review represents the most comprehensive synthesis currently available for the most promising treatments for bronchiolitis. The decisions of which nodes to include in the analysis were determined a priori by a group of pediatric emergency physicians and clinical experts; however, not all nodes are reflective of global practice patterns.³⁰  Finally, as we did not report adverse events (eg, tachycardia, hypertension, pallor, tremor, nausea, vomiting, diarrhea, and acute urinary retention) in this analysis, we cannot make inferences about the safety of these treatments; however, other systematic reviews investigating the use of steroids,³¹  hypertonic saline,³²  oxygen therapy,³³  heliox,³⁴  and antibiotics³⁵  for the management of bronchiolitis in children reported no major adverse events associated with short-term use of these therapies.

Reducing waste in bronchiolitis research requires prioritizing questions and designing, conducting, and reporting studies that are more likely to increase certainty in the body of evidence. A set of standardized minimum criteria, including individual or clustered phenotype and/or endotype traits for eligibility and analysis, would further this aim. The selection of outcomes should be based on a core outcome set relevant to bronchiolitis patients and health care stakeholders, paying attention to outcome definitions (eg, escalation of care for interventions such as high-flow nasal cannula) and clinical relevance thresholds. The impact of new trial results on existing syntheses, such as this one, should also be considered (eg, by using value of information and cumulative or prospective meta-analysis methods).

This NMA suggests there may be a benefit of hypertonic saline with salbutamol to reduce admission rates on initial presentation to the emergency department; however, no effect on admission to the hospital up to 7 days after presentation was seen. Furthermore, hypertonic saline alone, or in combination with epinephrine, seems to reduce an infant’s stay in the hospital. However, our confidence in the effects of these treatments is low because of RoB in contributing studies and imprecision in their effect sizes. Low strength of evidence suggests that additional, well-designed, and rigorous research on bronchiolitis treatments is needed within both inpatient and outpatient settings. Current clinical practice guidelines should recommend only supportive measures on the basis of the absence of convincing evidence for any other approach.

We thank and acknowledge the research assistants and librarians who supported the completion of this project.

ARD1

admission rate on day 1

ARD7

admission rate on day 7

CI

confidence interval

CINeMA

Confidence in Network Meta-Analysis

heliox

helium-oxygen

LOS

length of stay

MD

mean difference

NMA

network meta-analysis

OR

odds ratio

RoB

risk of bias