A few years ago at the national Pediatric Hospital Medicine Meeting, presenters in one session lamented the lack of treatment options for the “disease that must not be named”: bronchiolitis. Bronchiolitis is a particularly vexing condition for clinicians who care for children in the inpatient setting. Although we provide supportive care, there is little we can do to alter the course of the disease. Interventions that once held promise, including albuterol, ultimately end up on the “do not try” list because they have no therapeutic benefit and add costs and risk of adverse events to patient care. When the initial trials for nebulized hypertonic saline delivery appeared in the literature, there was hope that it would be used to speed recovery time and shorten the length of stay (LOS). Astute clinicians noted that the initial studies were performed at institutions outside of the United States,1,2 where care algorithms differed and the average LOS tended to be longer. Questions regarding generalizability ensued. Furthermore, researchers of the meta-analyses of the effectiveness of hypertonic saline reached conflicting conclusions.3,4 In this issue of Pediatrics, Harrison et al5 take another look at the hypertonic saline trials with a fresh analytic technique: trial sequential analysis (TSA).

TSA is a relatively new method that is used to address some of the weaknesses of meta-analysis. Imagine a long randomized controlled trial (RCT) occurring over several years with multiple interim analyses to determine whether significance had been reached. Trialists would note that multiple interim analyses are problematic, and they would be correct in performing a statistical penalty for these analyses. In essence, this phenomenon of multiple analyses is what happens with the accumulation of evidence over time through multiple studies. The publication of RCTs is, in a way, akin to interim analyses in the grand scheme of truth. TSA is used to penalize these multiple looks (ie, publications) by using an “α spending function,” which is used to correct for multiple analyses by imposing a more stringent level of significance.6 In a conventional meta-analysis, this multiple analysis penalty does not exist despite the inclusion of studies multiple times in multiple meta-analyses. In addition, in the theoretical example of 1 long RCT, the participants would have the same inclusion and exclusion criteria, the same exposure to the intervention, and the same follow-up and outcome assessments. This framework is not the case in the real-world accumulation of evidence, and the hypertonic saline body of evidence has significant heterogeneity, as the authors note. Authors of meta-analyses must acknowledge and evaluate heterogeneity before pooling trials together. In TSA, there is a sample size adjustment for heterogeneity, such that more participants are needed to reach definite conclusions when heterogeneity is found to exist. The net effect of both the α adjustment and the heterogeneity adjustment is that more studies and participants are needed to reach conclusions in TSA. TSA can be used to inform the user whether there are enough data to draw definite conclusions.

In this TSA analysis of hypertonic saline, Harrison et al5 note that there are not enough data to draw conclusions on the effectiveness (or lack of effectiveness) of hypertonic saline. They conclude that an observed benefit from hypertonic saline is likely a type I error or a false-positive result. However, for the LOS outcome, the cumulative z score is close to the “monitoring boundary.” The monitoring boundary is the point in TSA at which a significant difference can be concluded. If the researchers in additional studies were able to find that hypertonic saline can be used to decrease LOS, the cumulative z score could cross the monitoring boundary, and researchers would conclude that hypertonic saline can be used to decrease the LOS. Thus, given the current literature, the potential still exists that hypertonic saline may decrease the LOS. Premature affirmation of the null hypothesis, or the determination that researchers in future studies are unlikely to identify a meaningful effect, may dissuade future investigators when the question might be clearly answered through further study. If hypertonic saline can be conclusively proven to be ineffective, it can open the door for leaders of quality improvement initiatives to eliminate waste, as has been done in other interventions, leading to potentially significant cost savings.7 One notable limitation in the study is that Harrison et al5 did not perform a separate search of the literature to ensure that all studies of LOS and hospitalization were included; given the need for additional studies, an additional literature search or the inclusion of studies in which both clinical questions were addressed may have been useful. However, with the number of meta-analyses available, it is unlikely that large studies were missed.

Can hypertonic saline be used to decrease hospital LOS? We do not know definitively; this analysis by Harrison et al5 should not dissuade future researchers from doing trials on the effect of hypertonic saline on LOS. Additional trials of hypertonic saline are needed. With this publication, Harrison et al,5 in addition to helping us understand the importance of evaluating the quality of information over time, also highlight important questions regarding the assumptions that we make when analyzing the results of multiple trials.

LOS

length of stay

RCT

randomized controlled trial

TSA

trial sequential analysis

The opinions expressed in these commentaries are those of the authors and not necessarily those of the American Academy of Pediatrics or its Committees.

FUNDING: No external funding.

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2018-1144.

1
Mandelberg
A
,
Tal
G
,
Witzling
M
, et al
.
Nebulized 3% hypertonic saline solution treatment in hospitalized infants with viral bronchiolitis.
Chest
.
2003
;
123
(
2
):
481
487
[PubMed]
2
Tal
G
,
Cesar
K
,
Oron
A
,
Houri
S
,
Ballin
A
,
Mandelberg
A
.
Hypertonic saline/epinephrine treatment in hospitalized infants with viral bronchiolitis reduces hospitalization stay: 2 years experience.
Isr Med Assoc J
.
2006
;
8
(
3
):
169
173
[PubMed]
3
Zhang
L
,
Mendoza-Sassi
RA
,
Klassen
TP
,
Wainwright
C
.
Nebulized hypertonic saline for acute bronchiolitis: a systematic review [published correction appears in pediatrics. 2016;137(4):e20160017].
Pediatrics
.
2015
;
136
(
4
):
687
701
[PubMed]
4
Brooks
CG
,
Harrison
WN
,
Ralston
SL
.
Association between hypertonic saline and hospital length of stay in acute viral bronchiolitis: a reanalysis of 2 meta-analyses.
JAMA Pediatr
.
2016
;
170
(
6
):
577
584
[PubMed]
5
Harrison
W
,
Angoulvant
F
,
House
S
,
Gajdos
V
,
Ralston
SL
.
Hypertonic saline in bronchiolitis and type 1 error: a trial sequential analysis.
Pediatrics
.
2018
;
142
(
3
):
e20181144
6
Thorlund
K
,
Engstrøm
J
,
Wetterslev
J
, et al
. User manual for trial sequential analysis (TSA). Available at: www.ctu.dk/tsa/files/TSA_manual.pdf. Accessed June 6, 2018
7
Ralston
S
,
Garber
M
,
Narang
S
, et al
.
Decreasing unnecessary utilization in acute bronchiolitis care: results from the value in inpatient pediatrics network.
J Hosp Med
.
2013
;
8
(
1
):
25
30
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.