This article raises 2 head-scratching thoughts. First is to understand what an equivalence trial is, and how to use the results. Second, outcomes are important. What bronchiolitis outcome should we really care about?
Source: Cunningham S, Rodriguez A, Adams T, et al. Oxygen saturation targets in infants with bronchiolitis (BIDS): a double-blind, randomised, equivalence trial. Lancet.2015;386(9998):1041-1048; doi:10.1016/S0140-6736(15)00163-4. See AAP Grand Rounds commentary by Dr. Matthew Garber (subscription required).
PICO Question: Among infants hospitalized with bronchiolitis, is a minimum oxygen saturation target of 90% versus 94% associated with equivalent outcomes?
Question type: Intervention
Study design: Randomized controlled
About a year ago, I commented on an article that used fake pulse oximeter readings; the study implied an over-reliance on these readings by clinicians providing care to children with bronchiolitis. Now, we have a return to this trickery in the current study carried out at several hospitals in the United Kingdom, where infants hospitalized for bronchiolitis were randomized to either pulse oximeters displaying true readings, or to modified oximeters where, for pulse oximetry measurements in the 85 - 100% saturation range, the machine actually would display a true measurement of 90% to read 94% (i.e. measurments in this range were skewed higher). Basically, the investigators wanted to determine if the AAP oxygen saturation target of 90% saturation was equivalent to the UK's criteria that used 94% saturation. Without going into all the excellent study design elements that helped limit bias in this protocol, suffice to say that the investigators found that the 2 cutoffs were "equivalent," i.e. no difference in the primary outcome which was duration of cough. (More on that later.)
This study design may sound vaguely familiar to regular readers of this blog who recall posts about non-inferiority studies, first about 3 years ago. The main advantage of non-inferiority and equivalence trials, as opposed to superiority trials, is that the former trials require much fewer enrollees. Of course, the main danger is that we might conclude 2 treatments are equivalent, or non-inferior, when in fact a larger trial might have demonstrated a significant difference. But what's the difference between an equivalence trial and a non-inferiority trial? Most simply, they differ in the category of statistical analysis that is used: one-tailed versus two-tailed. A noninferiority trial is looking to ensure that a new treatment is not substantially inferior to an existing treatment. This allows use of one-tailed testing, which is easier to reach statistically. Two-tailed testing is required for equivalence, because the investigators are looking to ensure that the new intervention is neither substantially inferior nor superior to existing treatment. Both are risky study designs, especially if a placebo group is not included. One could reach non-inferiority or equivalence of the new treatment, but if the existing treatment isn't much better than placebo, we run into a slippery slope of possibly concluding that a treatment inferior to placebo is OK. (See the "3 years ago" link above for more explanation.)
A second source of head-scratching is the authors' choice of outcomes: duration of cough. While that is somewhat important to patients and parents, I can think of lots more outcomes of bronchiolitis that would be more important, like readmission to the hospital or time returning to full feedings. Totally ignored are long term outcomes potentially affected by hypoxemia, such as neurodevelopment.
The authors did perform a post-hoc analysis showing that the experimental group (i.e. the kids discharged with the lower oxygen saturations) actually had fewer readmissions and time to return to adequate feeding. Note, however, that post-hoc analyses, which are not included in determining sample size at the start of a study, have risks in leading to false conclusions. Their use violates certain statistical principles and must be interpreted with caution.
I don't think the current study really answers the debate of which oxygen saturation cutoff should be used as a criterion for hospital discharge in bronchiolitis, primarily because we don't have good long-term follow up information. I suspect many of the care providers in the US are using the AAP cutoff of 90%, and some may not be using pulse oximetry at all for this purpose, as detailed in the most recent AAP guideline. I'll be interested to see if the UK makes any changes.
This article raises 2 head-scratching thoughts. First is to understand what an equivalence trial is, and how to use the results. Second, outcomes are important. What bronchiolitis outcome should we really care about?
Source: Cunningham S, Rodriguez A, Adams T, et al. Oxygen saturation targets in infants with bronchiolitis (BIDS): a double-blind, randomised, equivalence trial. Lancet.2015;386(9998):1041-1048 doi:10.1016/S0140-6736(15)00163-4. See AAP Grand Rounds commentary by Dr. Matthew Garber (subscription required)
Question: Among infants hospitalized with bronchiolitis, is a minimum oxygen saturation target of 90% versus 94% associated with equivalent outcomes?
Question type: Intervention
Study design: Randomized controlled
About a year ago, I commented on an article that used fake pulse oximeter readings; the study implied an over-reliance on these readings by clinicians providing care to children with bronchiolitis. Now, we have a return to this trickery in the current study carried out at several hospitals in the United Kingdom, where infants hospitalized for bronchiolitis were randomized to either pulse oximeters displaying true readings, or to modified oximeters where, for pulse oximetry measurements in the 85 - 100% saturation range, the machine actually would display a true measurement of 90% to read 94% (i.e. measurments in this range were skewed higher). Basically, the investigators wanted to determine if the AAP oxygen saturation target of 90% saturation was equivalent to the UK's criteria that used 94% saturation. Without going into all the excellent study design elements that helped limit bias in this protocol, suffice to say that the investigators found that the 2 cutoffs were "equivalent," i.e. no difference in the primary outcome which was duration of cough. (More on that later.)
This study design may sound vaguely familiar to regular readers of this blog who recall posts about non-inferiority studies, first about 3 years ago. The main advantage of non-inferiority and equivalence trials, as opposed to superiority trials, is that the former trials require much fewer enrollees. Of course, the main danger is that we might conclude 2 treatments are equivalent, or non-inferior, when in fact a larger trial might have demonstrated a significant difference. But what's the difference between an equivalence trial and a non-inferiority trial? Most simply, they differ in the category of statistical analysis that is used: one-tailed versus two-tailed. A noninferiority trial is looking to ensure that a new treatment is not substantially inferior to an existing treatment. This allows use of one-tailed testing, which is easier to reach statistically. Two-tailed testing is required for equivalence, because the investigators are looking to ensure that the new intervention is neither substantially inferior nor superior to existing treatment. Both are risky study designs, especially if a placebo group is not included. One could reach non-inferiority or equivalence of the new treatment, but if the existing treatment isn't much better than placebo, we run into a slippery slope of possibly concluding that a treatment inferior to placebo is OK. (See the "3 years ago" link above for more explanation.)
A second source of head-scratching is the authors' choice of outcomes: duration of cough. While that is somewhat important to patients and parents, I can think of lots more outcomes of bronchiolitis that would be more important, like readmission to the hospital or time returning to full feedings. Totally ignored are long term outcomes potentially affected by hypoxemia, such as neurodevelopment.
The authors did perform a post-hoc analysis showing that the experimental group (i.e. the kids discharged with the lower oxygen saturations) actually had fewer readmissions and time to return to adequate feeding. Note, however, that post-hoc analyses, which are not included in determining sample size at the start of a study, have risks in leading to false conclusions. Their use violates certain statistical principles and must be interpreted with caution.
I don't think the current study really answers the debate of which oxygen saturation cutoff should be used as a criterion for hospital discharge in bronchiolitis, primarily because we don't have good long-term follow up information. I suspect many of the care providers in the US are using the AAP cutoff of 90%, and some may not be using pulse oximetry at all for this purpose, as detailed in the most recent AAP guideline. I'll be interested to see if the UK makes any changes.
