Compliance With Central Line Maintenance Bundle and Infection Rates

All data for this analysis were obtained from the Children’s Hospitals’ Solutions for Patient Safety (SPS) network. SPS is an international collaborative of children’s hospitals and was established initially as a Hospital Engagement Network and, later, as a Hospital Improvement and Innovation Network.^5,10  In 2018, SPS transitioned away from the Hospital Improvement and Innovation Network model to be exclusively member-funded.⁵  The basic philosophy of SPS is to recommend evidence-based bundles created by subject matter experts to decrease the incidence of various hospital-associated conditions (HACs).¹¹  Individual participating hospitals then implement these bundles and report to SPS their compliance with the bundles as the proportion of bundle observations completed correctly for all the elements (“all or none”). This standardized requirement for bundle audits has been emphasized for all hospitals since 2014. Before that, hospitals had the latitude to define a compliant bundle. Some hospitals perform electronic audits on all patients, whereas some form manual convenience/random audits on a smaller sample using directly observed practice. In addition to bundle compliance, the hospital reports to SPS the incidence of various HAC events (for example, CLABSI, CAUTI, etc) based on a standardized operational definition. The number of bundle observations, compliance, and HAC events are all self-reported voluntarily.¹² 

As part of QI, data submitted to SPS by participating hospitals included only hospital-level data. No patient-level information was submitted. For this analysis, aggregate data were obtained from the network, and hospital names were deidentified. The Institutional Review Board of the University of Illinois College of Medicine at Peoria reviewed and approved the study as an exempt study (1809273-1).

The study included data from all hospitals participating in the CLABSI HAC between January 1, 2011 to December 31, 2021 in the United States and Canada. Sites not reporting CLABSI maintenance data during the study period were excluded.

Data obtained from SPS included File A: aggregate data for all sites (n = 132, monthly data for 11 years), File B: deidentified hospital-level data (n = 14 170, monthly data over 11 years for each participating hospital), and File C: characteristics of 159 participating sites. For the primary objective, we obtained monthly aggregate reported bundle compliance at the network level and the monthly average CLABSI rate of the network (number of CLABSIs/1000 line days). Network reliability, as reported by SPS, differs slightly from mathematical bundle compliance (compliant bundles/observed bundles). Because there is a lot of variation in the number of audits submitted to SPS, by dividing the total number of compliant bundles by the total number of observed bundles across all hospitals, hospitals that do more audits will get more weight in the calculation. Instead, SPS weights each hospital’s bundle compliance by their reported outcomes data denominator (central line days) so that hospitals with a higher number of central line days get more weight in the aggregate calculation, getting a more accurate estimate of the bundle reliability for all patients with central lines at SPS hospitals. This correction only applies to the network-level data (File A) and not the monthly hospital-level data (File B).

In addition to the network reliability data, other extracted variables included monthly central line days, total bundles observed, the number of fully compliant bundles, and the number of CLABSI events per month. A derived variable of the central line audit rate was created as a percentage of total bundles observed (audits) out of the total monthly central line days (audit opportunities) of the individual site. Hospital information included hospital type (freestanding children’s hospital [FS] or hospital within a hospital [HWH]). In addition to the data on CLABSI bundle compliance and rate, we also obtained data on CAUTI bundle compliance. This was obtained as a proxy to assess the potential of overreporting bundle compliance by individual hospitals by correlating it with the central line infection rate, as described previously.⁷  In addition to the HAC bundles, SPS partners with hospitals to implement a comprehensive safety culture training program (safety culture training) through a formal partnership with Press Ganey Associates.¹³  This includes board education, event cause standardization, leadership strategies like safety huddles, safety rounding, peer safety coach, etc.⁵  For this analysis, we obtained the year participating hospitals completed their culture wave training as a surrogate to the implementation of a culture intervention. Summary details of the variables applied in this study are provided in Supplemental Table 7.

Statistical analysis consisted of basic descriptive statistics and calculation of the correlation coefficient (Pearson) of monthly network reliability with that month’s central line infection rate. To account for the secular trend of improvement in network reliability over the years, as well as improvement in CLABSI rate, a partial correlation coefficient (after including the serial no of the month in the model) was also calculated. Similarly, a correlation coefficient was calculated between CAUTI bundle compliance and CLABSI rate. The hospital-level data file included 14 170 U of data (site-month). One row was excluded for a biologically impossible CLABSI rate of 1000, 18 for missing CLABSI rates, and 3924 for missing CLABSI bundle compliance. Out of the 10 231, only data from sites with 20 or more months of data were selected for analysis (18/143 sites excluded). The final site-level analysis included 10 071 site months from 125 hospitals. To identify the quartile (Q) of compliance with bundle audits associated with the lowest CLABSI rates, median CLABSI rates across the 4 Qs of bundle compliance (based on site-month from hospital level data) were calculated and compared with the Kruskal Wallis test for group comparison, and Dunn test for pairwise comparisons using Q1 as control.

Mixed negative binomial regressions were performed to determine the impact of the hospital type, central line audit rate, culture wave training, and CLABSI bundle compliance on the CLABSI rate. CLABSI count was the primary outcome, and the number of central line days was included as an offset to account for the total amount of time that a CLABSI could occur each site month. Three separate models were created by using CLABSI bundle compliance ≥95% and either hospital type, central line audit rate, or safety culture training as independent variables. In each of the 3 models, the interaction between CLABSI bundle compliance ≥95% and the additional independent variable was included to determine if the relationship between bundle compliance and CLABSI rate is influenced by the additional independent variable. A random intercept was included to adjust for site-level clustering and repeated measures from the same site. The serial number of months the site reported data to the SPS was included in each model to account for temporal changes in CLABSI rate and bundle compliance. CLABSI bundle compliance was transformed into a binary variable indicating if the bundle compliance rate was ≥95% or not.⁷  The central line audit rate was also transformed into a categorical variable based on the quartiles (1st, <3.53%, second ≥3.53 to <7.63, third ≥7.63 to <17.24%, and fourth ≥17.24%). Safety culture training was labeled as “in progress” if it was completed in the same year as the reported data, “not done” if the reported data were before the completion date (or if it was not done by 2021), and “completed,” if reported data were after the completion of the safety culture training. Analysis was performed by using Statistical program R¹⁴  and JMP Pro v17 (SAS Institute, Cary, NC).

During this study period, participating hospitals reported 27 196 CLABSIs among 20 274 565 central line days for an overall median CLABSI rate of 1.34/1000 central line days. A total of 2 460 133 maintenance bundles were observed, of which 2 085 700 were fully compliant, for a median overall network bundle compliance of 84.2%. The number of participating hospitals increased from 33 at the network’s inception in 2011 to 145 in 2021. Data were available from 159 hospitals, of which 40 (25.8%) were FS hospitals and 115 (74.2%) HWH (missing n = 4). Over the 11 years of the network, there was a steady improvement in both CLABSI rate, decreasing from a median of 1.49/1000 central line days at inception to 1.20/1000 central line days in 2021, and network bundle reliability, increasing from a median of 75.5% in 2011 to 85.0% in 2021 (Table 1). Data from only 125 hospitals (each with ≥20 months of submitted data, total 10 071 site-months) was included in the regression analysis.

Overall, there was a negative Pearson correlation between the monthly network bundle reliability and monthly CLABSI rate with a correlation coefficient of –0.35 (95% confidence interval [CI] –0.49 to –0.19, P <.001). There was a significant negative correlation between CLABSI rate and months since inception (–0.25 [95% CI –0.41 to –0.09], P = .004) and a significant positive correlation between network reliability and months since inception (0.66 [95% CI 0.55 to 0.74], P <.001]). The partial correlation of network reliability with the CLABSI rate after accounting for the secular trend of improvement by months was –0.25 (P = .004). There was no significant correlation between the monthly network CLABSI rate and network reliability for the CAUTI bundle, with a correlation coefficient of 0.04 (95% CI –0.16 to 0.24, P = .69). (Figure)

Among all site months included in the analysis, the median CLABSI rate was significantly different (P <.001) across the 4 quartiles of bundle compliance, with the highest rate in the first quartile (1.118/1000 central line days, interquartile range [IQR] 0 to 1.99) and lowest in the fourth quartile (0.852/1000 central line days, IQR 0 to 1.727). Pairwise comparison, however, revealed a significant difference only between Q1 and Q4 (Table 2). CLABSI rates were also lower among all site-months, with compliance ≥95% on all sub-analyses, except for site-months at standalone locations (Table 3).

On negative binomial regression, CLABSI bundle compliance ≥95% in the reference level of the HWH was significantly associated with a lower CLABSI rate. No significant difference was found between HWH and FS hospital sites within the compliance reference level of <95%. However, a significant positive interaction was noted between FS hospital sites and compliance ≥95%, suggesting that compliance ≥95% was only associated with decreased CLABSI rates in the HWH group but not the FS hospital sites (Table 4). A CLABSI bundle compliance rate ≥95% was associated with significantly lower CLABSI rates after controlling for central line audit rate quartiles and months reporting to SPS, whereas central line audit rate quartiles were not associated with CLABSI rate. No significant interactions were found between bundle compliance rate ≥95% and Q 2, 3, or 4 of the central line audit rate, indicating that the amount of change in CLABSI rate between bundle compliance <95% and ≥95% was not significantly different across quartiles (Table 5). Similarly, compliance ≥95% was associated with significantly lower CLABSI rates after controlling for safety culture training and months of reporting to SPS. Safety culture training was not significantly associated with CLABSI rates, and no significant interactions were observed between stages of safety culture training (completed, in progress, or not done) and CLABSI bundle compliance, indicating the impact of bundle compliance was not significantly different across training stages (Table 6). Months of reporting to SPS were significantly associated with lower CLABSI rates in all models.

In large aggregate longitudinal data from 159 participating hospitals in the SPS network, we have shown a negative correlation between bundle compliance and CLABSI rate. This effect persists even after adjusting for the secular trend of improvement in reported reliability and CLABSI rates and is specific to CLABSI bundle-related compliance. High bundle compliance (≥95%) was significantly associated with lower CLABSI rates in HWH but not in FS hospitals. The association between compliance and CLABSI rates was unaffected by the central line audit rate or implementation of a comprehensive safety culture program. Within the limitation of ascribing causality in retrospective studies (we cannot conclude that “high bundle compliance caused the CLABSI rate to decrease”), this is the most robust evidence suggesting that efforts to improve compliance with evidence-based best care practices performed together as a bundle by themselves lead to decrease in CLABSI rates.

Although previous studies have revealed a reduction in CLABSI rates after the initiation of improvement efforts, including the implementation of a CLABSI bundle,^3,15  this reduction can also be due to the overall improvement in the quality and safety culture of the hospital or the Hawthorne effect (improvement in quality because of the awareness of measurement).¹⁶  Our study is the first large multicenter study to suggest a direct impact of compliance on the CLABSI rate. Our results agree with the previous single-center studies,^8,9  and a similar analysis from large survey-based data.⁷  Our finding of the correlation of CLABSI rate being specific to CLABSI bundle compliance (and not for CAUTI bundle compliance) potentially corrects 2 types of bias, (1) systematic overreporting and (2) culture of safety⁷  (both should lead to a similar improvement in CAUTI bundle compliance also), and further supports the argument that at least some of the improvement is directly related to the efforts to increase central line maintenance bundle compliance.

Our data revealed a significant interaction between hospital type and bundle compliance in relation to CLABSI rate, with only HWH showing a significant association. Because the determination of the negative correlation depends on a change, if the hospital had a stable CLABSI rate and high compliance, we might not be able to detect the difference (Hospital has high baseline compliance →stable performance over time →Absent correlation despite high performance). Because our data are not adjusted for patient acuity, it is possible that it is affecting the CLABSI rate of FS hospitals, which are expected to have more complex and sick patients.⁷ 

We believed that a higher central line audit rate should strongly affect the association between bundle compliance and the CLABSI rate. Although we observed a higher CLABSI rate in the site-months with the bundle compliance <95% versus ≥95%. This difference was equivalent across the 4 quartiles of the central line audit rate. Thus, our data do not reveal that a higher central line audit rate may lead to a higher impact on the association of compliance with the bundle elements on CLABSI rates. Although our results do not support our hypothesis, they support the argument that a larger number (or rate) of audits is not necessarily better. Although seemingly counterintuitive, this aligns with the anecdotal evidence among the leadership at SPS that a large number of electronic medical record-based audits are not as valuable as a small number of more labor-intensive in-person observations. Although they may result in fewer observations and lower reported compliance, they are more effective in driving the desired results by greater frontline staff engagement.¹⁷  However, this remains speculative as this study design cannot prove that the hospitals with fewer hospital audits are doing better audits.

Similarly, we did not observe any interaction of implementation of the comprehensive safety culture program on the relationship between bundle compliance and CLABSI rates. Although we expect that the hospitals that have implemented this may perform better in HAC reduction efforts if this leads to similar improvements in bundle compliance and CLABSI rates, our model would not detect an interaction. These results may also partially support our hypothesis that the decrease in CLABSI rates is specific to improvement in the CLABSI bundle compliance.

The main limitation of our study methodology is that the analysis depends on the correlation of reported compliance with the CLABSI rate. The true bundle reliability and reported bundle reliability can diverge significantly. It is possible that centers that do a more rigorous compliance assessment and thus report lower compliance have lower CLABSI rates than hospitals for which compliance assessment is not as thorough. This will result in an erroneous evaluation of correlation. Secondly, correlation means whether hospital compliance with the maintenance bundle is associated with lower CLABSI rates. For hospitals with stable high compliance or low CLABSI rate with minimal variance, we may be unable to detect a correlation signal although the hospital is by itself high performing. Thus, a negative correlation is just an operational measure and does not signify the desired outcome metric. A caution against overstating the relationship between bundle and outcomes must be taken. Patient acuity may affect both CLABSI rates and bundle compliance variably. However, because of the limitations of the dataset, it was not adjusted for in our analysis.

Within the limitations of retrospective correlation studies, this analysis reveals that adherence to best practice evidence-based bundles is negatively associated with the rate of hospital-acquired condition. Although our analysis was limited to CLABSI bundle compliance and CLABSI rate, the same principle may apply to other hospital-acquired conditions.

CAUTI

catheter-associated urinary tract infection

CI

confidence interval

CLABSI

central line-associated bloodstream infection

FS

freestanding

HAC

hospital-associated condition

HWH

hospitals within hospitals

IQR

interquartile range

Q

quartile

QI

quality improvement

SPS

Children’s Hospitals’ Solutions for Patient Safety