BACKGROUND:

In 2013, New York introduced regulations mandating that hospitals develop pediatric-specific protocols for sepsis recognition and treatment.

METHODS:

We used hospital discharge data from 2011 to 2015 to compare changes in pediatric sepsis outcomes in New York and 4 control states: Florida, Massachusetts, Maryland, and New Jersey. We examined the effect of the New York regulations on 30-day in-hospital mortality using a comparative interrupted time-series approach, controlling for patient and hospital characteristics and preregulation temporal trends.

RESULTS:

We studied 9436 children admitted to 237 hospitals. Unadjusted pediatric sepsis mortality decreased in both New York (14.0% to 11.5%) and control states (14.4% to 11.2%). In the primary analysis, there was no significant effect of the regulations on mortality trends (differential quarterly change in mortality in New York compared with control states: −0.96%; 95% confidence interval [CI]: −1.95% to 0.02%; P = .06). However, in a prespecified sensitivity analysis excluding metropolitan New York hospitals that participated in earlier sepsis quality improvement, the regulations were associated with improved mortality trends (differential change: −2.08%; 95% CI: −3.79% to −0.37%; P = .02). The regulations were also associated with improved mortality trends in several prespecified subgroups, including previously healthy children (differential change: −1.36%; 95% CI: −2.62% to −0.09%; P = .04) and children not admitted through the emergency department (differential change: −2.42%; 95% CI: −4.24% to −0.61%; P = .01).

CONCLUSIONS:

Implementation of statewide sepsis regulations was generally associated with improved mortality trends in New York State, particularly in prespecified subpopulations of patients, suggesting that the regulations were successful in affecting sepsis outcomes.

What’s Known on This Subject:

Early recognition and treatment of pediatric sepsis are associated with improved outcomes, leading several state governments to adopt or consider sepsis regulations. However, these regulations are controversial, and whether they lead to improved pediatric patient outcomes is unknown.

What This Study Adds:

Implementation of statewide sepsis regulations was associated with improved mortality trends, particularly in selected subgroups of patients. In future work, researchers should examine how to refine these regulations to increase their impact.

Sepsis is the dysregulated immune response to infection leading to life-threatening organ dysfunction.1  In children, sepsis is a leading cause of morbidity and mortality, occurring at an incidence of 89 cases per 100 000 children annually, with a case fatality rate of up to 20%.2,3  Early recognition and treatment with antibiotics are associated with improved outcomes in sepsis,4  leading several state governments to consider regulations requiring protocols for early recognition and treatment.5,6  The first of these regulations, known as Rory’s Regulations, were enacted in New York State in 2013 and named after a pediatric patient who died of sepsis in a New York hospital.7  These regulations mandate that all hospitals in the state adopt pediatric-specific sepsis protocols, provide sepsis education to hospital staff, and report protocol adherence and patient outcomes to the state government.8 

Although a laudable goal, statewide regulatory mandates for pediatric sepsis care are controversial.9  There are concerns that these mandates may lead to unintended consequences such as overtreatment and antibiotic overuse.10  Another concern is that pediatric sepsis is a heterogenous disease that may not be amenable to broad-based policy initiatives. Most cases of pediatric sepsis occur in neonates and children with complex chronic conditions,11  and most present to specialized children’s hospitals that are already highly capable of caring for children who are critically ill.12  In contrast, previously healthy children with sepsis presenting to community hospitals, such as the case that motivated the New York regulations, are relatively rare.13  In this context, effects of broad sepsis policies may vary across patients and hospital types.

Some of these concerns are partially allayed by data collected under the New York mandate, which suggest that children receiving early treatment have a lower risk of death.14  However, the authors of that study only evaluated sepsis care after the regulations; they did not compare outcomes before and after the regulations or compare outcomes in New York with those in other states.14  Therefore, whether the regulations themselves led to improved outcomes remains unknown.

To address these knowledge gaps, we evaluated the effects of the New York sepsis regulations on pediatric sepsis outcomes, comparing temporal changes in sepsis outcomes in New York with those in other states that did not adopt sepsis regulations. In addition to examining outcomes in the overall population of pediatric sepsis, we examined key subgroups that may be differentially impacted by the regulations, including subgroups based on age, the presence of chronic conditions, and hospital type.

We performed a retrospective cohort study of children hospitalized with sepsis in New York and 4 control states: Florida, Maryland, Massachusetts, and New Jersey. We used a quasi-experimental comparative interrupted time-series approach, examining longitudinal trends in sepsis outcomes before and after the New York sepsis regulations, comparing trends in New York with trends in control states.15,16  This approach is conceptually similar to difference-in-differences analysis and provides stronger inference than using data from New York alone.17 

We obtained patient-level sepsis hospitalization data from the Agency for Healthcare Research and Quality’s Healthcare Cost and Utilization Project State Inpatient Databases. These data were augmented with hospital-level characteristics by using data sets from the Centers for Medicare and Medicaid Services’ Healthcare Cost Reporting Information System (HCRIS), the American Hospital Association Annual Survey of Hospitals, and the Children’s Hospital Association membership database.

All analyses were prespecified, and a statistical analysis plan was prepublished on Open Science Framework on December 13, 2018 (see the Supplemental Information).18  Deviations from this plan are noted as post hoc, and a rationale for deviations is provided in the Supplemental Information. The University of Pittsburgh Human Research Protection Office reviewed and approved this study.

All hospital admissions for patients <18 years old with sepsis were initially eligible. We identified sepsis using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes. We used 2 sets of codes: the explicit ICD-9-CM codes for sepsis (785.52 and 995.92) and the ICD-9-CM codes for infection and organ failure.19  This approach is widely used in studies of pediatric sepsis epidemiology and captures a larger patient population with sepsis than the use of explicit sepsis codes alone.2  We excluded neonates born during the hospital encounter because they are not subject to the regulations. Additionally, we excluded patients missing data on key covariates.

We excluded hospitals not categorized as short-stay acute care hospitals, as defined in the HCRIS, and hospitals with no pediatric sepsis cases. Additionally, to create a more homogeneous sample between New York and control states, we stratified hospitals on the basis of key characteristics (hospital type, academic status, and the size of the regional population) and excluded hospital types that were not shared across New York and control states in both the pre- and postregulation period (see the Supplemental Information for more information on this process).

The primary outcome was in-hospital mortality by 30 days after the date of admission. We also examined 4 secondary outcomes: ICU admission rates; hospital length of stay; central venous catheter insertion rates, defined as traditional central venous catheters or peripherally inserted central catheters; and Clostridium difficile infection rates. These outcomes represent the intended effect of Rory’s Regulations (ie, lower sepsis-related mortality) as well as potential unintended effects (ie, increased resource use, increased use of invasive procedures, and overuse of antibiotics). Outcome variables were either obtained directly from the administrative record or defined by using validated ICD-9-CM codes (Supplemental Table 5).

Patient-level variables for risk-adjustment were based on previous studies of pediatric sepsis20  and included age, sex, race, admission source (ie, emergency department or transfer from an acute care hospital), organ failures present on admission,21  the presence of pediatric complex chronic conditions,22  and the season of admission.23 

Hospital-level variables for risk-adjustment included hospital type, hospital academic status, and the population of the hospital’s geographic area. To define hospital type, we categorized hospitals into 4 mutually exclusive groups: designated children’s hospital based on Children’s Hospital Association membership24 ; non–children’s hospital with high pediatric volume (for which we selected a cutoff of 1200 annual pediatric admissions, which was roughly equivalent to the smallest children’s hospital in our sample); non–children’s hospital with medium pediatric volume (for which we selected cutoffs of <1200 but >50 annual pediatric admissions); or non–children’s hospital with low pediatric volume (for which we selected a cutoff of ≤50 annual pediatric admissions, ie, <1 pediatric admission per week on average). Academic status was defined on the basis of the presence or absence of trainees by using data in the HCRIS. Geographic area population was defined by using regional population data based on the American Hospital Association Annual Survey of Hospitals: small (metropolitan statistical area [MSA] population <100 000 or non-MSA), medium (MSA population 100 000–1 million), or large (MSA population >1 million).

We examined hospital characteristics in New York versus control states using χ2 tests. We compared patient characteristics between New York and control states before and after the regulations by visually examining summary statistics; we did not perform statistical tests for these comparisons because our large sample sizes were likely to yield comparisons that were significant.

To examine the effects of the regulations, we performed a comparative interrupted time-series analysis in which we fit patient-level linear regression models with robust SEs.16  For each model, the dependent variable was the outcome of interest (described previously), and the independent variables were an indicator for the postregulation period, time (modeled as quarters), and state (New York versus control) and an interaction term for the indicators for state and postregulation implementation. We defined the preregulation period as January 1, 2011, to March 31, 2013, and the postregulation period as April 1, 2013, to September 30, 2015. Additional covariates included the complete set of patient- and hospital-level covariates defined previously. Detailed model specifications can be found in the Supplemental Information.

This model allowed us to estimate changes in outcomes over time in New York and control states before and after the regulations. Under this model, the primary test of the effect of the regulations was the differential change in slope between New York and control states (ie, the temporal trends in outcome could change with the regulations in New York and control states, but the change in New York would need to be significantly greater than the change in control states for the regulations to be considered to have a positive effect).

We examined the robustness of our findings by performing prespecified sensitivity analyses on our primary outcome. First, we repeated the analysis using 2 alternative ICD-9-CM coding schemas for sepsis: a broader definition by using additional codes for organ failure25  and a narrower definition by using only the explicit codes for sepsis and septic shock.2  Second, we repeated the analysis, excluding hospitals that participated in the Greater New York Hospital Association (GNYHA) Strengthening Treatment and Outcomes for Patients-Sepsis (STOP-SEPSIS) initiative, a sepsis-focused regional quality improvement initiative that began in 2010 before the implementation of Rory’s Regulations.26  To the degree that these hospitals had already adopted sepsis quality improvement efforts, including them might blunt any observed policy-related effects. Third, we repeated the analysis by moving the preregulation period back by 2 quarters to account for anticipatory practice changes in response to the coming policy implementation. Fourth, we performed an analysis in which the outcome variable was sepsis rates over time to examine the possibility that the regulations affected sepsis coding in a way that might bias the analysis.

We performed prespecified subgroup analyses, hypothesizing that the regulations might have had a differential impact on the basis of key patient and hospital-level factors. Subgroups of interest included patient age (categorized as <60 days, 60 days to 12 months, and 1–4, 5–9, 10–14, and 15–17 years), the number of complex chronic conditions (categorized as 0, 1, 2, and ≥3), the number of organ failures present on admission (categorized as ≤2 and >2), whether the patient was admitted through the emergency department, and hospital type, as categorized previously. The rationale for each subgroup and the prespecified hypotheses are provided in the Supplemental Information.

We conducted all analyses using StataMP version 15 (Stata Corp, College Station, TX). All tests for statistical significance were 2-tailed and evaluated at a significance level of α < .05.

After exclusions, there were 9436 pediatric patients with sepsis in 237 hospitals (Supplemental Fig 2). Hospitals in New York were more likely to be larger non–children’s hospitals as well as teaching hospitals when compared with hospitals in control states (Table 1). Patient characteristics were generally comparable between New York and control states before and after implementation of the regulations (Table 2). Most patients had at least 1 complex chronic condition, and a large majority of patients were cared for in specialized children’s hospitals or general hospitals with pediatric volumes equivalent to specialized children’s hospitals. Less than 10% of patients were cared for in general hospitals with small or medium pediatric volumes.

TABLE 1

Hospital Characteristics

CharacteristicNew York (N = 83), n (%)Control States (N = 154), n (%)P
Hospital typea   .03 
 General hospital, small pediatric volume 7 (8.4) 17 (11.0)  
 General hospital, medium pediatric volume 49 (59.0) 99 (64.3)  
 General hospital, large pediatric volume 15 (18.1) 10 (6.5)  
 Specialized children’s hospital 12 (14.5) 28 (18.2)  
MSA size   .19 
 <100 000 7 (8.4) 5 (3.3)  
 100 000–1 million 18 (21.7) 40 (26.0)  
 >1 million 58 (69.9) 109 (70.8)  
Teaching statusb   .03 
 Nonteaching 21 (25.3) 60 (39.0)  
 Teaching 62 (74.7) 94 (61.0)  
GNYHA    
 Yes 38 (45.8) n/a  
 No 45 (54.2) n/a  
CharacteristicNew York (N = 83), n (%)Control States (N = 154), n (%)P
Hospital typea   .03 
 General hospital, small pediatric volume 7 (8.4) 17 (11.0)  
 General hospital, medium pediatric volume 49 (59.0) 99 (64.3)  
 General hospital, large pediatric volume 15 (18.1) 10 (6.5)  
 Specialized children’s hospital 12 (14.5) 28 (18.2)  
MSA size   .19 
 <100 000 7 (8.4) 5 (3.3)  
 100 000–1 million 18 (21.7) 40 (26.0)  
 >1 million 58 (69.9) 109 (70.8)  
Teaching statusb   .03 
 Nonteaching 21 (25.3) 60 (39.0)  
 Teaching 62 (74.7) 94 (61.0)  
GNYHA    
 Yes 38 (45.8) n/a  
 No 45 (54.2) n/a  

n/a, not applicable.

a

Children’s hospitals based on membership in the Children’s Hospital Association. Non–children’s hospitals were categorized by the number of annual pediatric admissions (≥1200 = large volume; <1200 and >50 = medium volume; ≤50 = small volume).

b

Teaching status was defined by using the resident-to-bed ratio from the Centers for Medicare and Medicaid HCRIS (0 = nonteaching; >0 = teaching).

TABLE 2

Patient Characteristics

CharacteristicPreimplementation, n (%)Postimplementation, n (%)
New York, N = 1628Control States, N = 2473New York, N = 2336Control States, N = 2999
Agea     
 ≤60 d 322 (19.8) 249 (19.1) 351 (15.1) 221 (14.1) 
 61–364 d 203 (12.5) 122 (9.3) 276 (11.9) 161 (10.2) 
 1–4 y 311 (19.1) 255 (19.5) 469 (20.2) 358 (22.7) 
 5–9 y 233 (14.3) 197 (15.1) 393 (16.9) 258 (16.4) 
 10–14 y 297 (18.2) 218 (16.7) 446 (19.2) 285 (18.1) 
 15–17 y 262 (16.1) 266 (20.4) 393 (16.9) 292 (18.5) 
Sex     
 Male 848 (52.1) 1364 (55.2) 1246 (53.3) 1577 (52.6) 
 Female 780 (47.9) 1109 (44.8) 1090 (46.7) 1422 (47.4) 
Race and/or ethnicity     
 White 563 (34.6) 1056 (42.7) 854 (36.6) 1278 (42.6) 
 African American 294 (18.1) 596 (24.1) 482 (20.6) 794 (26.5) 
 Hispanic 322 (19.8) 539 (21.8) 423 (18.1) 616 (20.5) 
 Other 449 (27.6) 282 (11.4) 577 (24.7) 311 (10.4) 
Complex chronic conditions     
 0 382 (23.5) 595 (24.1) 606 (25.9) 711 (23.7) 
 1 412 (25.3) 580 (23.5) 557 (23.8) 746 (24.9) 
 2 395 (24.3) 602 (24.3) 482 (20.6) 681 (22.7) 
 3+ 439 (27.0) 696 (28.1) 691 (29.6) 861 (28.7) 
Organ failures on admission     
 0–2 1504 (92.4) 2314 (93.6) 2177 (93.2) 2765 (92.2) 
 3+ 124 (7.6) 159 (6.4) 159 (6.8) 234 (7.8) 
Admission source     
 Emergency department 884 (54.3) 1393 (56.3) 1453 (62.2) 1868 (62.3) 
 Other 744 (45.7) 1080 (43.7) 883 (37.8) 1131 (37.7) 
Hospital typeb     
 General hospital, small pediatric volume 3 (0.2) 9 (0.4) 8 (0.3) 14 (0.5) 
 General hospital, medium pediatric volume 113 (6.9) 205 (8.3) 122 (5.2) 282 (9.4) 
 General hospital, large pediatric volume 297 (18.2) 196 (7.9) 360 (15.4) 216 (7.2) 
 Specialized children’s hospital 1215 (74.6) 2063 (83.4) 1846 (79.0) 2487 (82.9) 
CharacteristicPreimplementation, n (%)Postimplementation, n (%)
New York, N = 1628Control States, N = 2473New York, N = 2336Control States, N = 2999
Agea     
 ≤60 d 322 (19.8) 249 (19.1) 351 (15.1) 221 (14.1) 
 61–364 d 203 (12.5) 122 (9.3) 276 (11.9) 161 (10.2) 
 1–4 y 311 (19.1) 255 (19.5) 469 (20.2) 358 (22.7) 
 5–9 y 233 (14.3) 197 (15.1) 393 (16.9) 258 (16.4) 
 10–14 y 297 (18.2) 218 (16.7) 446 (19.2) 285 (18.1) 
 15–17 y 262 (16.1) 266 (20.4) 393 (16.9) 292 (18.5) 
Sex     
 Male 848 (52.1) 1364 (55.2) 1246 (53.3) 1577 (52.6) 
 Female 780 (47.9) 1109 (44.8) 1090 (46.7) 1422 (47.4) 
Race and/or ethnicity     
 White 563 (34.6) 1056 (42.7) 854 (36.6) 1278 (42.6) 
 African American 294 (18.1) 596 (24.1) 482 (20.6) 794 (26.5) 
 Hispanic 322 (19.8) 539 (21.8) 423 (18.1) 616 (20.5) 
 Other 449 (27.6) 282 (11.4) 577 (24.7) 311 (10.4) 
Complex chronic conditions     
 0 382 (23.5) 595 (24.1) 606 (25.9) 711 (23.7) 
 1 412 (25.3) 580 (23.5) 557 (23.8) 746 (24.9) 
 2 395 (24.3) 602 (24.3) 482 (20.6) 681 (22.7) 
 3+ 439 (27.0) 696 (28.1) 691 (29.6) 861 (28.7) 
Organ failures on admission     
 0–2 1504 (92.4) 2314 (93.6) 2177 (93.2) 2765 (92.2) 
 3+ 124 (7.6) 159 (6.4) 159 (6.8) 234 (7.8) 
Admission source     
 Emergency department 884 (54.3) 1393 (56.3) 1453 (62.2) 1868 (62.3) 
 Other 744 (45.7) 1080 (43.7) 883 (37.8) 1131 (37.7) 
Hospital typeb     
 General hospital, small pediatric volume 3 (0.2) 9 (0.4) 8 (0.3) 14 (0.5) 
 General hospital, medium pediatric volume 113 (6.9) 205 (8.3) 122 (5.2) 282 (9.4) 
 General hospital, large pediatric volume 297 (18.2) 196 (7.9) 360 (15.4) 216 (7.2) 
 Specialized children’s hospital 1215 (74.6) 2063 (83.4) 1846 (79.0) 2487 (82.9) 
a

Values for age in control states exclude patients in Florida because of a lack of data availability (n in preimplementation period = 1575; n in postimplementation period = 1307).

b

Children’s hospitals based on membership in the Children’s Hospital Association. Non–children’s hospitals were categorized by the number of annual pediatric admissions (≥1200 = large volume; <1200 and >50 = medium volume; ≤50 = small volume).

In the unadjusted model, annual pediatric sepsis mortality decreased over the study period in New York and control states (New York before: 14.0%; New York after: 11.5%; control states before: 14.4%; control states after: 11.2%). In the fully adjusted model, changes in mortality trends contemporaneous with the regulations were not significantly different in New York compared with control states (Table 3). There was no difference in the temporal trends between the pre- and postregulation period in New York (difference in slopes: −0.79% per quarter; 95% confidence interval (CI): −1.59% to 0.01%) or control states (difference in slopes: 0.17% per quarter; 95% CI: −0.38% to 0.72%) (Fig 1). The primary test of the regulations’ effect was the difference between these 2 changes, which was not statistically significant (differential change in slopes: −0.96% per quarter; 95% CI: 1.95% to 0.02%; P = .06). The mortality in control states remained relatively flat before and after the regulations, whereas the mortality in New York was increasing before the regulations and then started decreasing after the regulations (Fig 1). However, none of these trends were statistically significant. There were no significant differences in any of our secondary outcomes (Supplemental Table 8, Supplemental Fig 3).

TABLE 3

Comparative Interrupted Time-Series Analysis for Risk-Adjusted In-Hospital Mortality at 30 Days, Both for the Primary Analysis and Sensitivity Analyses

CohortStudy GroupnPreimplementation Quarterly Trend, % (95% CI)Postimplementation Quarterly Trend, % (95% CI)Change in Quarterly Trend, % (95% CI)Differential Change, % (95% CI)P
Primary analysis New York 3964 0.23 (−0.43 to 0.89) −0.56 (−1.03 to 0.09) −0.79 (−1.59 to 0.01) −0.96 (−1.95 to 0.02) .06 
 Control 5472 −0.24 (−0.65 to 0.16) −0.07 (−0.52 to 0.38) 0.17 (−0.38 to 0.72) — — 
Broader case identification strategya New York 13 806 0.08 (−0.13 to 0.29) −0.09 (−0.30 to 0.12) −0.17 (−0.43 to 0.08) −0.19 (−0.50 to 0.13) .24 
 Control 23 877 0.01 (−0.14 to 0.16) 0.02 (−0.11 to 0.15) 0.01 (−0.17 to 0.19) — — 
Narrower case identification strategyb New York 2815 0.15 (−0.61 to 0.91) −0.46 (−0.95 to 0.03) −0.61 (−1.62 to 0.40) −1.11 (−2.31 to 0.09) .07 
 Control 3495 −0.41 (−0.93 to 0.12) 0.10 (−0.44 to 0.63) 0.50 (−0.13 to 1.13) — — 
Excluding hospitals in GNYHAc New York 1335 0.82 (−0.55 to 2.19) −1.08 (−1.68 to 0.47) −1.90 (−3.48 to 0.31) −2.08 (−3.79 to 0.37) .02 
 Control 5472 −0.92 (−3.50 to 1.66) −0.05 (−0.50 to 0.39) 0.18 (−0.37 to 0.73) — — 
Changing implementation quarterd New York 3964 −0.06 (−0.82 to 0.71) −0.53 (−0.95 to 0.11) −0.47 (−1.32 to 0.37) −0.94 (−1.95 to 0.07) .07 
 Control 5472 −0.64 (−1.25 to 0.03) −0.18 (−0.51 to 0.16) 0.47 (−0.08 to 1.02) — — 
CohortStudy GroupnPreimplementation Quarterly Trend, % (95% CI)Postimplementation Quarterly Trend, % (95% CI)Change in Quarterly Trend, % (95% CI)Differential Change, % (95% CI)P
Primary analysis New York 3964 0.23 (−0.43 to 0.89) −0.56 (−1.03 to 0.09) −0.79 (−1.59 to 0.01) −0.96 (−1.95 to 0.02) .06 
 Control 5472 −0.24 (−0.65 to 0.16) −0.07 (−0.52 to 0.38) 0.17 (−0.38 to 0.72) — — 
Broader case identification strategya New York 13 806 0.08 (−0.13 to 0.29) −0.09 (−0.30 to 0.12) −0.17 (−0.43 to 0.08) −0.19 (−0.50 to 0.13) .24 
 Control 23 877 0.01 (−0.14 to 0.16) 0.02 (−0.11 to 0.15) 0.01 (−0.17 to 0.19) — — 
Narrower case identification strategyb New York 2815 0.15 (−0.61 to 0.91) −0.46 (−0.95 to 0.03) −0.61 (−1.62 to 0.40) −1.11 (−2.31 to 0.09) .07 
 Control 3495 −0.41 (−0.93 to 0.12) 0.10 (−0.44 to 0.63) 0.50 (−0.13 to 1.13) — — 
Excluding hospitals in GNYHAc New York 1335 0.82 (−0.55 to 2.19) −1.08 (−1.68 to 0.47) −1.90 (−3.48 to 0.31) −2.08 (−3.79 to 0.37) .02 
 Control 5472 −0.92 (−3.50 to 1.66) −0.05 (−0.50 to 0.39) 0.18 (−0.37 to 0.73) — — 
Changing implementation quarterd New York 3964 −0.06 (−0.82 to 0.71) −0.53 (−0.95 to 0.11) −0.47 (−1.32 to 0.37) −0.94 (−1.95 to 0.07) .07 
 Control 5472 −0.64 (−1.25 to 0.03) −0.18 (−0.51 to 0.16) 0.47 (−0.08 to 1.02) — — 

—, not applicable.

a

In this analysis, we use a broader case identification strategy for sepsis on the basis of a larger subset of codes for organ failure, resulting in a larger patient population with lower baseline mortality compared with the primary strategy.25 

b

In this analysis, we use a narrower case identification strategy for sepsis on the basis of only the explicit codes for sepsis and septic shock, resulting in a smaller patient population with higher baseline mortality compared with the primary strategy.2 

c

In this analysis, we exclude New York hospitals that participated in the GNYHA’s STOP-SEPSIS initiative, a sepsis-focused regional quality improvement initiative that began in 2010 before the implementation of the New York State regulations.26 

d

In this analysis, we use a different definition of the preregulation period, moving it back by 2 quarters to account for anticipatory practice changes in response to the coming policy implementation.

FIGURE 1

Quarter-specific 30-day in-hospital sepsis mortality over time in New York and control states. A, Unadjusted: quarter-specific unadjusted mortality, with the regulation implementation noted as a vertical dotted line. B, Adjusted: quarter-specific adjusted mortality, with the regulation implementation noted as a vertical dotted line. In the prespecified model, linear mortality trends were estimated separately for both the pre- and postregulation periods, allowing for a discrete change in mortality just after the regulations (as indicated by the gray shaded bar).

FIGURE 1

Quarter-specific 30-day in-hospital sepsis mortality over time in New York and control states. A, Unadjusted: quarter-specific unadjusted mortality, with the regulation implementation noted as a vertical dotted line. B, Adjusted: quarter-specific adjusted mortality, with the regulation implementation noted as a vertical dotted line. In the prespecified model, linear mortality trends were estimated separately for both the pre- and postregulation periods, allowing for a discrete change in mortality just after the regulations (as indicated by the gray shaded bar).

Close modal

In a prespecified sensitivity analysis excluding the 38 hospitals in metropolitan New York (45.8% of New York hospitals) that participated in the GNYHA STOP-SEPSIS quality initiative, we found that the regulations were associated with significant improvements in mortality trends in New York compared with control states (differential change in slopes: −2.08% per quarter; 95% CI: −3.79% to −0.37%; P = .02) (Table 3). There were no significant differences in mortality trends by using different case identification strategies or after moving the implementation period up 2 quarters to account for potential anticipatory changes. There was no differential change in the rate of pediatric sepsis after implementation of the regulations in New York compared with control states, suggesting that the regulations did not influence sepsis rates in a way that influenced our analysis (Supplemental Table 9).

The results of the prespecified subgroup analyses in which we examined risk-adjusted mortality trends stratified by New York and control states are shown in Table 4. After implementation of the regulations, New York experienced statistically significantly greater decreases in mortality among patients who were previously healthy (differential change in slopes: −1.36%; 95% CI: −2.62% to −0.09%; P = .04) and those who were not admitted through the emergency department (differential change in slopes: −2.42%; 95% CI: −4.23% to −0.61%; P = .01). There were no significant differences in mortality trends based on patient age categories, the number of organ failures present on admission, the type of hospital the child was admitted to, or the hospital’s teaching status. We did observe a clinically significant difference in patients admitted to small- and medium-sized non–children’s hospitals, although only 8% of patients were admitted to these types of hospitals, and the difference was not statistically significant (differential change in slopes: −2.37%; 95% CI: −4.91% to 0.18%; P = .07).

TABLE 4

Comparative Interrupted Time-Series Analysis for Risk-Adjusted In-Hospital Mortality at 30 Days for Prespecified Subgroups

CohortChange in Quarterly Trend After Implementation, % (95% CI)Differential Change, % (95% CI)P
New YorkControl
Age     
 <60 d −0.41 (−2.04 to 1.22) −1.83 (−5.24 to 1.58) 1.42 (−2.15 to 5.00) .43 
 3–11 mo −1.09 (−3.05 to 0.87) −0.12 (−3.99 to 3.76) −0.97 (−5.43 to 3.49) .66 
 1–4 y −0.13 (−1.47 to 1.21) −0.49 (−2.21 to 1.22) 0.36 (−1.89 to 2.62) .75 
 5–9 y −0.25 (−2.55 to 2.05) 2.10 (0.55 to 3.66) −2.35 (−5.08 to 0.37) .09 
 10–14 y −1.68 (−3.06 to 0.30) −0.05 (−2.42 to 2.32) −1.63 (−4.47 to 1.21) .26 
 15–17 y −0.44 (−2.37 to 1.48) 0.44 (−0.98 to 1.86) −0.88 (−3.32 to 1.55) .47 
Complex chronic conditions     
 None, previously healthy −1.02 (−2.01 to 0.04) 0.33 (−0.50 to 1.16) −1.36 (−2.62 to 0.09) .04 
 1 −0.41 (−1.64 to 0.82) 0.55 (−0.86 to 1.97) −0.96 (−2.81 to 0.89) .31 
 2 −2.37 (−4.04 to 0.69) 0.20 (−1.61 to 2.01) −2.57 (−5.05 to 0.09) .04 
 3+ 1.15 (−0.49 to 2.80) −0.34 (−1.56 to 0.88) 1.49 (−0.65 to 3.65) .17 
Organ failures on admission     
 0–2 −0.63 (−1.46 to 0.20) 0.14 (−0.44 to 0.72) −0.77 (−1.79 to 0.25) .14 
 3+ 1.74 (−5.91 to 2.43) 0.60 (−3.85 to 5.06) −2.34 (−8.43 to 3.74) .45 
Admission source     
 Emergency department 0.10 (−0.65 to 0.85) 0.02 (−0.58 to 0.53) 0.13 (−0.81 to 1.06) .79 
 Direct admission −1.83 (−3.13 to 0.52) 0.60 (−0.69 to 1.88) −2.42 (−4.23 to 0.61) .01 
Hospital type     
 Children’s hospital −0.42 (−1.26 to 0.42) 0.30 (−0.32 to 0.91) −0.72 (−1.77 to 0.33) .18 
 General hospital, large pediatric volume −1.67 (−4.05 to 0.71) −0.75 (−3.67 to 2.16) −0.92 (−4.66 to 2.82) .62 
 General hospital, medium and small pediatric volume −2.64 (−4.73 to 0.55) −0.27 (−1.83 to 1.28) −2.37 (−4.91 to 0.18) .07 
Hospital teaching status     
 Nonteaching −0.70 (−1.50 to 0.09) 0.21 (−0.36 to 0.77) −0.91 (−1.90 to 0.08) .07 
 Teaching −8.39 (−15.26 to 1.52) −1.30 (−3.28 to 0.69) −7.10 (−14.04 to 0.15) .05 
CohortChange in Quarterly Trend After Implementation, % (95% CI)Differential Change, % (95% CI)P
New YorkControl
Age     
 <60 d −0.41 (−2.04 to 1.22) −1.83 (−5.24 to 1.58) 1.42 (−2.15 to 5.00) .43 
 3–11 mo −1.09 (−3.05 to 0.87) −0.12 (−3.99 to 3.76) −0.97 (−5.43 to 3.49) .66 
 1–4 y −0.13 (−1.47 to 1.21) −0.49 (−2.21 to 1.22) 0.36 (−1.89 to 2.62) .75 
 5–9 y −0.25 (−2.55 to 2.05) 2.10 (0.55 to 3.66) −2.35 (−5.08 to 0.37) .09 
 10–14 y −1.68 (−3.06 to 0.30) −0.05 (−2.42 to 2.32) −1.63 (−4.47 to 1.21) .26 
 15–17 y −0.44 (−2.37 to 1.48) 0.44 (−0.98 to 1.86) −0.88 (−3.32 to 1.55) .47 
Complex chronic conditions     
 None, previously healthy −1.02 (−2.01 to 0.04) 0.33 (−0.50 to 1.16) −1.36 (−2.62 to 0.09) .04 
 1 −0.41 (−1.64 to 0.82) 0.55 (−0.86 to 1.97) −0.96 (−2.81 to 0.89) .31 
 2 −2.37 (−4.04 to 0.69) 0.20 (−1.61 to 2.01) −2.57 (−5.05 to 0.09) .04 
 3+ 1.15 (−0.49 to 2.80) −0.34 (−1.56 to 0.88) 1.49 (−0.65 to 3.65) .17 
Organ failures on admission     
 0–2 −0.63 (−1.46 to 0.20) 0.14 (−0.44 to 0.72) −0.77 (−1.79 to 0.25) .14 
 3+ 1.74 (−5.91 to 2.43) 0.60 (−3.85 to 5.06) −2.34 (−8.43 to 3.74) .45 
Admission source     
 Emergency department 0.10 (−0.65 to 0.85) 0.02 (−0.58 to 0.53) 0.13 (−0.81 to 1.06) .79 
 Direct admission −1.83 (−3.13 to 0.52) 0.60 (−0.69 to 1.88) −2.42 (−4.23 to 0.61) .01 
Hospital type     
 Children’s hospital −0.42 (−1.26 to 0.42) 0.30 (−0.32 to 0.91) −0.72 (−1.77 to 0.33) .18 
 General hospital, large pediatric volume −1.67 (−4.05 to 0.71) −0.75 (−3.67 to 2.16) −0.92 (−4.66 to 2.82) .62 
 General hospital, medium and small pediatric volume −2.64 (−4.73 to 0.55) −0.27 (−1.83 to 1.28) −2.37 (−4.91 to 0.18) .07 
Hospital teaching status     
 Nonteaching −0.70 (−1.50 to 0.09) 0.21 (−0.36 to 0.77) −0.91 (−1.90 to 0.08) .07 
 Teaching −8.39 (−15.26 to 1.52) −1.30 (−3.28 to 0.69) −7.10 (−14.04 to 0.15) .05 

In an evaluation of a novel state-level health policy designed to improve pediatric sepsis outcomes, the totality of the results suggests that there was an overall decrease in risk-adjusted mortality associated with the regulations. This decrease was particularly significant in key patient subpopulations when compared with control states that did not implement the regulations, specifically previously healthy children and children directly admitted to the hospital. In addition, after excluding New York hospitals that participated in a previous regional sepsis-focused quality improvement initiative, implementation of the regulations was associated with statistically significant improvements in sepsis outcomes. Although not every analysis demonstrated a statistically significant reduction in mortality, when interpreted in context, these findings indicate a high likelihood that the regulations were associated with improved sepsis outcomes in New York State.

Given these results, New York State sepsis regulations have implications for wider sepsis regulation implementation. Given that a large majority of children with sepsis receive care at specialized children’s hospitals, many of which are already well prepared to care for a wide range of pediatric emergencies,2729  regulations for all hospitals to maintain pediatric sepsis readiness regardless of case volume may be too blunt an instrument to address pediatric sepsis mortality. It is notable that we observed a clinically significant, although not statistically significant, effect in non–children’s hospitals with relatively low pediatric admissions. It is also possible that effects of the regulations might have been dampened by larger efforts to improve pediatric sepsis quality in New York and other states.30  Indeed, in a prespecified sensitivity analysis excluding New York hospitals that voluntarily participated in a previous regional sepsis-focused quality improvement initiative in pediatric sepsis, we did observe a statistically significant effect of the regulations.

These results should be interpreted in the context of other data published on pediatric sepsis outcomes in New York.14  These data, after the regulations, reveal completion of a sepsis bundle: obtaining a blood culture, administration of broad-spectrum antibiotics, and a fluid bolus within one hour of identification of sepsis was associated with lower in-hospital mortality.14  Our study differs from that previous study in that we evaluated the effect of the regulations mandating early sepsis care, not the effects of early sepsis care itself. Early sepsis care can improve outcomes, and policy can drive efforts to implement evidence-based sepsis care.

These results also suggest similar improvements in sepsis mortality seen in adult patients in New York after implementation of the regulations.31  Although the regulations include a mandate for pediatric-specific sepsis protocols, the data underlying pediatric sepsis care are far less robust than those for adult sepsis care. The lack of consensus surrounding evidence-based pediatric sepsis care may contribute to variation in care protocols and less definitive population-level differential outcomes after implementation of Rory’s Regulations. In addition, qualitative evaluation of hospitals’ operationalization of the regulations may unearth mechanisms that lead to improvement in specific populations of patients.

Our results provide insight into strategies to refine health policy for pediatric sepsis. Predominantly, our study reveals that efforts to mandate pediatric-focused sepsis quality improvement should be balanced against the current landscape, in which pediatric sepsis patients are most often admitted to large hospitals and children’s hospitals rather than small community hospitals. Policies targeted to specific types of hospitals or policies designed to direct children with suspected sepsis to children’s hospitals in the prehospital period (ie, regionalization) may be more beneficial.32,33  Secondly, our study suggests that these policies may be most impactful in the absence of strong regional quality improvement dedicated to pediatric sepsis. When active regional quality improvement efforts are in place, government regulation may not be necessary.30,3436  Still, our study provides important support for regulations that can positively affect pediatric sepsis outcomes.

This study has several limitations. First, in this study, we used administrative data to identify sepsis. With this method, we may have failed to identify some patients with sepsis. However, when we used different identification strategies, analyses yielded similar results as our primary analysis. Additionally, there is the possibility that the administrative codes for sepsis were used differently over time; however, we did not find evidence of a differential change in the rates of sepsis coding during the study period in New York compared with control states (Supplemental Table 9). Second, our results could be sensitive to our modeling approach or our choice of control states. However, prepublication of our statistical analysis plan likely reduced the likelihood that our results are an artifact of our statistical methodology.37  In addition, we were unable to control for sepsis quality improvement initiatives in control states implemented toward the end of the study period. Given the study method, we believe the existence of these small-scale efforts serves to strengthen the generalizability of our findings because it is likely that other states considering whether to adopt these regulations would contain some hospitals engaging in local efforts. Third, because of data limitations, we could not examine postdischarge outcomes such as long-term morbidity and mortality. These outcomes are important to patients and will be important for future evaluations.

In New York, the implementation of statewide sepsis regulations was generally associated with improved mortality trends, particularly in subpopulations of pediatric patients, compared with control states that did not implement sepsis regulations. Refinement of statewide sepsis care policies may be needed to further influence outcomes for all pediatric patients with sepsis.

Drs Gigli, Davis, and Kahn conducted the analysis and drafted the initial manuscript; and all authors conceptualized and designed the study, interpreted the study results, critically reviewed the manuscript for important intellectual content, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.

FUNDING: Funded by the US Agency for Healthcare Research and Quality (R01HS025146) and the US National Institutes of Health (T32HL007820). Funded by the National Institutes of Health (NIH).

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2020-1525.

     
  • CI

    confidence interval

  •  
  • GNYHA

    Greater New York Hospital Association

  •  
  • HCRIS

    Healthcare Cost Reporting Information System

  •  
  • ICD-9-CM

    International Classification of Diseases, Ninth Revision, Clinical Modification

  •  
  • MSA

    metropolitan statistical area

  •  
  • STOP-SEPSIS

    Strengthening Treatment and Outcomes for Patients-Sepsis

1
Singer
M
,
Deutschman
CS
,
Seymour
CW
, et al
.
The third international consensus definitions for sepsis and septic shock (Sepsis-3)
.
JAMA
.
2016
;
315
(
8
):
801
810
2
Balamuth
F
,
Weiss
SL
,
Neuman
MI
, et al
.
Pediatric severe sepsis in US children’s hospitals
.
Pediatr Crit Care Med
.
2014
;
15
(
9
):
798
805
3
Hartman
ME
,
Linde-Zwirble
WT
,
Angus
DC
,
Watson
RS
.
Trends in the epidemiology of pediatric severe sepsis*
.
Pediatr Crit Care Med
.
2013
;
14
(
7
):
686
693
4
Rhodes
A
,
Evans
LE
,
Alhazzani
W
, et al
.
Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016
.
Intensive Care Med
.
2017
;
43
(
3
):
304
377
5
Massachusetts Sepsis Consortium
.
State-based strategies for combating sepsis
. Available at: https://www.betsylehmancenterma.gov/assets/uploads/BLC_Sepsis_Landscape_FinalEdit02_v21.pdf. Accessed August 4, 2019
6
Cooke
CR
,
Iwashyna
TJ
.
Sepsis mandates: improving inpatient care while advancing quality improvement
.
JAMA
.
2014
;
312
(
14
):
1397
1398
7
Hershey
TB
,
Kahn
JM
.
State sepsis mandates - a new era for regulation of hospital quality
.
N Engl J Med
.
2017
;
376
(
24
):
2311
2313
8
Medical staff, 10 NYCRR §405.4 (2013)
. Available at: https://regs.health.ny.gov/content/section-4054-medical-staff. Accessed August 4, 2019
9
Rhee
C
,
Gohil
S
,
Klompas
M
.
Regulatory mandates for sepsis care–reasons for caution
.
N Engl J Med
.
2014
;
370
(
18
):
1673
1676
10
Schlapbach
LJ
,
Weiss
SL
,
Wolf
J
.
Reducing collateral damage from mandates for time to antibiotics in pediatric sepsis-primum non nocere
.
JAMA Pediatr
.
2019
;
173
(
5
):
409
410
11
Weiss
SL
,
Fitzgerald
JC
,
Pappachan
J
, et al;
Sepsis Prevalence, Outcomes, and Therapies (SPROUT) Study Investigators and Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network
.
Global epidemiology of pediatric severe sepsis: the sepsis prevalence, outcomes, and therapies study [published correction appears in Am J Respir Crit Care Med 2016;193(2):223–224]
.
Am J Respir Crit Care Med
.
2015
;
191
(
10
):
1147
1157
12
Ames
SG
,
Davis
BS
,
Marin
JR
, et al
.
Emergency department pediatric readiness and mortality in critically ill children
.
Pediatrics
.
2019
;
144
(
3
):
e20190568
13
Evans
IVR
,
Watson
RS
,
Carcillo
J
,
Angus
DC
,
Seymour
CW
.
Epidemiology of sepsis among adolescents at community hospital emergency departments: implications for Rory’s regulations
.
JAMA Pediatr
.
2017
;
171
(
10
):
1011
1012
14
Evans
IVR
,
Phillips
GS
,
Alpern
ER
, et al
.
Association between the New York sepsis care mandate and in-hospital mortality for pediatric sepsis
.
JAMA
.
2018
;
320
(
4
):
358
367
15
Jacob
R
,
Somers
MA
,
Zhu
P
,
Bloom
H
.
The validity of the comparative interrupted time series design for evaluating the effect of school-level interventions
.
Eval Rev
.
2016
;
40
(
3
):
167
198
16
Penfold
RB
,
Zhang
F
.
Use of interrupted time series analysis in evaluating health care quality improvements
.
Acad Pediatr
.
2013
;
13
(
suppl 6
):
S38
S44
17
Dimick
JB
,
Ryan
AM
.
Methods for evaluating changes in health care policy: the difference-in-differences approach
.
JAMA
.
2014
;
312
(
22
):
2401
2402
18
Kahn
JM
.
Quantitative evaluation of the effects the 2013 New York State sepsis regulations
. Available at: https://osf.io/76u3x. Accessed August 4, 2019
19
Dombrovskiy
VY
,
Martin
AA
,
Sunderram
J
,
Paz
HL
.
Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003
.
Crit Care Med
.
2007
;
35
(
5
):
1244
1250
20
Ames
SG
,
Davis
BS
,
Angus
DC
,
Carcillo
JA
,
Kahn
JM
.
Hospital variation in risk-adjusted pediatric sepsis mortality
.
Pediatr Crit Care Med
.
2018
;
19
(
5
):
390
396
21
Elias
KM
,
Moromizato
T
,
Gibbons
FK
,
Christopher
KB
.
Derivation and validation of the acute organ failure score to predict outcome in critically ill patients: a cohort study
.
Crit Care Med
.
2015
;
43
(
4
):
856
864
22
Feudtner
C
,
Christakis
DA
,
Connell
FA
.
Pediatric deaths attributable to complex chronic conditions: a population-based study of Washington State, 1980-1997
.
Pediatrics
.
2000
;
106
(
1 pt 2
):
205
209
23
Danai
PA
,
Sinha
S
,
Moss
M
,
Haber
MJ
,
Martin
GS
.
Seasonal variation in the epidemiology of sepsis
.
Crit Care Med
.
2007
;
35
(
2
):
410
415
24
Children’s Hospital Association
.
Children’s hospital directory
. Available at: https://www.childrenshospitals.org/Directories/Hospital-Directory. Accessed August 4, 2019
25
Iwashyna
TJ
,
Odden
A
,
Rohde
J
, et al
.
Identifying patients with severe sepsis using administrative claims: patient-level validation of the Angus implementation of the international consensus conference definition of severe sepsis
.
Med Care
.
2014
;
52
(
6
):
e39
e43
26
Greater New York Hospital Association
.
Sepsis
. Available at: https://www.gnyha.org/topic/sepsis/. Accessed August 4, 2019
27
Berry
JG
,
Hall
M
,
Hall
DE
, et al
.
Inpatient growth and resource use in 28 children’s hospitals: a longitudinal, multi-institutional study
.
JAMA Pediatr
.
2013
;
167
(
2
):
170
177
28
Gupta
P
,
Rettiganti
M
,
Fisher
PL
,
Chang
AC
,
Rice
TB
,
Wetzel
RC
.
Association of freestanding children’s hospitals with outcomes in children with critical illness
.
Crit Care Med
.
2016
;
44
(
12
):
2131
2138
29
Hsu
BS
,
Schimelpfenig
M
,
Lakhani
S
.
Comparison of transferred versus nontransferred pediatric patients admitted for sepsis
.
Air Med J
.
2016
;
35
(
1
):
43
45
30
Huff
C
.
Preventing sepsis by reimagining systems and engaging patients
.
Health Aff (Millwood)
.
2019
;
38
(
5
):
704
708
31
Kahn
JM
,
Davis
BS
,
Yabes
JG
, et al
.
Association between state-mandated protocolized sepsis care and in-hospital mortality among adults with sepsis
.
JAMA
.
2019
;
322
(
3
):
240
250
32
França
UL
,
McManus
ML
.
Trends in regionalization of hospital care for common pediatric conditions
.
Pediatrics
.
2018
;
141
(
1
):
e20171940
33
Ray
KN
,
Olson
LM
,
Edgerton
EA
, et al
.
Access to high pediatric-readiness emergency care in the United States
.
J Pediatr
.
2018
;
194
:
225
232.e1
34
Balamuth
F
,
Weiss
SL
,
Fitzgerald
JC
, et al
.
Protocolized treatment is associated with decreased organ dysfunction in pediatric severe sepsis
.
Pediatr Crit Care Med
.
2016
;
17
(
9
):
817
822
35
Cruz
AT
,
Perry
AM
,
Williams
EA
,
Graf
JM
,
Wuestner
ER
,
Patel
B
.
Implementation of goal-directed therapy for children with suspected sepsis in the emergency department
.
Pediatrics
.
2011
;
127
(
3
). Available at: www.pediatrics.org/cgi/content/full/127/3/e758
36
Larsen
GY
,
Mecham
N
,
Greenberg
R
.
An emergency department septic shock protocol and care guideline for children initiated at triage
.
Pediatrics
.
2011
;
127
(
6
). Available at: www.pediatrics.org/cgi/content/full/127/6/e1585
37
Thomas
L
,
Peterson
ED
.
The value of statistical analysis plans in observational research: defining high-quality research from the start
.
JAMA
.
2012
;
308
(
8
):
773
774
38
Feudtner
C
,
Feinstein
JA
,
Zhong
W
,
Hall
M
,
Dai
D
.
Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation
.
BMC Pediatr
.
2014
;
14
:
199

Competing Interests

POTENTIAL CONFLICT OF INTEREST: Dr Angus received personal fees from Ferring Pharmaceutical Inc, Bristol-Myers Squibb, Bayer AG, GenMark Diagnostics, Sobi Inc, Beckman Coulter Inc, and ALung Technologies Inc and has patents pending from Selepressin and Proteomic; the other authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: Dr Angus received personal fees from Ferring Pharmaceutical Inc, Bristol-Myers Squibb, Bayer AG, GenMark Diagnostics, Sobi Inc, Beckman Coulter Inc, and ALung Technologies Inc and has patents pending from Selepressin and Proteomic; the other authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data