Prevalence of Sepsis as Defined by Phoenix Sepsis Definition Among Children With COVID-19 Free

Worldwide, sepsis is a significant contributor to morbidity and mortality in children.¹ The International Pediatric Sepsis Consensus Conference (IPSCC) developed pediatric sepsis criteria in 2005.² IPSCC defined pediatric sepsis as the presence of 2 or more systemic inflammatory response syndrome (SIRS) criteria in the setting of suspected or confirmed infection. With a greater understanding of the pathophysiology of sepsis, SIRS is now considered a physiologic host response to infection rather than a pathological state.³ This led to the modified adult sepsis definition in 2014 (Sepsis-3), which is based on the Sequential Organ Failure Assessment score⁴ that defined sepsis as a “life-threatening organ dysfunction caused by a dis-regulated host response to infection.” Cognizant of the limitations of the IPSCC definition, an international task force revised the pediatric sepsis definition as Phoenix Sepsis Criteria (PSC), which is based on a novel Phoenix Sepsis Score (PSS). The PSS is a composite 4-organ system model, including cardiovascular, respiratory, neurological, and coagulation dysfunction criteria. Sepsis was defined as 2 or more points on the PSS, septic shock was defined as sepsis and one or more cardiovascular points.^5,6 However, as a novel score, its transportability in patient populations other than those used in its development has not yet been evaluated.

Serious infections from the novel coronavirus (SARS-CoV-2) are relatively rare in children compared with older adults.⁷ However, similar to adults,⁸ severe COVID-19 presentation shares characteristics of sepsis and septic shock. As such, organ dysfunction from COVID-19 can be considered as a phenotype of pediatric sepsis or septic shock.^8–10 In addition, a significant cause of organ dysfunction from SARS-CoV-2 in children is a postinfectious syndrome (multisystem inflammatory syndrome in children [MIS-C]) that can present a few weeks after acute infection.¹¹ Close to 50% of patients with MIS-C require intensive care unit (ICU)-level care, 38% present with shock, and 30% present with cardiac dysfunction.¹² Severe COVID-19 and MIS-C, during its initial presentation, are often indistinguishable from sepsis.^13–15 Since the adult equivalent of PSC (Sepsis-3) was published before the pandemic, there have been multiple reports on sepsis prevalence in adult COVID-19 patients, with a meta-analysis identifying a sepsis prevalence of 78% in the ICU and 33% in general floor patients admitted for COVID-19 based on the Sepsis-3 criteria.¹⁶ However, the dataset used to derive and validate the PSS did not include patients with COVID-19, and so far, no study has been published regarding the prevalence of sepsis in children with COVID-19 using the PSC.

The primary objective of this study was to calculate the prevalence of sepsis as defined by PSC among patients who were admitted with COVID-19 or MIS-C (henceforth referred to only as COVID-19 unless specified) in the participating centers in the Society of Critical Care Medicine: Viral Infection and Respiratory Illness Universal Study (SCCM: VIRUS) COVID-19 registry and to validate the categorization of sepsis by PSC among children with COVID-19 with respect to outcomes (mortality, hospital length of stay). We hypothesize that PSS will identify patients with sepsis and septic shock in patients with COVID-19 and has equivalent sensitivity and specificity in predicting the outcome as in non–COVID-19 pediatric sepsis.

For this retrospective medical record review, we obtained data from the SCCM: COVID-19 VIRUS registry. The VIRUS registry was established at the onset of the COVID-19 pandemic (March 2020) and includes patients admitted to the hospital with suspected or confirmed COVID-19 or associated conditions.^17,18 This deidentified registry has been approved by the Mayo Clinic Institutional Review Board (Mayo Clinic IRB #20-002610) and the IRBs of all participating centers. It is also registered at ClinicalTrials.gov (NCT04323787). This ancillary study was separately reviewed by the IRB at the University of Illinois College of Medicine at Peoria and approved as an exempt study (IRB #2186155)

In this study, we included all patients with laboratory-confirmed COVID-19 or who met the Centers for Disease Control and Prevention definition of MIS-C¹⁹ who were admitted within the participating institutions of the Discovery VIRUS COVID-19 registry and entered into the registry from its inception. We excluded patients aged 18 years or older or those with missing age. Patients with missing confirmatory diagnoses of COVID-19 or MIS-C and those marked explicitly as incidental diagnoses of COVID-19 were excluded. Furthermore, patients with missing data on all PSS variables were excluded per the original study.⁵ 

Variables extracted from the VIRUS registry included demographics, presenting vitals, laboratory values, and outcomes. Body mass index percentiles using Centers for Disease Control and Prevention and World Health Organization criteria²⁰ were calculated using the childsds package in R,²¹ and patients were classified as obese or nonobese. Vital signs in the VIRUS registry were entered as the highest and the lowest values for the day (for example, highest and lowest heart rate). These included vitals on admission (first available data at admission) and then daily. Similarly, for laboratory values, the first laboratory values were entered on admission, and then daily (worst value as applicable) values were entered. All the variables required for calculating the PSS were entered into the VIRUS registry often at multiple places. To ensure maximal data capture, a derived variable was created that included their minimal or maximum values as required to calculate the PSS. The PSS is derived from vitals and laboratory parameters within the first 24 hours of admission. The VIRUS registry includes data from day 0 (time to admission up to midnight) and day 1. As a deidentified registry, no timestamp was available, and thus, identifying the first 24 hours of hospital admission was not possible. Therefore, we included variables entered on day 0 or day 1 for this analysis and their smallest/largest value as applicable. For PF (Pao₂/FiO₂) and SF (Sao₂/FiO₂) ratios, only when Pao₂ or Sao₂ and FiO₂ were explicitly marked as contemporary were they obtained. Outcome variables included hospital mortality, length of stay in the hospital and ICU, and ventilator duration. The length of stay and ventilator duration for patients who died were excluded from the respective analysis. Laboratory values were entered in different units in the VIRUS registry. All the laboratory values were harmonized into those needed to calculate the PSS.

Two investigators independently calculated PSS for each patient using JMP (Dr Tripathi) and R (Mr McGarvey). JMP scripts for calculating cardiovascular, respiratory, coagulation, and neurological PSS were specifically written for this analysis, and R codes were adapted from CRAN (CRAN: Package Phoenix; r-project.org).²² The scores created by the 2 programs were matched, and errors in the codes were reconciled. These were combined to create the ordinal PSS and then categorized as nonsepsis, sepsis, and septic shock per the requirements outlined in the PSS calculation.⁵ 

Missing data analysis was performed to calculate the total number and proportion of missing data on the 10 variables needed to calculate the PSS. The use of inotropes in the data obtained from the VIRUS registry was available as “checked or unchecked,” so true missing data on inotrope use was not available. If a patient had inotrope use “checked,” then it was not counted as missing, and these patients were included in the analysis even if the rest of the variables were missing (11 patients). As described in the PSS,⁵ no data imputation was performed, and missingness was assumed to be not at random (missing data = normal value = a score of 0 in PSS calculation).²³ The highest number of missing data was in lactate level (85.4%), and the lowest was in platelet count (21.7%). Details of missingness are provided in Supplemental Materials.

Standard summary statistics, including median (IQR) and number (proportion), were calculated. Demographics and outcomes were compared using the chi-square or Fisher exact tests for categorical variables and Wilcoxon rank sum and Kruskal-Wallis tests for continuous variables. Post hoc pairwise comparisons were performed with a Holm-Bonferroni correction for multiple comparisons. All of the 2 group comparisons were performed with the nonseptic group (vs sepsis or septic shock group). Mixed linear effect models were used to examine the relationship between PSS and hospital length of stay (HLOS) while controlling for potential confounding variables. The HLOS variables were right skewed, so log transformations were used to meet the assumptions of linear regression, and parameter estimates were exponentiated. Exponentiated parameter estimates can be interpreted similarly to odds ratios. Variables included in the model were based on evidence from the literature on the causal relationship between severity of illness and HLOS from COVID-19. They included age in years, country (United States vs non–United States), diagnosis of MIS-C, comorbidities (yes/no), presence of obesity, bacterial coinfection, race, and ethnicity (as a surrogate for socioeconomic disparities).²⁴ Missing values in the control variables, specifically MIS-C, obesity, and bacterial coinfections, were coded as missing and included in the analysis. If the missing category was not significantly different from the reference (negative) category, the reference and missing categories were combined. A random intercept for the hospital was included for all adjusted analyses to account for clustering at the hospital level. A “base” model for HLOS with only sepsis categories (without covariates) was also created. The performance of PSS in discriminating mortality was assessed using the receiver operating curve and precision-recall curve. Cross-validation was performed using leave-one-out cross-validation (LOOCV). Supplemental separate analyses were also conducted for patients with and without MIS-C. Statistical analysis was performed using JMP Pro V17 (SAS Institute) and open-source statistical program R.²⁵ 

Out of the 2564 children entered into the registry during the study period, 833 patients were excluded based on predefined criteria. The remaining 1731 (167 [9.6%] non–United States) patients were included in the analysis. Among these patients, 326 (18.8%) met the criteria for sepsis, and 167 (9.7%) met the criteria for septic shock as per PSC (Figure 1). The numerical PSS ranged from 0 to 10 (out of a maximum possible 13), with a large majority of patients in the sepsis group (without septic shock) having a score of 2 (118/159, 74%), whereas the numerical score was more evenly spread across 0 to 10 in the septic shock group (Supplemental Materials).

The median age of the study population was 7 (IQR 1.4–13.7) years. Patients in the sepsis and septic shock groups were significantly older than the nonseptic group. There were significantly higher proportions of non-US patients in the sepsis (16.3%) and septic shock groups (20.9%) compared with the nonseptic group (8.1%). A total of 479 (27.7%) of the patients met the criteria for MIS-C. There were significantly higher proportions of patients with MIS-C in the sepsis group (190/326, 59.2%) and the septic shock group (130/167, 78.8%) as compared with the nonseptic patients (289/1405, 21.6%; P < .001 for both comparisons). A total of 741 (42.8%) of the patients had any medical comorbidity, and 194 (11.2%) of the patients had bacterial coinfection (Table 1).

The overall mortality rate of the study population was 1.4% (25/1731), with significant differences in the 3 groups (nonseptic [10/1405, 0.7%], sepsis [15/326, 4.6%; P < .001], septic shock [9/167, 5.4%; P < .001]). The mortality rate was higher in the international patients, with an overall mortality of 10/167 (5.9%) and mortality rates of 3.5%, 11.3% (P = .075), and 11.4% (P = .088) in the nonseptic, sepsis, and septic shock categories, respectively. The overall mortality for patients with MIS-C was 2.5% (12/479), whereas it was 1.04% (13/1252) for patients without MIS-C. Among the patients admitted to the ICU, the overall mortality was 2.96%, with the highest mortality in the septic shock group at 5.5% (9/164; P = .025; Tables 2 and 3).

A total 809 (46.7%) patients were admitted to the ICU. The proportion was lowest in the nonseptic category (525/1405, 37.4%). However, almost all patients in the septic shock category were admitted to the ICU (164/167, 98.2%; P < .001). Invasive and noninvasive ventilation was required in 9.2% and 7.6% of the total cohort, again, with significant differences between nonseptic and sepsis/septic shock categories, with the highest proportion in the septic shock group (37.7% and 19.8%, respectively). The unadjusted median HLOS of the total cohort was 3.7 (IQR 1.9–7) days, with higher HLOS in the sepsis and septic shock groups. On mixed linear regression on log-transformed HLOS, with a random intercept of site (unadjusted “base” model), the exponentiated parameter estimate of HLOS for the septic group vs nonseptic group was 2.32 (95% CI, 2.06–2.61; P < .001; (Table 2) and Supplemental Materials). The oxygen support durations by ventilator, noninvasive ventilator, and high-flow nasal cannula were 5.0 (IQR, 2.1–9.7), 2.2 (IQR, 1.0–4.3), and 2.2 (IQR, 0.9–3.6) days in the total cohort. There was no significant difference in the respiratory support durations between the sepsis and nonseptic group. However, the septic shock group had significantly lower noninvasive ventilator duration and high-flow nasal cannula duration compared with the nonseptic group (Table 2).

After adjusting for the potential confounding variables listed previously, the sepsis group (parameter estimate, 1.73; 95% CI, 1.54–1.95; P < .001) was associated with significantly longer HLOS than the nonseptic group. The adjusted geometric mean HLOS was 5.98 days (95% CI, 5.02–7.13) for the nonseptic group and 10.38 days (95% CI, 8.59–12.53) for the sepsis group (Supplemental Materials). Separate analysis for patients with and without MIS-C also showed higher adjusted HLOS for patients with sepsis compared with patients who were nonseptic (Supplemental Materials)

One unit increase in the numeric PSS was associated with a 70% increase in the risk of mortality (odds ratio, 1.70; 95% CI, 1.46–1.99; P < .001). There was a significant difference in the odds of mortality between the sepsis vs nonseptic group and septic shock vs nonseptic group. However, there was no significant difference in odds of mortality between the septic shock and sepsis groups (Table 4 and Supplemental Materials). The sample area under the receiver operating characteristic curve, or AUROC, was 0.90, and the area under the precision-recall curve, or AUPRC, was 0.27. The LOOCV AUROC was 0.80, and the LOOCV AUPRC was 0.13 (Supplemental Materials). The threshold of 2 or more for detection of mortality had a sensitivity of 0.63, specificity of 0.82, and positive predictive value of 0.05. Separate analysis for patients with MIS-C showed a 50% increase in the risk of mortality (odds ratio, 1.51; 95% CI, 1.22–1.87; P < .001) with each unit increase in PSS; however, the increase in odds of mortality among the 3 groups did not reach statistical significance (Supplemental Materials). Patients without MIS-C had a higher odds ratio of mortality for each unit increase of PSS (odds ratio, 1.98; 95% CI, 1.52–2.58; P < .001) and also significantly higher odds of mortality between sepsis and septic shock vs nonseptic patients (Supplemental Materials).

In this manuscript, we describe the application of the PSC in patients with COVID-19. We noted that a relatively small proportion of hospitalized patients in the participating institutions with COVID-19 and MIS-C meet the criteria for sepsis by the PSC. The PSS had a good AUC to predict mortality, and patients meeting the criteria for sepsis by PSC had higher mortality and longer HLOS. Our report is the first to demonstrate the organ dysfunction–based approach to classify pediatric sepsis in pediatric patients with COVID-19 and can be used to quantify the disease burden in this unique patient population.

The relatively high AUROC and lower AUPRC in the PSS model (of children with COVID-19) suggest that the PSS has a good ability to distinguish between children hospitalized with COVID-19 who have positive vs adverse outcomes, but higher scores have a limited ability to correctly identify positive cases (mortality), resulting in a high false positive rate. However, AUPRC is generally best interpreted using the baseline prevalence as a reference or what the expected performance of a random model would be, which in this case was a mortality rate of 1.4% (0.014). Compared with the baseline mortality rate, the validation dataset (LOOCV) AUPRC of 0.13 would represent a nearly 10-fold higher performance than a random model. The LOOCV AUROC (0.80) was within the AUROC range observed in patients without COVID-19 (0.71–0.92). However, the LOOCV AUPRC of 0.13 in patients with COVID-19 was somewhat lower than the range found in patients with non–COVID-19 sepsis.⁵ The sensitivity and positive predictive value of mortality by PSS criteria in patients with COVID-19 (60% and 5%) are also comparable to the sensitivity and positive predictive value among high-resource sites in the development set of PSS criteria (69.2% and 5.3%). This study’s positive predictive value/precision of 5% is similar to the mortality rate (6%–7%) associated with the threshold chosen to define sepsis by the PSC.⁵ The positive predictive value reflects the overall low prevalence of mortality in both studies. Overall, this reflects the moderate validity of PSS criteria in children admitted with COVID-19.

Our results suggest that patients meeting the criteria for sepsis and septic shock have higher mortality, which reflects on the validity of this scoring system. However, we did not observe a significant difference between sepsis and septic shock. While this seemingly conflicts with Sanchez-Pinto et al,⁵ we wish to caution our interpretation because of the low sample size. Frequentist statistical analysis such as ours is highly dependent on sample size. There are a few patients with COVID-19 whose illness severity is not completely captured by the PSS criteria. This may be related to missing variables required to calculate the PSS (which may incorrectly categorize the patient as nonseptic) or the presence of other organ system failures not captured by the 4-organ system score. This suggests that the 4-organ system PSS may not capture the full spectrum of illness due to COVID-19, and additional organ system failures may need to be included. Since the PSS is only applied to capture the organ dysfunction within the first 24 hours of admission, it is possible that some patients deteriorated afterward, which contributed to mortality in these patients not meeting the PSC upon admission. Further studies may evaluate if an expanded organ system model performs better for patients with COVID-19 than the standard 4-organ system model currently described. Our results are similar to a study on the application of Sepsis-3 criteria in adult patients with COVID-19, where Sepsis-3 criteria of septic shock excluded almost 1/3 of patients with a similar high risk of poor outcome and mortality.²⁶ 

In our analysis, we have included all presentations of SARS-CoV-2–related illnesses. In particular, MIS-C, which is considered a postinfectious immune response,¹³ is included with acute COVID-19. We believe the inclusion of patients with MIS-C is justified, as its clinical presentation overlaps with sepsis,¹⁰ and it is often not possible to differentiate within the first 24 hours (the validity period of PSS). Thus, an epidemiological classification into sepsis and septic shock can provide a uniform language for reporting and disease burden assessment from a public policy standpoint. However, we acknowledge the differences between acute COVID-19 and MIS-C and have also separately analyzed the 2 cohorts, which shows the application of PSC in both presentations. In our data, a very large proportion of patients with septic shock (78.8%) had a diagnosis of MIS-C; however, patients with MIS-C had slightly lower mortality (4.6%) compared with the total cohort of patients with septic shock (5.4%). The odds ratio of mortality of patients with septic shock in the non–MIS-C cohort was also higher than that of septic shock in the MIS-C cohort. Our study, however, was not designed to detect this difference, and differences observed may be related to unadjusted confounding and sample size bias. This observation needs to be explored in specifically designed further studies.

It is also critical to discuss the limitations due to the design choices in our study. First, inclusion criteria for PSS include patients with any antimicrobial testing performed and antimicrobials prescribed in the first 24 hours.⁵ All patients suspected of COVID-19 would have antimicrobial testing; however, only a small proportion would have antivirals prescribed. Second, the PSS is a cumulative score of each time point of the 4-organ systems. The highest PSS over the first 24 hours is selected as the patient score. In the VIRUS registry, however, such granularity of data was not available, and we had to select the worst score for each organ system over the day. This implies that we may be overcounting the PSS, which may reflect that the actual prevalence of sepsis by PSS criteria in patients with COVID-19 may be lower. Third, the PSS is calculated for the first 24 hours of hospitalization. As described in the methods section, because of the limitation of the registry data, our timeframe may extend from a minimum of 24 hours to as long as 48 hours. Fourth, missing data are considered as not done in PSS, as it is based on the hypothesis that if the test was not done, it was clinically expected to be normal. While this rule applies to the VIRUS registry also, a large majority of data in the VIRUS registry is manually entered by clinicians and data coordinators voluntarily. The possibility of data availability but either not entered or entered incorrectly cannot be excluded. Since missing data are considered normal in PSS, a high degree of missingness may have led to undercounting of sepsis in our cohort.

In summary, the PSC was able to identify most of the relatively small cohort of children with COVID-19 who developed sepsis and septic shock. Our study provides external validation (transportability) of the PSS on an untested population with a different case mix. However, our dataset’s relatively small number of events (mortality) may lead to variability in predictive performance and reduce generalizability.²⁷ To increase the discriminant ability of the PSC in patients with COVID-19, further PSS revisions may include additional organ system dysfunction or a longer assessment interval. Additional analysis with a larger number of mortalities should be conducted to draw more definitive conclusions.

We want to thank Ms Mary Reidy, Ms Lynn Retford, and Ms Colleen McNamara for their contributions to the Discovery VIRUS: COVID-19 registry activities.

AUPRC

area under the precision-recall curve

AUROC

area under the receiver operating characteristic curve

CDC

Centers for Disease Control and Prevention

MIS-C

multisystem inflammatory syndrome in children

PSS

Phoenix Sepsis Score

SCCM

Society of Critical Care Medicine

VIRUS

Viral Infection and Respiratory Illness Universal Study