Video Abstract

Video Abstract

Close modal
BACKGROUND AND OBJECTIVES:

The identification of life-threatening infection in febrile children presenting to the emergency department (ED) remains difficult. The quick Sequential Organ Failure Assessment (qSOFA) was only derived for adult populations, implying an urgent need for pediatric scores. We developed and validated a novel, adapted qSOFA score (Liverpool quick Sequential Organ Failure Assessment [LqSOFA]) and compared its performance with qSOFA, Pediatric Early Warning Score (PEWS), and National Institute for Health and Care Excellence (NICE) high-risk criteria in predicting critical care (CC) admission in febrile children presenting to the ED.

METHODS:

The LqSOFA (range, 0–4) incorporates age-adjusted heart rate, respiratory rate, capillary refill, and consciousness level on the Alert, Voice, Pain, Unresponsive scale. The primary outcome was CC admission within 48 hours of ED presentation, and the secondary outcome was sepsis-related mortality. LqSOFA, qSOFA, PEWS, and NICE high-risk criteria scores were calculated, and performance characteristics, including area under the receiver operating characteristic curve, were calculated for each score.

RESULTS:

In the initial (n = 1121) cohort, 47 CC admissions (4.2%) occurred, and in the validation (n = 12 241) cohort, 135 CC admissions (1.1%) occurred, and there were 5 sepsis-related deaths. In the validation cohort, LqSOFA predicted CC admission with an area under the receiver operating characteristic curve of 0.81 (95% confidence interval [CI], 0.76 to 0.86), versus qSOFA (0.66; 95% CI, 0.60 to 0.71), PEWS (0.93; 95% CI, 0.90 to 0.95), and NICE high-risk criteria (0.81; 95% CI, 0.78 to 0.85). For predicting CC admission, the LqSOFA outperformed the qSOFA, with a net reclassification index of 10.4% (95% CI, 1.0% to 19.9%).

CONCLUSIONS:

In this large study, we demonstrate improved performance of the LqSOFA over qSOFA in identifying febrile children at risk for CC admission and sepsis-related mortality. Further validation is required in other settings.

What’s Known on This Subject:

The quick Sequential Organ Failure Assessment has been shown to more accurately predict mortality or ICU transfer than systemic inflammatory response syndrome or the quick Pediatric Logistic Organ Dysfunction-2 in an emergency department population, but with only moderate prognostic accuracy.

What This Study Adds:

In this retrospective study of >12 000 febrile children, the Liverpool quick Sequential Organ Failure Assessment outperforms the quick Sequential Organ Failure Assessment in predicting critical care admission. Liverpool quick Sequential Organ Failure Assessment is a rapid bedside tool that should undergo implementation testing.

Acute febrile illness is one of the most common reasons for presentation to the pediatric emergency department (ED).1  Within this heterogeneous group of patients, only a small minority will have sepsis. However, for these children, mortality rates remain high,2  and delays in recognition and treatment of sepsis lead to an increased risk of mortality.3  To date, a sensitive and specific tool to identify patients with sepsis in the ED remains elusive. In adults, the quick Sequential Organ Failure Assessment (qSOFA) score is proposed as a screening tool for clinicians outside the ICU to promptly identify patients with infection who are more likely to have poorer outcomes.4,5  The presence of ≥2 of the 3 qSOFA components, altered mentation, raised respiratory rate (RR), and low systolic blood pressure (BP), was associated with an increased risk of mortality, but the derivation and validation of Sepsis-3 and the qSOFA did not involve pediatric data.

The use of the qSOFA to identify pediatric sepsis has many limitations. First, researchers in several studies in both adult and pediatric populations have reported poor sensitivity, ranging from 37% to 70% in predicting in-hospital mortality68  and only 36% to 54% in predicting ICU admission.710  Second, researchers in many qSOFA studies have used mortality as their primary outcome,4,68  but mortality in pediatric ED settings is <1%, therefore critical care (CC) admission is a more suitable outcome because this allows assessment of whether the score can identify those patients requiring additional support, regardless of survival.10  Third, hypotension is a late sign of pediatric septic shock.3  Furthermore, BP is infrequently measured in the pediatric ED.11 

Schlapbach et al6  compared the prognostic performance of an age-adjusted qSOFA with systemic inflammatory response syndrome (SIRS) criteria in PICU patients with infection and demonstrated that the qSOFA was not substantially superior to SIRS criteria at predicting mortality and prolonged ICU stay. Researchers of a small pediatric ED study comparing the qSOFA, quick Pediatric Logistic Organ Dysfunction-2, and SIRS criteria for their ability to predict mortality or ICU admission found the qSOFA had the greatest predictive validity but achieved only moderate prognostic accuracy.9 

To address these limitations, we amended the age-adjusted qSOFA to create a novel score known as the Liverpool quick Sequential Organ Failure Assessment (LqSOFA). Although different scoring systems vary in relation to the number of items considered, and the cutoffs applied, past literature indicates that respiratory, cardiovascular, and central nervous system alterations are the most consistent features of severe illness in children and adults. Accordingly, the qSOFA in adults is based on RR, systolic BP, and Glasgow Coma Scale (GCS).4  In contrast to adults, hypotension represents a late sign of pediatric septic shock.3  Therefore, a predictive score that incorporates earlier signs of shock, such as tachycardia and prolonged capillary refill time (CRT),3,12  could improve on the sensitivity demonstrated by the qSOFA. Researchers of a study assessing various vital signs for their ability to predict severity of infection in an ED population reported heart rate (HR) to be the most sensitive.13  Researchers in several studies in both adults and children have found a prolonged CRT acts as a marker for sepsis12,13  and is associated with a greater degree of organ dysfunction (as measured by Sequential Organ Failure Assessment score)14  and risk of mortality.15,16  The Alert, Voice, Pain, Unresponsive (AVPU) scale is the standard assessment of neurologic state in pediatric EDs in the United Kingdom. It is quick and simple to measure and has been shown to correlate well with GCS.17  For these reasons, we a priori selected RR, HR, CRT, and AVPU for the LqSOFA. A recent secondary analysis of the Fluid Expansion as Supportive Therapy study demonstrated the relevance of respiratory, cardiovascular, and central nervous system dysfunction in identifying patients at a greater risk of mortality, further supporting this approach.18  The 2005 Pediatric Sepsis Consensus19  thresholds for abnormal HR and RR, used in both SIRS and the age-adjusted qSOFA, were based on values that were likely determined through clinical consensus.20  In contrast, evidence-based sets of reference ranges have since been developed, on ED21  and hospitalized patients,22  which may be more discriminative in the febrile ED population. Therefore, we compared these 2 sets of reference ranges to identify the most suitable for use in the LqSOFA.

Our aim with the current study was to develop and validate a novel, rapid, bedside scoring system for use in febrile children attending the ED to predict CC admission within 48 hours. We compared the performance of our score with the age-adjusted qSOFA, the Pediatric Early Warning Score (PEWS) in routine use at our hospital, and National Institute for Health and Care Excellence (NICE) high-risk criteria.23 

The study was conducted at a single center: Alder Hey Children’s Hospital, Liverpool, United Kingdom. This is one of Europe’s largest pediatric hospitals, managing ∼60 000 ED attendances annually.24  The initial cohort used to amend the qSOFA into the LqSOFA (referred to as amendment cohort) was used in a previously published cohort from a prospective data set of febrile children (<16 years) attending the ED between November 2010 and April 2012.24  Children were eligible if the treating clinician decided to perform blood tests. Of the 1872 patients recruited to this study, 670 were excluded because of a lack of capacity for study recruitment in the ED, 15 declined consent, and 4 were excluded because of a primary immunodeficiency. The remaining cohort of 1183 patients was used to develop the LqSOFA score by using age-adjusted percentile ranges and rapid-assessment adaptations of other components of the qSOFA score (Supplemental Information). For the validation cohort, children were identified retrospectively from the hospital electronic patient record if they presented to the ED between September 1, 2015, and August 31, 2017. Eligible patients were identified by reviewing ED and CC admissions during this period. Fever was defined as a temperature ≥38°C recorded in the ED or a history of fever reported in the previous 3 days. Exclusion criteria were missing observation data, no history of fever, and transfer from another hospital. Missing observation data were defined as missing ≥2 components required to calculate any of the scoring systems. If only 1 component was missing (ie, not measured), this was deemed to be normal. Ethical approval was granted for both cohorts (research ethics committee reference: 10/H1014/53 and 16/LO/1684).

The primary outcome was CC admission within 48 hours of ED attendance. The secondary outcome was sepsis-related mortality. This was defined as CC admission within 48 hours of ED presentation with suspected sepsis, which led directly or indirectly to in-hospital death within 28 days of admission, as determined by a review of the clinical notes.

The variables for LqSOFA were developed by using adapted qSOFA components (Supplemental Information). Two evidence-based sets of reference ranges for abnormal HR and RR (O’Leary et al21  and Bonafide et al22 ) were compared in the amendment cohort, with the highest performing reference range selected for use in the LqSOFA (Supplemental Information). Scores for LqSOFA, qSOFA, PEWS, and NICE high-risk criteria were calculated by using the worst observations recorded in the ED. Each scoring system differs in both the included vital signs and the age-thresholds used; these can be seen in full in the Supplemental Information. PEWS was in routine use clinically and was therefore calculated electronically in real-time, whereas scores for the other 3 systems were calculated retrospectively for study purposes only, by using individually documented vital sign information in the electronic patient record. As with the qSOFA, each component in the LqSOFA could be scored as 0 or 1, making a total possible score of 4 (Table 1). If blood tests were performed, the worst lactate and worst C-reactive protein (CRP) recorded in the ED were collected.

TABLE 1

Scoring Thresholds for Each Component of LqSOFA

CriterionPoints Allocated
1 Point0 Points
CRT ≥3 s <3 s 
AVPU VPU Alert 
HR >99th centile Bonafide et al22  age-specific thresholds ≤99th centile Bonafide et al22  age-specific thresholds 
RR >99th centile Bonafide et al22  age-specific thresholds ≤99th centile Bonafide et al22  age-specific thresholds 
CriterionPoints Allocated
1 Point0 Points
CRT ≥3 s <3 s 
AVPU VPU Alert 
HR >99th centile Bonafide et al22  age-specific thresholds ≤99th centile Bonafide et al22  age-specific thresholds 
RR >99th centile Bonafide et al22  age-specific thresholds ≤99th centile Bonafide et al22  age-specific thresholds 

VPU, Voice, Pain, Unresponsive.

We assessed the prognostic ability of each scoring system using the area under the receiver operating characteristic curve (AUC). Sensitivity, specificity, positive and negative predictive value, positive and negative likelihood ratio, odds ratio, and accuracy were calculated for each score alongside an asymptotic 95% confidence interval (CI). LqSOFA and qSOFA were compared by using net reclassification improvement (NRI).25  Statistical analyses were performed by using SPSS version 25 (IBM SPSS Statistics, IBM Corporation) and R version 3.6.1.26  Graphics were generated by using ggplot227  and eulerr.28 

A total of 62 (5.2%) patients were excluded from the amendment cohort because of missing observation data, leaving 1121 patients for the amendment cohort (Supplemental Information). Of 14 121 patients identified for the validation cohort, 1880 were excluded: 1249 (8.8%) because of missing observation data, 454 (3.2%) because of no history of fever, 96 (0.7%) because of duplicates, 77 (0.5%) because patients were ≥16 years of age, and 4 (0.03%) because of transfer from another hospital (Fig 1). Importantly, 848 (75.6%) and 8354 (68.2%) patients in the amendment and validation cohorts, respectively, did not have any BP measurements recorded while in the ED. Demographic characteristics, clinical outcomes, and scores of the amendment and validation cohorts are summarized in Table 2.

FIGURE 1

Flow diagram summarizing the composition of the validation cohort.

FIGURE 1

Flow diagram summarizing the composition of the validation cohort.

Close modal
TABLE 2

Summary of Demographic Characteristics, Clinical Outcomes, and Scores of the Amendment and Validation Cohorts

CharacteristicsAmendment CohortValidation Cohort
No. patients 1121 12 241 
Age in y, median (IQR) 2.3 (4.7) 2.5 (3.8) 
Male sex, n (%) 621 (55.4) 6527 (53.3) 
Admitted to ward, n (%) 817 (72.9) 1481 (12.1) 
CC admission within 48 h, n (%) 47 (4.2) 135 (1.1) 
Hospital LOS, d, median (IQR) 3 (3) 1.04 (2.73) 
CC LOS, d, median (IQR) 2 (5) 2.23 (4.33) 
In-hospital mortality, n (%) 1 (0.09) 10 (0.08) 
Sepsis-related mortality, n (%) 1 (0.09) 5 (0.04) 
LqSOFA ≥2, n (%) 79 (7.0) 155 (1.3) 
LqSOFA ≥1, n (%) 369 (32.9) 1919 (15.7) 
qSOFA ≥2, n (%) 31 (2.8) 149 (1.2) 
qSOFA ≥1, n (%) 1027 (91.6) 11 579 (94.6) 
PEWS ≥3, n (%) — 1465 (12.0) 
PEWS ≥2, n (%) — 5211 (42.6) 
NICE high-risk criteria ≥2, n (%) 199 (17.8) 1357 (11.1) 
NICE high-risk criteria ≥1, n (%) 737 (65.7) 6787 (55.4) 
CharacteristicsAmendment CohortValidation Cohort
No. patients 1121 12 241 
Age in y, median (IQR) 2.3 (4.7) 2.5 (3.8) 
Male sex, n (%) 621 (55.4) 6527 (53.3) 
Admitted to ward, n (%) 817 (72.9) 1481 (12.1) 
CC admission within 48 h, n (%) 47 (4.2) 135 (1.1) 
Hospital LOS, d, median (IQR) 3 (3) 1.04 (2.73) 
CC LOS, d, median (IQR) 2 (5) 2.23 (4.33) 
In-hospital mortality, n (%) 1 (0.09) 10 (0.08) 
Sepsis-related mortality, n (%) 1 (0.09) 5 (0.04) 
LqSOFA ≥2, n (%) 79 (7.0) 155 (1.3) 
LqSOFA ≥1, n (%) 369 (32.9) 1919 (15.7) 
qSOFA ≥2, n (%) 31 (2.8) 149 (1.2) 
qSOFA ≥1, n (%) 1027 (91.6) 11 579 (94.6) 
PEWS ≥3, n (%) — 1465 (12.0) 
PEWS ≥2, n (%) — 5211 (42.6) 
NICE high-risk criteria ≥2, n (%) 199 (17.8) 1357 (11.1) 
NICE high-risk criteria ≥1, n (%) 737 (65.7) 6787 (55.4) 

PEWS not in use at the hospital when amendment cohort data were collected. IQR, interquartile range; LOS, length of stay; —, not applicable.

By using the 99th centile cutoffs from Bonafide et al,22  the LqSOFA score achieved an AUC of 0.78 (95% CI, 0.71 to 0.85) in predicting CC admission within 48 hours, compared with the qSOFA (AUC, 0.61; 95% CI, 0.52 to 0.70) and NICE high-risk criteria (AUC, 0.73; 95% CI, 0.65 to 0.81). The performance of each score and biomarkers at different cutoffs are shown in Fig 2 and Supplemental Table 10.

FIGURE 2

Receiver operating characteristic curves for each scoring system or blood biomarker in predicting CC admission in the amendment cohort. PEWS data are not available because it was not in use at the hospital when amendment cohort data were collected.

FIGURE 2

Receiver operating characteristic curves for each scoring system or blood biomarker in predicting CC admission in the amendment cohort. PEWS data are not available because it was not in use at the hospital when amendment cohort data were collected.

Close modal

LqSOFA was then applied to the independent validation cohort. A total of 135 children (1.1%) were admitted to CC within 48 hours: 115 to the high-dependency unit (HDU), 28 to the ICU, and 8 to both units. Of those 135 admitted to CC, 116 (85.9%) met Sepsis-3 diagnostic criteria on day 1 of admission, whereas 99 (73.3%) met 2005 Pediatric Sepsis Consensus criteria.19,29  Lactate was measured in 451 (3.7%) patients; of these, 186 (41.2%) had a lactate ≥2 and 27 (6.0%) of ≥4. CRP was measured in 1628 (13.3%) patients. Ten children (0.08%) died, 5 of which (0.04%) were directly attributable to sepsis.

For predicting CC admission in the validation cohort, PEWS revealed the greatest discriminative ability (AUC, 0.93; 95% CI, 0.90 to 0.95), followed by LqSOFA (AUC, 0.81; 95% CI, 0.76 to 0.86) and NICE high-risk criteria (AUC 0.81; 95% CI, 0.78 to 0.85), with qSOFA (AUC, 0.66; 95% CI, 0.61 to 0.71) the least discriminative (Table 3, Fig 3). For predicting sepsis-related mortality, PEWS revealed the greatest discrimination (AUC, 0.96; 95% CI, 0.92 to 1), followed by LqSOFA (AUC, 0.87; 95% CI, 0.65 to 1), qSOFA (AUC, 0.81; 95% CI, 0.55 to 1), and NICE high-risk criteria (0.72; 95% CI, 0.58 to 0.86) (Table 3, Supplemental Fig 6). Performance of the scores at various cutoffs as binarized scores are shown in Tables 3 and 4. When comparing LqSOFA with qSOFA by using a cutoff of ≥2 for both scores, the NRI for those admitted to CC within 48 hours was 10.37% (95% CI, 0.90 to 19.84) and for those not admitted 0.07% (95% CI, −0.15 to 0.29), yielding an overall NRI of 10.44% (95% CI, 0.96 to 19.91). Figure 4 reveals the percentage of patients admitted to CC within 48 hours, by score, for each of the 4 systems; in Supplemental Fig 7, we demonstrate this for sepsis-related mortality. In Supplemental Fig 8, we show the number of CC admissions predicted by each scoring system or combination of systems.

TABLE 3

AUCs for Primary and Secondary Outcomes for LqSOFA, qSOFA, PEWS, NICE High-Risk Criteria, Lactate, and CRP in the Validation Cohort

Predictor and DefinitionPrimary Outcome: CC Admission Within 48 hSecondary Outcome: Sepsis-Related Mortality
AUC (95% CI)AUC (95% CI)
LqSOFA   
 LqSOFA score 0.81 (0.76 to 0.86) 0.87 (0.65 to 1) 
 LqSOFA ≥2 0.69 (0.64 to 0.75) 0.79 (0.53 to 1) 
 LqSOFA ≥1 0.78 (0.74 to 0.83) 0.82 (0.62 to 1) 
qSOFA   
 qSOFA score 0.66 (0.61 to 0.71) 0.81 (0.55 to 1) 
 qSOFA ≥2 0.64 (0.58 to 0.70) 0.79 (0.53 to 1) 
 qSOFA ≥1 0.53 (0.48 to 0.57) 0.53 (0.29 to 0.77) 
PEWS   
 PEWS 0.93 (0.90 to 0.95) 0.96 (0.92 to 1) 
 PEWS ≥3 0.88 (0.84 to 0.91) 0.94 (0.91 to 0.97) 
 PEWS ≥2 0.75 (0.72 to 0.78) 0.79 (0.68 to 0.90) 
NICE high-risk criteria   
 NICE high-risk criteria 0.81 (0.78 to 0.85) 0.72 (0.58 to 0.86) 
 NICE high-risk criteria ≥2 0.75 (0.70 to 0.80) 0.55 (0.28 to 0.81) 
 NICE high-risk criteria ≥1 0.70 (0.66 to 0.73) 0.72 (0.58 to 0.86) 
Lactate   
 Lactate 0.63 (0.56 to 0.69) 0.89 (0.71 to 1) 
 Lactate ≥4 mmol/L 0.58 (0.51 to 0.65) 0.85 (0.59 to 1) 
 Lactate ≥2 mmol/L 0.61 (0.55 to 0.68) 0.66 (0.41 to 0.92) 
CRP 0.55 (0.49 to 0.61) 0.83 (0.68 to 0.98) 
Predictor and DefinitionPrimary Outcome: CC Admission Within 48 hSecondary Outcome: Sepsis-Related Mortality
AUC (95% CI)AUC (95% CI)
LqSOFA   
 LqSOFA score 0.81 (0.76 to 0.86) 0.87 (0.65 to 1) 
 LqSOFA ≥2 0.69 (0.64 to 0.75) 0.79 (0.53 to 1) 
 LqSOFA ≥1 0.78 (0.74 to 0.83) 0.82 (0.62 to 1) 
qSOFA   
 qSOFA score 0.66 (0.61 to 0.71) 0.81 (0.55 to 1) 
 qSOFA ≥2 0.64 (0.58 to 0.70) 0.79 (0.53 to 1) 
 qSOFA ≥1 0.53 (0.48 to 0.57) 0.53 (0.29 to 0.77) 
PEWS   
 PEWS 0.93 (0.90 to 0.95) 0.96 (0.92 to 1) 
 PEWS ≥3 0.88 (0.84 to 0.91) 0.94 (0.91 to 0.97) 
 PEWS ≥2 0.75 (0.72 to 0.78) 0.79 (0.68 to 0.90) 
NICE high-risk criteria   
 NICE high-risk criteria 0.81 (0.78 to 0.85) 0.72 (0.58 to 0.86) 
 NICE high-risk criteria ≥2 0.75 (0.70 to 0.80) 0.55 (0.28 to 0.81) 
 NICE high-risk criteria ≥1 0.70 (0.66 to 0.73) 0.72 (0.58 to 0.86) 
Lactate   
 Lactate 0.63 (0.56 to 0.69) 0.89 (0.71 to 1) 
 Lactate ≥4 mmol/L 0.58 (0.51 to 0.65) 0.85 (0.59 to 1) 
 Lactate ≥2 mmol/L 0.61 (0.55 to 0.68) 0.66 (0.41 to 0.92) 
CRP 0.55 (0.49 to 0.61) 0.83 (0.68 to 0.98) 
FIGURE 3

Receiver operating characteristic curves for each scoring system or blood biomarker in predicting CC admission in the validation cohort.

FIGURE 3

Receiver operating characteristic curves for each scoring system or blood biomarker in predicting CC admission in the validation cohort.

Close modal
TABLE 4

Comparison of Prognostic Performance for LqSOFA, qSOFA, PEWS, NICE High-Risk Criteria, and Lactate in the Validation Cohort

Scoring SystemOdds Ratio (95% CI)Sensitivity, % (95% CI)Specificity, % (95% CI)Positive Predictive Value, % (95% CI)Negative Predictive Value, % (95% CI)Positive Likelihood Ratio (95% CI)Negative Likelihood Ratio (95% CI)Accuracy, % (95% CI)
Primary outcome: CC admission within 48 h         
 LqSOFA ≥2 76.1 (51.2 to 113.1) 39.2 (31.0 to 48.0) 99.2 (99.0 to 99.3) 34.2 (28.1 to 40.9) 99.3 (99.2 to 99.4) 46.6 (35.0 to 62.0) 0.6 (0.5 to 0.7) 98.5 (98.3 to 98.7) 
 LqSOFA ≥1 14.4 (9.9 to 21.0) 71.9 (63.4 to 79.3) 85.0 (84.3 to 85.6) 5.1 (4.5 to 5.6) 99.6 (99.5 to 99.7) 4.8 (4.3 to 5.4) 0.3 (0.3 to 0.4) 84.8 (84.2 to 85.4) 
 qSOFA ≥2 44.3 (29.2 to 67.2) 28.9 (21.4 to 37.3) 99.1 (98.9 to 99.3) 26.2 (20.4 to 32.9) 99.2 (99.1 to 99.3) 31.8 (23.0 to 43.9) 0.7 (0.6 to 0.8) 98.3 (98.1 to 98.5) 
 qSOFA ≥1 15.7 (1.0 to 252.4) 100 (97.3 to 100) 5.5 (5.1 to 5.9) 1.2 (1.2 to 1.2) 100 1.1 (1.1 to 1.1) 6.5 (6.1 to 7.0) 
 PEWS ≥3 51.9 (31.5 to 85.5) 86.7 (79.8 to 91.9) 88.9 (88.3 to 89.4) 8.0 (7.4 to 8.6) 99.8 (99.7 to 99.9) 7.8 (7.1 to 8.5) 0.2 (0.1 to 0.2) 88.8 (88.3 to 89.4) 
 PEWS ≥2 17.3 (9.1 to 32.9) 92.6 (86.8 to 96.4) 58.0 (57.1 to 58.9) 2.4 (2.3 to 2.5) 99.9 (99.7 to 99.9) 2.2 (2.1 to 2.3) 0.1 (0.1 to 0.2) 58.4 (57.5 to 59.2) 
 NICE high-risk criteria ≥2 13.1 (9.3 to 18.7) 60.7 (52.0 to 69.0) 89.5 (88.9 to 90.0) 6.0 (5.3 to 6.9) 99.5 (99.4 to 99.6) 5.8 (5.0 to 6.7) 0.4 (0.4 to 0.5) 89.2 (88.6 to 89.7) 
 NICE high-risk criteria ≥1 13.0 (6.3 to 26.5) 94.1 (88.7 to 97.4) 45.0 (44.1 to 45.9) 1.9 (1.8 to 2.0) 99.9 (99.7 to 99.9) 1.7 (1.6 to 1.8) 0.1 (0.1 to 0.3) 45.5 (44.6 to 46.4) 
 Lactate ≥4 mmol/L 8.2 (3.6 to 18.9) 17.8 (10.9 to 26.7) 97.4 (95.2 to 98.8) 66.7 (48.1 to 81.2) 80.4 (78.9 to 81.8) 6.9 (3.2 to 15.0) 0.8 (0.8 to 0.9) 79.6 (75.6 to 83.2) 
 Lactate ≥2 mmol/L 2.1 (1.3 to 3.3) 55.5 (45.2 to 65.3) 62.9 (57.6 to 67.9) 30.1 (25.7 to 35.0) 83.0 (79.5 to 86.0) 1.5 (1.2 to 1.9) 0.7 (0.6 to 0.9) 61.2 (56.5 to 65.7) 
Secondary outcome: sepsis-related mortality         
 LqSOFA ≥2 119.3 (19.8 to 718.8) 60.0 (14.7 to 94.7) 98.8 (98.6 to 99.0) 1.9 (0.9 to 4.0) 100 48.3 (23.2 to 100.1) 0.4 (0.1 to 1.2) 98.7 (98.5 to 98.9) 
 LqSOFA ≥1 21.6 (2.4 to 193.0) 80.0 (28.4 to 99.5) 84.4 (83.7 to 85.0) 0.2 (0.1 to 0.3) 100 5.1 (3.3 to 7.9) 0.2 (0.0 to 1.4) 84.4 (83.7 to 85.0) 
 qSOFA ≥2 124.2 (20.6 to 748.9) 60.0 (14.7 to 94.7) 98.8 (98.6 to 99.0) 2.0 (1.0 to 4.0) 100 50.3 (24.1 to 104.7) 0.4 (0.1 to 1.2) 98.8 (98.6 to 99.0) 
 qSOFA ≥1 0.6 (0.0 to 11.4) 100 (47.8 to 100) 5.4 (5.0 to 5.8) 0.04 (0.04 to 0.04) 100 1.1 (1.1 to 1.1) 5.5 (5.1 to 5.9) 
 PEWS ≥3 81.2 (4.5 to 1468.7) 100 (47.8 to 100) 88.1 (87.5 to 88.6) 0.3 (0.3 to 0.4) 100 8.4 (8.0 to 8.8) 88.1 (87.5 to 88.6) 
 PEWS ≥2 14.9 (0.8 to 268.7) 100 (47.8 to 100) 57.5 (56.6 to 58.3) 0.1 (0.1 to 0.1) 100 2.4 (2.3 to 2.4) 57.5 (56.6 to 58.4) 
 NICE high-risk criteria ≥2 2.0 (0.2 to 18.0) 20.0 (0.5 to 71.6) 88.9 (88.4 to 89.5) 0.1 (0.0 to 0.4) 100 1.8 (0.3 to 10.4) 0.9 (0.6 to 1.4) 88.9 (88.3 to 89.4) 
 NICE high-risk criteria ≥1 8.8 (0.5 to 160.0) 100 (47.8 to 100) 44.6 (43.7 to 45.5) 0.1 (0.1 to 0.1) 100 1.8 (1.8 to 1.8) 44.6 (43.7 to 45.5) 
 Lactate ≥4 mmol/L 52.9 (5.3 to 527.5) 75.0 (19.4 to 99.4) 94.6 (92.1 to 96.5) 11.1 (5.9 to 19.9) 99.8 (98.7 to 100) 14.0 (7.0 to 27.8) 0.3 (0.1 to 1.4) 94.5 (91.9 to 96.4) 
 Lactate ≥2 mmol/L 4.3 (0.4 to 41.9) 75.0 (19.4 to 99.4) 59.1 (54.3 to 63.7) 1.6 (0.9 to 2.8) 99.6 (98.0 to 99.9) 1.8 (1.0 to 3.3) 0.4 (0.1 to 2.3) 59.2 (54.5 to 63.8) 
Scoring SystemOdds Ratio (95% CI)Sensitivity, % (95% CI)Specificity, % (95% CI)Positive Predictive Value, % (95% CI)Negative Predictive Value, % (95% CI)Positive Likelihood Ratio (95% CI)Negative Likelihood Ratio (95% CI)Accuracy, % (95% CI)
Primary outcome: CC admission within 48 h         
 LqSOFA ≥2 76.1 (51.2 to 113.1) 39.2 (31.0 to 48.0) 99.2 (99.0 to 99.3) 34.2 (28.1 to 40.9) 99.3 (99.2 to 99.4) 46.6 (35.0 to 62.0) 0.6 (0.5 to 0.7) 98.5 (98.3 to 98.7) 
 LqSOFA ≥1 14.4 (9.9 to 21.0) 71.9 (63.4 to 79.3) 85.0 (84.3 to 85.6) 5.1 (4.5 to 5.6) 99.6 (99.5 to 99.7) 4.8 (4.3 to 5.4) 0.3 (0.3 to 0.4) 84.8 (84.2 to 85.4) 
 qSOFA ≥2 44.3 (29.2 to 67.2) 28.9 (21.4 to 37.3) 99.1 (98.9 to 99.3) 26.2 (20.4 to 32.9) 99.2 (99.1 to 99.3) 31.8 (23.0 to 43.9) 0.7 (0.6 to 0.8) 98.3 (98.1 to 98.5) 
 qSOFA ≥1 15.7 (1.0 to 252.4) 100 (97.3 to 100) 5.5 (5.1 to 5.9) 1.2 (1.2 to 1.2) 100 1.1 (1.1 to 1.1) 6.5 (6.1 to 7.0) 
 PEWS ≥3 51.9 (31.5 to 85.5) 86.7 (79.8 to 91.9) 88.9 (88.3 to 89.4) 8.0 (7.4 to 8.6) 99.8 (99.7 to 99.9) 7.8 (7.1 to 8.5) 0.2 (0.1 to 0.2) 88.8 (88.3 to 89.4) 
 PEWS ≥2 17.3 (9.1 to 32.9) 92.6 (86.8 to 96.4) 58.0 (57.1 to 58.9) 2.4 (2.3 to 2.5) 99.9 (99.7 to 99.9) 2.2 (2.1 to 2.3) 0.1 (0.1 to 0.2) 58.4 (57.5 to 59.2) 
 NICE high-risk criteria ≥2 13.1 (9.3 to 18.7) 60.7 (52.0 to 69.0) 89.5 (88.9 to 90.0) 6.0 (5.3 to 6.9) 99.5 (99.4 to 99.6) 5.8 (5.0 to 6.7) 0.4 (0.4 to 0.5) 89.2 (88.6 to 89.7) 
 NICE high-risk criteria ≥1 13.0 (6.3 to 26.5) 94.1 (88.7 to 97.4) 45.0 (44.1 to 45.9) 1.9 (1.8 to 2.0) 99.9 (99.7 to 99.9) 1.7 (1.6 to 1.8) 0.1 (0.1 to 0.3) 45.5 (44.6 to 46.4) 
 Lactate ≥4 mmol/L 8.2 (3.6 to 18.9) 17.8 (10.9 to 26.7) 97.4 (95.2 to 98.8) 66.7 (48.1 to 81.2) 80.4 (78.9 to 81.8) 6.9 (3.2 to 15.0) 0.8 (0.8 to 0.9) 79.6 (75.6 to 83.2) 
 Lactate ≥2 mmol/L 2.1 (1.3 to 3.3) 55.5 (45.2 to 65.3) 62.9 (57.6 to 67.9) 30.1 (25.7 to 35.0) 83.0 (79.5 to 86.0) 1.5 (1.2 to 1.9) 0.7 (0.6 to 0.9) 61.2 (56.5 to 65.7) 
Secondary outcome: sepsis-related mortality         
 LqSOFA ≥2 119.3 (19.8 to 718.8) 60.0 (14.7 to 94.7) 98.8 (98.6 to 99.0) 1.9 (0.9 to 4.0) 100 48.3 (23.2 to 100.1) 0.4 (0.1 to 1.2) 98.7 (98.5 to 98.9) 
 LqSOFA ≥1 21.6 (2.4 to 193.0) 80.0 (28.4 to 99.5) 84.4 (83.7 to 85.0) 0.2 (0.1 to 0.3) 100 5.1 (3.3 to 7.9) 0.2 (0.0 to 1.4) 84.4 (83.7 to 85.0) 
 qSOFA ≥2 124.2 (20.6 to 748.9) 60.0 (14.7 to 94.7) 98.8 (98.6 to 99.0) 2.0 (1.0 to 4.0) 100 50.3 (24.1 to 104.7) 0.4 (0.1 to 1.2) 98.8 (98.6 to 99.0) 
 qSOFA ≥1 0.6 (0.0 to 11.4) 100 (47.8 to 100) 5.4 (5.0 to 5.8) 0.04 (0.04 to 0.04) 100 1.1 (1.1 to 1.1) 5.5 (5.1 to 5.9) 
 PEWS ≥3 81.2 (4.5 to 1468.7) 100 (47.8 to 100) 88.1 (87.5 to 88.6) 0.3 (0.3 to 0.4) 100 8.4 (8.0 to 8.8) 88.1 (87.5 to 88.6) 
 PEWS ≥2 14.9 (0.8 to 268.7) 100 (47.8 to 100) 57.5 (56.6 to 58.3) 0.1 (0.1 to 0.1) 100 2.4 (2.3 to 2.4) 57.5 (56.6 to 58.4) 
 NICE high-risk criteria ≥2 2.0 (0.2 to 18.0) 20.0 (0.5 to 71.6) 88.9 (88.4 to 89.5) 0.1 (0.0 to 0.4) 100 1.8 (0.3 to 10.4) 0.9 (0.6 to 1.4) 88.9 (88.3 to 89.4) 
 NICE high-risk criteria ≥1 8.8 (0.5 to 160.0) 100 (47.8 to 100) 44.6 (43.7 to 45.5) 0.1 (0.1 to 0.1) 100 1.8 (1.8 to 1.8) 44.6 (43.7 to 45.5) 
 Lactate ≥4 mmol/L 52.9 (5.3 to 527.5) 75.0 (19.4 to 99.4) 94.6 (92.1 to 96.5) 11.1 (5.9 to 19.9) 99.8 (98.7 to 100) 14.0 (7.0 to 27.8) 0.3 (0.1 to 1.4) 94.5 (91.9 to 96.4) 
 Lactate ≥2 mmol/L 4.3 (0.4 to 41.9) 75.0 (19.4 to 99.4) 59.1 (54.3 to 63.7) 1.6 (0.9 to 2.8) 99.6 (98.0 to 99.9) 1.8 (1.0 to 3.3) 0.4 (0.1 to 2.3) 59.2 (54.5 to 63.8) 
FIGURE 4

Percentage of validation cohort patients admitted to CC by LqSOFA score, qSOFA score, NICE high-risk criteria, and PEWS, annotated with raw figures above each bar.

FIGURE 4

Percentage of validation cohort patients admitted to CC by LqSOFA score, qSOFA score, NICE high-risk criteria, and PEWS, annotated with raw figures above each bar.

Close modal

For the primary outcome, poor discrimination was seen with both lactate (AUC, 0.63; 95% CI, 0.56 to 0.69) and CRP (AUC, 0.55; 95% CI, 0.49 to 0.61) (Table 3, Fig 3). For the secondary outcome, both lactate (AUC, 0.89; 95% CI, 0.71 to 1) and CRP (AUC, 0.83; 95% CI, 0.68 to 0.98) revealed good predictive ability (Table 3, Supplemental Fig 6). Several standard performance metrics of CRP and lactate, as well as each of the 4 scoring systems, are presented in Tables 3 and 4.

With our data, we show that a simple 4-item score reveals improved discriminant ability compared with the age-adjusted qSOFA and NICE high-risk criteria. Our novel, rapid, bedside score for children, the LqSOFA score, has revealed good discriminative ability in predicting CC admission in febrile ED patients in a large independent validation cohort. In the ED, it is important to identify the “needle in the haystack”: those children requiring further investigation and urgent treatment. The use of a sepsis-specific score in a pediatric ED may be of limited value because of the low prevalence, whereas predicting which children might require CC admission may be more useful.

For predicting CC admission, the LqSOFA demonstrated good prognostic value. With a cutoff of ≥2, the score revealed excellent specificity but was limited by a relatively low sensitivity, consistent with sensitivities reported for the qSOFA in both adults and children.710  Given that the these scores are designed as tools to aid clinician decision-making, rather than simply predict outcome, it has been argued that this rate of false-negatives is unacceptably high.10  However, a cutoff of ≥1 for the LqSOFA produced a more favorable balance of sensitivity and specificity. Overall, the LqSOFA demonstrates superior discriminative ability than that reported with the qSOFA in the pediatric population, both in the current study and compared with previous research.6,9,30  In the current study, the LqSOFA revealed the greatest positive predictive value of all 4 scoring systems, although this was only 34%. The low values for positive predictive value seen across all scoring systems likely reflect the low prevalence of sepsis within the ED population and highlight the difficulty in identifying such cases. There has rightly been a focus in recent years on preventing cases of sepsis being missed, leading to the prioritization of sensitivity over other performance measures. In the current study, NICE high-risk criteria achieved excellent sensitivity, greater than the LqSOFA and PEWS, but at the cost of limited specificity. The potential disadvantages of relying on such scoring systems, in overdiagnosis and overtreatment of sepsis, are significant.31  Therefore, scoring systems with high sensitivity but poor specificity should be avoided in favor of more balanced scoring systems, ideally those that are quick and simple to calculate. The use of CRT and HR as a proxy for cardiovascular dysfunction means the LqSOFA can be calculated rapidly by using routine clinical observations and without any equipment, such as appropriately sized BP cuffs, a significant advantage over other scoring systems. In addition, shock in children may be present before hypotension is measured3 ; therefore, these parameters may detect impending shock earlier. This facilitates its potential for standardized use across many other settings including primary care and resource-poor settings. The large number of age categories in the LqSOFA potentially provides greater accuracy but also increases the complexity of the score in comparison with other systems with fewer age categories (eg, NICE high-risk criteria). However, when used electronically, this greater complexity would not equate to a more time-consuming or complicated tool for the clinician; the number of components in any given scoring system has a far greater impact in this regard. Overall, the primary limitation of the use of the LqSOFA is the lower sensitivity achieved in comparison with PEWS. However, the excellent specificity suggests the LqSOFA may be useful in determining which children are at low risk of CC admission or in situations in which discriminant ability is prioritized over sensitivity.

For predicting both the primary and secondary outcome, the qSOFA performed less well than the LqSOFA and PEWS. Our results are consistent with previous research. Schlapbach et al6  reported an AUC of 0.64 for the qSOFA in predicting mortality in a PICU cohort. A limitation of their study is the use of a PICU population, given that “quick” organ dysfunction scores are designed to alert the clinician to those at risk for sepsis, and most PICU patients will already have some sort of organ dysfunction and/or organ support. In an ED study, Van Nassau et al9  assessed the ability of the qSOFA to predict PICU transfer and/or mortality, reporting an AUC of 0.72 and a sensitivity of 50%, although their study used a relatively small sample (n = 864). With our study, we provide further evidence that the qSOFA lacks the prognostic ability to justify its introduction into routine practice in the pediatric ED.

A further limitation of using the qSOFA in the pediatric population, highlighted by the current study, is the infrequency of BP measurements in the pediatric ED. In the validation cohort, 8354 (68.2%) patients did not have their BP recorded while in the ED; this is consistent with previous research.32  The lack of BP data may have affected the performance of the qSOFA; however, to exclude these patients from the study would mean excluding the vast majority of eligible patients. The results generated from such an analysis would provide little information on the potential “real-world” performance of the score. BP is included in many PEWS,33  however, BP may not be measured at triage; NICE sepsis guidance recommends BP is measured only when tachycardia or prolonged CRT are present.23  The infrequency of BP measurements may lead to missed opportunities for the early detection of sepsis. In the current study, almost one-quarter of those admitted to CC within 48 hours did not have their BP measured while in the ED, suggesting the absence of BP measurement is not confined to “well” children. A score that does not require BP measurement, such as LqSOFA, therefore presents a significant advantage.

PEWS demonstrated the greatest discriminative ability for both the primary and secondary outcomes, consistent with previous research.33  Most PEWS are designed to identify deterioration in hospitalized children and have therefore not been validated in an ED setting. A previous ED study revealed that PEWS was an independent predictor of ICU admission within 48 hours, or death within 30 days, in unselected pediatric ambulance patients presenting to the ED.34  The large number of different heterogeneous PEWS in use is a disadvantage because this leads to inconsistency in detecting and responding to acute illness. qSOFA and especially LqSOFA therefore offer an advantage over PEWS as standardized tools to improve sepsis recognition. Importantly, PEWS was the only scoring system in routine clinical use during the study, and hence PEWS scores were known by treating physicians. This may have influenced decisions regarding transfer to CC, potentially altering the performance of the score in comparison with the other scoring systems, which had not been applied clinically. Furthermore, compared with only 3 and 4 items in the qSOFA and LqSOFA, respectively, PEWS often include several more variables and is therefore more complicated and time-consuming to calculate (Supplemental Fig 5).33  Although a granular score may perform better than a parsimonious score such as ours, it may be less feasible in practice for rapid assessment in a busy ED.

A previous study compared the NICE high-risk criteria in adults, with qSOFA, National Early Warning Score, and SIRS for their ability to predict mortality in an ED and ward setting.35  NICE high-risk criteria achieved a sensitivity of only 58.9% and were not independently associated with an adverse outcome. In the current study, for the primary outcome we found good discriminative ability suggesting NICE high-risk criteria correlate well with risk of CC admission, and the criteria correctly identified all 5 sepsis-related deaths. However, 55% of validation cohort patients were positive for NICE high-8risk criteria, suggesting the criteria lack the specificity to be useful alone as a screening tool for possible sepsis in the ED. Given that the guidelines recommend senior review for children with NICE high-risk criteria features,23  implementation is likely to have huge resource implications in an already stretched health service. Importantly, our analysis did not include many other NICE high-risk criteria features such as cyanosis, nonblanching rash, and “appears ill to health care worker,” suggesting the NICE high-risk criteria could be simplified.

A strong association between lactate and mortality in pediatric sepsis has previously been reported.36  In the current study, lactate revealed good discriminative ability for sepsis-related mortality, providing further support for including lactate measurement in risk stratification in pediatric sepsis.37  CRP revealed poor versus good discrimination for the primary versus secondary outcomes, respectively, which is consistent with several studies in which researchers report CRP as a poorer predictor of sepsis than other available biomarkers.38 

All 4 scoring systems in the current study used different age-specific thresholds. Schlapbach et al6  noted the lack of age-specific cutoffs for the original qSOFA and therefore applied those used in the corrected 2005 Pediatric Sepsis definitions.19,39  These age-specific thresholds are notably lower than those described by Bonafide et al22  and used in the LqSOFA. NICE high-risk criteria and the PEWS in the current study generally use thresholds in between these 2. This lack of consensus in age-specific thresholds may explain a large part of the difference in prognostic ability seen between the scoring systems and highlights the urgent need for standardization of reference ranges. The Bonafide et al22  reference ranges for hospitalized children are closer to the febrile child population than the O’Leary et al21  reference ranges for low acuity children presenting to the ED or the Fleming et al20  reference ranges for healthy children used in our PEWS and are therefore the best ranges to identify children with severe enough illness to require CC admission.

For operationalization of the score in the ED, a highly sensitive score (NICE high-risk criteria ≥1) could be used as a screening test in triage to identify the group at risk for sepsis, followed by a highly specific score (LqSOFA≥2) to identify those children at highest risk of poor outcome. This can be implemented electronically (such as reported by Balamuth et al40 ), with an LqSOFA score ≥2 prompting immediate senior review.

To our knowledge, this is the largest study in which the performance of quick assessment scores, including the qSOFA, was assessed in a pediatric ED population. Our population reflects a real-world high-income ED setting, with a low prevalence of sepsis. We compared different age-specific cutoffs and 2 other prediction scores for sepsis: NICE high-risk criteria and qSOFA. The LqSOFA can be calculated rapidly across a variety of different health care settings, because no equipment is required. Limitations include use of retrospective data, patients from a single center, criteria of CC admission which may depend on local practice, and the low mortality. In addition, hypothermic patients presenting with sepsis may have been missed.

Overall, with the findings of the current study, we demonstrate the superior performance of LqSOFA and question the use of the qSOFA to identify patients at risk for sepsis in the pediatric ED. The LqSOFA requires further evaluation in other settings before recommending more widespread use as entry criterion into randomized controlled trials in the pediatric ED.

We thank members of the research study team who helped recruit patients into the observational, prospective study which formed the amendment cohort. We also thank the children included in the prospective study, and their parents for giving consent for them to participate in the study. We also extend thanks to the nursing and medical staff in the ED at Alder Hey Children’s NHS Foundation Trust for their contribution to the study.

Prof Carrol conceptualized and designed the study, analyzed data, reviewed and revised the manuscript, and oversaw all aspects of the study; Dr Romaine drafted the initial manuscript, cleaned data, analyzed data, and reviewed and revised the manuscript; Drs Khanijau, Wright, and McGalliard, Mr Leigh, and Ms Potter collected data, cleaned data, conducted initial analyses, and reviewed and revised the manuscript; Mr Edwardson and Mr Johnston designed the data collection instruments, conducted data extraction from the electronic patient record, and reviewed and revised the manuscript; Dr Kerr supervised management and flow of patients in the emergency department and reviewed and revised the manuscript; Ms Sefton iteratively developed the Pediatric Early Warning Score used in the study, provided advice on earlier drafts of the manuscript, and reviewed and revised the manuscript; Dr Schlapbach contributed to study design, analyzed data, provided advice on earlier drafts of the manuscript, and critically reviewed and revised the manuscript; Dr Pallmann supervised the data analysis, analyzed data, and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: Dr Romaine was funded by the Academic Foundation Programme, and Dr McGalliard was funded by the National Institutes of Health Research Academic Clinical Fellowship Programme.

AUC

area under the receiver operating characteristic curve

AVPU

Alert, Voice, Pain, Unresponsive

BP

blood pressure

CC

critical care

CI

confidence interval

CRP

C-reactive protein

CRT

capillary refill time

ED

emergency department

GCS

Glasgow Coma Scale

HDU

high-dependency unit

HR

heart rate

LqSOFA

Liverpool quick Sequential Organ Failure Assessment

NICE

National Institute for Health and Care Excellence

NRI

net reclassification improvement

PEWS

Pediatric Early Warning Score

qSOFA

quick Sequential Organ Failure Assessment

RR

respiratory rate

SIRS

systemic inflammatory response syndrome

1
Sands
R
,
Shanmugavadivel
D
,
Stephenson
T
,
Wood
D
.
Medical problems presenting to paediatric emergency departments: 10 years on
.
Emerg Med J
.
2012
;
29
(
5
):
379
382
2
Liu
L
,
Johnson
HL
,
Cousens
S
, et al;
Child Health Epidemiology Reference Group of WHO and UNICEF
.
Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000
.
Lancet
.
2012
;
379
(
9832
):
2151
2161
3
Brierley
J
,
Carcillo
JA
,
Choong
K
, et al
.
Clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock: 2007 update from the American College of Critical Care Medicine
.
Crit Care Med
.
2009
;
37
(
2
):
666
688
4
Seymour
CW
,
Liu
VX
,
Iwashyna
TJ
, et al
.
Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (sepsis-3)
.
JAMA
.
2016
;
315
(
8
):
762
774
5
Singer
M
,
Deutschman
CS
,
Seymour
CW
, et al
.
The third international consensus definitions for sepsis and septic shock (sepsis-3)
.
JAMA
.
2016
;
315
(
8
):
801
810
6
Schlapbach
LJ
,
Straney
L
,
Bellomo
R
,
MacLaren
G
,
Pilcher
D
.
Prognostic accuracy of age-adapted SOFA, SIRS, PELOD-2, and qSOFA for in-hospital mortality among children with suspected infection admitted to the intensive care unit
.
Intensive Care Med
.
2018
;
44
(
2
):
179
188
7
Churpek
MM
,
Snyder
A
,
Han
X
, et al
.
Quick sepsis-related organ failure assessment; systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit
.
Am J Respir Crit Care Med
.
2017
;
195
(
7
):
906
911
8
Goulden
R
,
Hoyle
MC
,
Monis
J
, et al
.
qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis
.
Emerg Med J
.
2018
;
35
(
6
):
345
349
9
van Nassau
SC
,
van Beek
RH
,
Driessen
GJ
,
Hazelzet
JA
,
van Wering
HM
,
Boeddha
NP
.
Translating sepsis-3 criteria in children: prognostic accuracy of age-adjusted quick SOFA score in children visiting the emergency department with suspected bacterial infection
.
Front Pediatr
.
2018
;
6
:
266
10
Moskowitz
A
,
Patel
PV
,
Grossestreuer
AV
, et al;
Center for Resuscitation Science
.
Quick sequential organ failure assessment and systemic inflammatory response syndrome criteria as predictors of critical care intervention among patients with suspected infection
.
Crit Care Med
.
2017
;
45
(
11
):
1813
1819
11
Marlais
M
,
Lyttle
MD
,
Inwald
D
.
Ten concerns about blood pressure measurement and targets in paediatric sepsis
.
Intensive Care Med
.
2017
;
43
(
3
):
433
435
12
Fleming
S
,
Gill
P
,
Jones
C
, et al
.
The diagnostic value of capillary refill time for detecting serious illness in children: a systematic review and meta-analysis
.
PLoS One
.
2015
;
10
(
9
):
e0138155
13
Thompson
M
,
Coad
N
,
Harnden
A
,
Mayon-White
R
,
Perera
R
,
Mant
D
.
How well do vital signs identify children with serious infections in paediatric emergency care?
Arch Dis Child
.
2009
;
94
(
11
):
888
893
14
Lima
A
,
Jansen
TC
,
van Bommel
J
,
Ince
C
,
Bakker
J
.
The prognostic value of the subjective assessment of peripheral perfusion in critically ill patients
.
Crit Care Med
.
2009
;
37
(
3
):
934
938
15
Ait-Oufella
H
,
Bige
N
,
Boelle
PY
, et al
.
Capillary refill time exploration during septic shock
.
Intensive Care Med
.
2014
;
40
(
7
):
958
964
16
Carcillo
JA
,
Kuch
BA
,
Han
YY
, et al
.
Mortality and functional morbidity after use of PALS/APLS by community physicians
.
Pediatrics
.
2009
;
124
(
2
):
500
508
17
Hoffmann
F
,
Schmalhofer
M
,
Lehner
M
,
Zimatschek
S
,
Grote
V
,
Reiter
K
.
Comparison of the AVPU scale and the pediatric GCS in prehospital setting
.
Prehosp Emerg Care
.
2016
;
20
(
4
):
493
498
18
Levin
M
,
Cunnington
AJ
,
Wilson
C
, et al
.
Effects of saline or albumin fluid bolus in resuscitation: evidence from re-analysis of the FEAST trial
.
Lancet Respir Med
.
2019
;
7
(
7
):
581
593
19
Goldstein
B
,
Giroir
B
,
Randolph
A
;
International Consensus Conference on Pediatric Sepsis
.
International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics
.
Pediatr Crit Care Med
.
2005
;
6
(
1
):
2
8
20
Fleming
S
,
Thompson
M
,
Stevens
R
, et al
.
Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: a systematic review of observational studies
.
Lancet
.
2011
;
377
(
9770
):
1011
1018
21
O’Leary
F
,
Hayen
A
,
Lockie
F
,
Peat
J
.
Defining normal ranges and centiles for heart and respiratory rates in infants and children: a cross-sectional study of patients attending an Australian tertiary hospital paediatric emergency department
.
Arch Dis Child
.
2015
;
100
(
8
):
733
737
22
Bonafide
CP
,
Brady
PW
,
Keren
R
,
Conway
PH
,
Marsolo
K
,
Daymont
C
.
Development of heart and respiratory rate percentile curves for hospitalized children
.
Pediatrics
.
2013
;
131
(
4
).
23
National Institute for Health and Care Excellence
.
Sepsis: recognition, diagnosis and early management: © NICE (2017) Sepsis: recognition, diagnosis and early management
.
BJU Int
.
2018
;
121
(
4
):
497
514
24
Irwin
AD
,
Grant
A
,
Williams
R
, et al
.
Predicting risk of serious bacterial infections in febrile children in the emergency department
.
Pediatrics
.
2017
;
140
(
2
):
e201162853
25
Pencina
MJ
,
D’Agostino
RB
 Sr.
,
D’Agostino
RB
 Jr.
,
Vasan
RS
.
Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond
.
Stat Med
.
2008
;
27
(
2
):
157
172–212
26
R Core Team
. R: A Language and Environment for Statistical Computing [computer program]. Version 3.6.1. Vienna, Austria: R Core Team;
2017
27
Wickham
H
.
ggplot2: Elegant Graphics for Data Analysis
, 2nd ed.
New York, NY
:
Springer
;
2016
28
Larsson
J
.
eulerr: area-proportional euler and venn diagrams with ellipses [computer program]. Version 5.1.0. Available at: https://CRAN.R-project.org/package=eulerr. Accessed August 5, 2019
29
Matics
TJ
,
Sanchez-Pinto
LN
.
Adaptation and validation of a pediatric sequential organ failure assessment score and evaluation of the sepsis-3 definitions in critically ill children
.
JAMA Pediatr
.
2017
;
171
(
10
):
e172352
30
Zallocco
F
,
Osimani
P
,
Carloni
I
,
Romagnoli
V
,
Angeloni
S
,
Cazzato
S
.
Assessment of clinical outcome of children with sepsis outside the intensive care unit
.
Eur J Pediatr
.
2018
;
177
(
12
):
1775
1783
31
Klompas
M
,
Calandra
T
,
Singer
M
.
Antibiotics for sepsis - finding the equilibrium
.
JAMA
.
2018
;
320
(
14
):
1433
1434
32
Gilhotra
Y
,
Willis
F
.
Blood pressure measurements on children in the emergency department
.
Emerg Med Australas
.
2006
;
18
(
2
):
148
154
33
Chapman
SM
,
Wray
J
,
Oulton
K
,
Pagel
C
,
Ray
S
,
Peters
MJ
.
‘The Score Matters’: wide variations in predictive performance of 18 paediatric track and trigger systems
.
Arch Dis Child
.
2017
;
102
(
6
):
487
495
34
Corfield
AR
,
Silcock
D
,
Clerihew
L
, et al
.
Paediatric early warning scores are predictors of adverse outcome in the pre-hospital setting: a national cohort study
.
Resuscitation
.
2018
;
133
:
153
159
35
Kopczynska
M
,
Sharif
B
,
Cleaver
S
, et al;
Welsh Digital Data Collection Platform Collaborators
.
Red-flag sepsis and SOFA identifies different patient population at risk of sepsis-related deaths on the general ward
.
Medicine (Baltimore)
.
2018
;
97
(
49
):
e13238
36
Schlapbach
LJ
,
MacLaren
G
,
Festa
M
, et al;
Australian & New Zealand Intensive Care Society (ANZICS) Centre for Outcomes & Resource Evaluation (CORE) and Australian & New Zealand Intensive Care Society (ANZICS) Paediatric Study Group
.
Prediction of pediatric sepsis mortality within 1 h of intensive care admission
.
Intensive Care Med
.
2017
;
43
(
8
):
1085
1096
37
Weiss
SL
,
Peters
MJ
,
Alhazzani
W
, et al
.
Surviving sepsis campaign international guidelines for the management of septic shock and sepsis-associated organ dysfunction in children
.
Intensive Care Med
.
2020
;
46
(
suppl 1
):
10
67
38
Simon
L
,
Gauvin
F
,
Amre
DK
,
Saint-Louis
P
,
Lacroix
J
.
Serum procalcitonin and C-reactive protein levels as markers of bacterial infection: a systematic review and meta-analysis [published correction appears in Clin Infect Dis. 2005;40(9):1386–1388]
.
Clin Infect Dis
.
2004
;
39
(
2
):
206
217
39
Gebara
BM
.
Values for systolic blood pressure
.
Pediatr Crit Care Med
.
2005
;
6
(
4
):
500
501–501
40
Balamuth
F
,
Alpern
ER
,
Abbadessa
MK
, et al
.
Improving recognition of pediatric severe sepsis in the emergency department: contributions of a vital Sign–Based electronic alert and bedside clinician identification
.
Ann Emerg Med
.
2017
;
70
(
6
):
759
768.e2

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data