A Prediction Model to Identify Febrile Infants ≤60 Days at Low Risk of Invasive Bacterial Infection

We conducted a case-control study of non–ill-appearing infants ≤60 days of age with a documented rectal temperature ≥38.0°C evaluated in the emergency departments (EDs) of 11 children’s hospitals between July 1, 2011 and June 30, 2016. Infants with IBI (case patients) were matched by hospital and date of visit to 2 febrile infants without IBI (control patients). The study protocol was approved by each hospital’s institutional review board.

Infants ≤60 days of age with IBI were identified through query of each hospital’s microbiology laboratory database or electronic medical record system for blood (bacteremia) and/or CSF (bacterial meningitis) cultures with growth of a pathogen, as described previously.^8,9 Infants were excluded if the culture was documented to have been treated as a contaminant by the medical team.^8,16 

Infants with IBI were eligible for inclusion if they presented to a participating hospital’s ED either from home or from an outpatient clinic (ie, were not transferred from another ED) and if the infant was (1) febrile, defined as a rectal temperature ≥38.0°C (100.4°F) at home, in an outpatient clinic, or in the ED^1,5,11; (2) not ill appearing, as documented on the ED physical examination¹⁷; and (3) without a complex chronic condition.^18,19 Ill-appearing infants and those with complex chronic conditions were excluded because the prevalence of IBI may be higher in these infants,² and these infants would not be classified as low risk.

Potentially eligible control patients were identified through query of the Pediatric Health Information System for infants ≤60 days of age with an ED visit during the 5-year study period and who had both urine and blood cultures obtained.^9,15 For 1 participating site that did not contribute data to the Pediatric Health Information System, potential control patients were identified through query of the hospital’s electronic medical record by using the same search criteria.

Each case patient was matched to 2 control infants at the same hospital with the closest date of visit to the case patient. If >2 infants were eligible as control patients on the basis of date of visit, a random number generator was used to select which control patients to include. Medical records were then reviewed for each potential control patient to assess for eligibility. Control patients were eligible if they met the same inclusion criteria as case patients with the exception that (1) their blood and, if obtained, CSF culture had no growth of a pathogen and (2) they had not received antibiotics within 7 days before the ED visit. Control patients were not excluded on the basis of peripartum maternal antibiotic exposure. Febrile infants with UTIs^8,12 without concomitant IBI were eligible for inclusion as control patients. Encounters subsequent to the index ED visit were reviewed to ensure that the control patient was not ultimately diagnosed with an IBI. If a potentially eligible control patient was deemed ineligible after medical record review, the infant with the next-closest date of visit was selected, and the process was repeated until 2 eligible control patients were identified for each case patient.

For each case patient and control patient, we extracted data from the medical record on variables that we hypothesized to be associated with the presence of IBI.^4,–6,14,20 These variables included the following: demographics (age and sex); past medical history (gestational age); historical features (household sick contacts, symptoms of upper respiratory infection [URI], and duration of fever); measured temperature (at home or in an outpatient clinic, in triage, and highest recorded in the ED); physical examination findings (triage heart rate and respiratory rate and signs of URI and/or bronchiolitis); laboratory data (WBC count, ANC, CRP, procalcitonin, and urinalysis results¹²); and bacterial culture results (urine, blood, and CSF). CSF parameters were not assessed as predictors. Data were entered into a secure Research Electronic Data Capture tool²¹ that was pilot tested at each site and on which investigators were trained before data collection.

A risk-prediction model for IBI was constructed by using a weighted multiple logistic regression approach with weights determined by the 2% prevalence of IBI in the population.^7,22 Each predictor of interest was first considered in the model by itself. For continuous predictors, we used a graphical aid to assess whether the relationship was linear with the logit (log_e[Pr(IBI = 1)/(1−Pr(IBI = 1))]) and followed up with the Box-Tidwell test.²³ An α level of 0.10 was used to select candidate predictors for the model. Before building an adjusted model, we used Spearman rank correlation (|ρ| ≥0.90) to assess for potential multicollinearity among continuous predictors and compared the distributions of continuous predictors by categorical predictors using the Wilcoxon rank test. All possible 2-way interactions were considered in the final multiple logistic regression model. Adjusted odds ratios (aORs) with 95% confidence intervals (CIs) were used to quantify the magnitude of the association between each predictor and the primary outcome (IBI). Analyses were adjusted for study site by using this variable as a fixed effect in the model. An α level of 0.05 was required for the retention of a variable in the final adjusted model. For continuous predictors, we determined the cutoff values that maximized the area under the receiver operating characteristic curve (AUC) and in turn provided the optimal combination of sensitivity and specificity. The cutoff values were the same regardless of the prevalence of IBI used in the weighted model (eg, 2% vs 3% prevalence).

The performance of our prediction model was assessed by using calibration and discrimination metrics.²⁴ The former is evaluated by using a qualitative aid, a calibration curve, to assess how well the model predicts the absolute risk relative to the observed probability of an outcome.^25,26 The latter is a measure of how well the model discriminates between infants with and without IBI; an AUC was used to quantify this measure.²⁷ Given that predictions from a model for a sample of observations used to create the model itself will result in the underestimation of prediction error, we used k-fold cross-validation for internal validation.^28,29 We randomly divided the data into 10 folds (sets) to refit the model on any given 9 folds by leaving 1 fold out and obtaining predictions for the held-out fold. This internal validation approach provides a new estimate of the precision interval around the AUC, which is corrected for the estimated prediction error.

By using the predictors identified in the final model, an IBI score was developed with points assigned on the basis of the strength of the association of the predictor and the outcome (ie, presence of IBI) as expressed by the aOR for each predictor. The predictor with the aOR of the smallest magnitude was assigned 1 point, and the other predictors were assigned points on the basis of the relative magnitude of their aORs.³⁰ Infants with missing data for any of the components of the score were excluded from the main analysis, but the IBI score was applied to case patients with missing data in a sensitivity analysis.

Statistical analyses were implemented by using Stata Data Analysis and Statistical Software version 15.0 (Stata Corp, College Station, TX) and SAS 9.4 (SAS Institute, Inc, Cary, NC).

Of the 394 infants with IBI who presented to the 11 participating EDs during the 5-year study period, 181 (45.9%) met inclusion criteria as case patients and were matched to 362 control patients (Fig 1). Among the 181 infants with IBI, 155 (85.6%) had bacteremia without meningitis, and 26 (14.4%) had bacterial meningitis (19 [73.1%] with concomitant bacteremia). The pathogens are listed in Supplemental Table 5. Seventy-seven infants with IBI (42.5%) had a concomitant UTI compared with 21 of 362 control patients (5.8%). Twenty-three case patients (12.7%) and 138 control patients (38.1%) had a history of a rectal temperature ≥38.0°C but did not have a fever recorded in the ED.

Compared with controls, case patients with IBI were younger and had higher triage heart rates, maximum temperatures recorded in the ED, median WBC counts, and median ANCs. A higher proportion of infants with IBI had an abnormal urinalysis result, although a lower proportion had household sick contacts or symptoms or signs of URI (Table 1).

Four predictors were associated with the presence of IBI: age, highest temperature recorded in the ED, abnormal urinalysis result, and ANC (Table 2). To identify the cutoff value for ANC, the ANC level was incrementally varied to evaluate the effect on the discriminatory ability of the model; an ANC value of 5185 cells per μL was determined to be optimal. The model fit the observed data well, as shown by the calibration plot, in which the calibration curve overlapped with the diagonal reference line of perfect calibration (Fig 2). The AUC was 0.83 (95% CI: 0.79–0.86) for the weighted model. On 10-fold cross-validation, the AUC was 0.83 (95% CI: 0.78–0.87).

Using the 4 predictors identified in the final model, an IBI score was developed with a range of possible scores of 0 to 10 points (Table 3). Among the 492 infants (90.6% of the sample) for whom data were available for each of the predictors (169 case patients and 323 control patients), the sensitivity and specificity of a score ≥2 were 98.8% (95% CI: 95.8%–99.9%) and 31.3% (95% CI: 26.3%–36.6%), respectively (Table 4). The 2 case patients with fever by history only and a low-risk score of <2 were a 3-day-old and a 40-day-old, both of whom had bacteremia due to Enterococcus spp. Moderate-risk IBI scores of ≥3 and ≥4 had higher specificity (52.0% and 57.3%, respectively) but lower sensitivity (92.9% and 88.2%, respectively). Of 12 infants with IBI who had missing data, 11 (91.7%) had scores ≥2, and one 27-day-old infant with group B streptococcal bacteremia and a missing urinalysis result had a score of 0.

Eighty-one of 169 case patients (47.9%) and 282 of 323 control patients (87.3%) with available data for all the predictors had a normal urinalysis result. Among infants with a normal urinalysis result, the sensitivity of an IBI score ≥2 was 97.5% (95% CI: 91.4%–99.7%), and the specificity was 35.8% (95% CI: 30.2%–41.7%). Among 129 infants with abnormal urinalysis results and available data, the sensitivity of a score ≥4 (ie, abnormal urinalysis result with any additional predictor) was 97.7% (95% CI: 92.0%–99.7%), but the specificity was 9.8% (95% CI: 2.7%–23.1%).

None of the 26 infants with bacterial meningitis had a low-risk IBI score <2. Two infants with meningitis had a moderate-risk score of 2, and none had a score of 3. The infants with scores of 2 both had group B streptococcal bacteremia and meningitis. One infant was a 36-day-old born at 31 weeks’ gestational age with 14% bands on complete blood count, and the other was a 54-day-old with 25% bands. Eight infants with meningitis had abnormal urinalysis results, and all had scores ≥5.

In this multicenter study, we derived a highly sensitive clinical score to identify non–ill-appearing febrile infants ≤60 days old at low and moderate probability of IBI. Infants with fever by history only, a normal urinalysis result, and ANC <5185 cells per μL had a low probability of IBI. Because the IBI score does not incorporate results of CSF testing, this score could help clinicians avoid lumbar punctures for these infants.

Commonly cited low-risk criteria (eg, Boston, Philadelphia, and Rochester) risk stratify febrile infants using a combination of demographic, historical, physical examination, and laboratory factors determined by consensus.^1,4,–6 The Step-by-Step approach includes predictors with commonly used cutoff values associated with IBI.^31,32 The IBI score was derived statistically to find optimal cutpoints for each continuous predictor from a sample that included by far the largest number of non–ill-appearing infants with IBI to date.^1,14 Because an IBI score ≥2 had a sensitivity of 98.8%, with a narrow 95% CI of 95.8% to 99.9%, the highly sensitive score has improved precision for the identification of non–ill-appearing febrile infants at low probability of IBI compared with other risk-stratification algorithms that also do not include CSF testing (Rochester, modified Philadelphia, Step-by-Step).^9,14 

Previous investigations have reported a similar prevalence of IBI among infants febrile in the ED when compared with those with only a history of fever.^33,34 However, we observed that infants ≤60 days old with a history of fever, but no recorded fever in the ED, had a low probability of IBI if they additionally had a normal urinalysis result and an ANC <5185 cells per μL. The IBI score is the first algorithm to separately risk stratify infants with fever by history only and can be used to potentially avoid lumbar punctures and hospitalization for these low-risk infants, who are often managed similarly to infants with a measured fever in the ED.^1,9,14 Specifically, with an IBI prevalence (or pretest probability) of 2% and a negative likelihood ratio of 0.04 for a score ≥2, infants with a score <2 would have a posttest probability of IBI of 0.08%. Therefore, among 1000 non–ill-appearing febrile infants, only 1 infant will have a score <2 and harbor an IBI, and lumbar punctures and hospitalization could be avoided in the ∼300 infants without IBI who have scores <2. Additionally, although age <21 days was a predictor of IBI, this age cutoff identified only 1 additional infant with IBI. Because neonates ≤28 days old evaluated in the ED routinely undergo CSF testing and admission,^15,35 using an IBI score <2 to define low risk could have a significant impact by reducing lumbar punctures and hospitalization in neonates with a low probability of IBI.

Although the sensitivity of an IBI score ≥2 was 98.8%, the specificity (31.3%) is lower than the Rochester criteria (44.5%) and Step-by-Step approach (46.9%)¹⁴ although similar to that of the modified Philadelphia criteria (34.5%).⁹ Using scores of ≥3 or ≥4 increased the specificity to >50%. Although the sensitivities of these score cutoffs were reduced to 93% and 88%, respectively, they are similar to those of other risk-stratification algorithms that do not include CSF testing.^9,14 Two infants with a score of 2 had meningitis, although both infants had substantial bandemia, and 1 was also premature. In assessing the need for lumbar puncture in moderate-risk infants, clinicians should weigh the full clinical picture and consider the use of shared decision-making with parents. Additionally, all 8 infants with abnormal urinalysis results and meningitis had scores ≥5. Overall, because of its low specificity and positive likelihood ratio, and because the presence of fever in the ED would achieve a score of 2, the IBI score alone should not be used to determine the need for lumbar puncture in infants with scores ≥2. Rather, a score <2 can be used to identify infants with a history of fever only, who have a low probability of having an IBI, and who may not require CSF testing.

Our study has several limitations. First, although we likely identified all infants with IBI using our search strategy for case patients, our case-control design prohibited the derivation of the IBI score among the entire population of febrile infants without IBI. Second, although we applied weights to the multiple logistic regression analysis to account for the 2% prevalence of IBI among all febrile infants in the population,⁷ our IBI score should be externally validated before being used in clinical practice. Third, clinical variables such as clinical appearance may not be accurately assessed by using medical records. Fourth, only 26 infants had bacterial meningitis. Nevertheless, our study included a larger number of non–ill-appearing infants with meningitis than previous investigations that derived or validated a prediction model for febrile infants.^1,4,–6,14 Fifth, because the prevalence of IBI is higher among ill-appearing infants² and there is consensus on the management of these infants, we excluded 98 febrile infants who were ill appearing. It is likely that ill appearance would be a significant predictor of IBI, and therefore, our results apply only to non–ill-appearing infants. Sixth, although ∼7% of case patients did not have data available for all predictors, all but 1 of these case patients still had IBI scores ≥2. Lastly, CRP and procalcitonin were obtained in only 22 and 4 infants in our study, respectively, and we were unable to evaluate these laboratory parameters as predictors of IBI. However, this low use of CRP and procalcitonin reflects their lack of availability or use at some hospitals.

A clinical score that includes age, highest temperature recorded in the ED, urinalysis, and ANC can be used by clinicians to identify infants with fever by history only who have a low probability of bacteremia and/or bacterial meningitis. A prospective investigation is needed to externally validate this IBI score.

We acknowledge the following collaborators in the Febrile Young Infant Research Collaborative who collected data and contributed as group authors for this study: Elana A. Feldman, MD (Lucile Packard Children’s Hospital Stanford, Palo Alto, CA); Catherine E. Lumb, BS (University of Alabama at Birmingham School of Medicine, Birmingham, AL); Christine E. Mitchell, BSN (Children’s Hospital of Philadelphia, Philadelphia, PA); and Sarah J. Shin, BSN (Children’s Hospital of Philadelphia, Philadelphia, PA).

ANC

absolute neutrophil count

AUC

area under the receiver operating characteristic curve

CI

confidence interval

CRP

C-reactive protein

CSF

cerebrospinal fluid

ED

emergency department

IBI

invasive bacterial infection

URI

upper respiratory infection

UTI

urinary tract infection

WBC

white blood cell