Performance of Febrile Infant Decision Tools on Hypothermic Infants Evaluated for Infection Free

We performed a planned secondary analysis of the multicenter retrospective cohort study of infants with hypothermia (≤36.0 C) who presented to an ED or inpatient unit of 1 of 9 sites participating in the Hypothermic Young Infant Collaborative between September 1, 2016 and May 5, 2021.⁴  The study protocol was approved by the institutional review board at each participating site.

Infants were identified by using billing codes and vital signs according to the methodology previously described.⁴  Subsequent chart review was performed, and infants were only eligible for inclusion if they were hypothermic on initial vital signs or had historical hypothermia measured by a caregiver or outside health care facility. In this secondary analysis, we only included well-appearing infants ≤90 days old who were evaluated for bacterial infection via laboratory testing (complete blood count [CBC], inflammatory markers [IM], urinalysis, CSF studies, or cultures; Fig 1). Infants were considered well-appearing if they did not meet the criteria for “ill-appearing” which we defined as documentation of 1 of the following terms on physical examination: ill appearance, toxic, limp, unresponsive, gray, cyanotic, apneic, weak cry, poorly perfused, grunting, listless, lethargic, or irritable.^23,24  We excluded infants with a gestational age of <37 weeks and those with chronic medical conditions (defined as a condition expected to last ≥12 months and requiring subspecialty care or involving 1 or more organ systems).^24,25 

Each febrile infant risk stratification tool has its own inclusion and exclusion criteria (Supplemental Fig 2). Infants were included in the analysis for each tool only if they met the criteria for the specific tool. Therefore, a patient may have met the eligibility criteria for the application of one of the decision tools, but not others.

We extracted demographics (age, sex, insurance), medical history (gestational age, previous admission to NICU, previous inpatient hospitalizations, previous antibiotic therapy, immunizations in the preceding 48 hours), clinical data (presenting vital signs, symptoms, physical examination findings, treatment, diagnoses), laboratory results (CBC and differential, urine dip, urine microscopy, C-reactive protein [CRP], procalcitonin, stool studies, CSF cell counts, CSF Gram stain, CSF meningitis/encephalitis panel), imaging (chest radiograph), and microbiology results (blood, urine, and CSF cultures). Data from each medical chart was entered into a shared Research Electronic Data Capture (REDCap) tool.

The outcomes of interest were the ability of each febrile infant decision tool to appropriately stratify infants with SBI (primary outcome) and invasive bacterial infection (IBI; secondary outcome). Diagnostic performance was determined by using sensitivity, specificity, negative predictive value, positive predictive value, negative likelihood ratio, and positive likelihood ratio. SBI was defined as urinary tract infection, bacteremia, or bacterial meningitis; IBI was defined as bacteremia or bacterial meningitis. Similar to established guidelines, we defined UTI as the growth of ≥50 000 colony-forming units per mL of pathogenic bacteria from a urine culture obtained by catheterization or suprapubic aspiration, in association with an abnormal urinalysis (leukocyte esterase, nitrite, or >5 white blood cell count [WBC] per high powered field) or treatment course of antibiotics.²⁶  We defined bacteremia as the growth of a bacterial organism from a blood culture treated with a full treatment course of antibiotics. Bacterial meningitis was defined as the presence of bacteria on CSF culture or polymerase chain reaction testing treated with a full treatment course of antibiotics. Infants without a full sepsis evaluation or those discharged from the ED were still included but considered to not have SBI if it was not diagnosed within 7 days of disposition after review of return visits.

Table 1 outlines the risk predictors for bacterial infection for each febrile infant decision tool applied to our study cohort. Rochester, Boston, Philadelphia, IBI Score (also known as Aronson’s prediction model), AAP CPG, and the AAP CPG without procalcitonin were modified from the original published criteria for application to our patient population. The PECARN rule did not require modification. The original Rochester and Philadelphia criteria require urine microscopy to stratify risk; however, this was not performed on all urine samples in our cohort. Therefore, if urine dip leukocyte esterase testing was performed without microscopy, a positive urine leukocyte esterase result was classified as a positive result to maximize the criteria’s sensitivity.²⁷  At 2 of the study sites, urine microscopy results are reported in categories of, for example, 5 to 10 WBC per high powered field (hpf) and 11 to 15 per hpf. Therefore, we modified both the Boston and Philadelphia criteria to define pyuria as >10 WBC per hpf.²⁷  The IBI Score identifies documented fever on presentation, not reported by history only, as increasing the probability of IBI. This part of the score was modified so that hypothermia documented at a health care facility, instead of by history only, would be considered a higher-risk predictor. The AAP CPG lists a temperature of ≥38.5C as an abnormal IM that can be used to predict IBI. Given that our cohort included hypothermic infants and that no known temperature threshold reliably identifies SBI or IBI in hypothermic infants,²⁸  this IM was removed. For the main analysis of the AAP CPG, we considered all infants 8 to 21 days of age to be higher risk as well as those with procalcitonin >0.5ng/mL, CRP >20mg/L, or ANC >4000/mm³. We also performed analyses of the AAP CPG without procalcitonin given limited accessibility to this IM at some hospitals. This was modeled after a recent study by Burstein et al, which revealed the performance of the AAP CPG in the absence of procalcitonin values in febrile infants.²⁹  For this analysis, we used an ANC threshold of >5200/mm³ to stratify infants as higher risk because this threshold was derived when procalcitonin was not available.¹⁴  Infants with any higher risk predictor identified for a particular tool were not considered low risk for SBI or IBI in that tool’s analysis. Infants were considered low risk for SBI or IBI if the data were available to confirm that each predictor was low risk. Infants with missing data for any of the predictors of a particular tool were excluded from that specific analysis unless they had a higher risk predictor identified.²⁷ 

We calculated the diagnostic performance of 6 febrile infant decision tools in identifying hypothermic infants with IBI and SBI. We reported the sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio of each tool. We also described infants with SBI who were misclassified as low-risk by each tool. All statistical analyses were conducted by using R version 4.4.2.³⁰  Confidence intervals (CIs) for sensitivity, specificity, negative predictive value, and positive predictive value were calculated by using the exact binomial method with the package “epiR.”³¹ 

Of the 10 916 infants initially identified, 314 met the general inclusion criteria (Fig 1). The median age of our cohort was 5 days (interquartile range, 4–7 days), with 237 (75.5%) infants presenting within the first week of life. In total, 214 infants (68.1%) underwent a full sepsis evaluation, and 291 infants (92.6%) received a full or partial sepsis evaluation. Overall, 306 infants (97.4%) were hospitalized. Fourteen cases of SBI (4.5%) and 7 cases of IBI (2.2%) were identified. There was 1 infant death; this patient was diagnosed with dilated cardiomyopathy and died during initial hospitalization. Additional study demographics, clinical characteristics, and outcomes are presented in Table 2.

The number of hypothermic infants included in the analysis for each febrile infant decision tool is shown in Supplemental Fig 2. The Boston and Philadelphia age criteria resulted in the exclusion of the majority of identified infants, leaving <10 infants eligible for analysis. The remaining febrile infant decision tools analyses included between 44 and 292 infants.

The modified Rochester criteria had the lowest sensitivity (29.0%; CI 4.0%–71.0%) but the highest specificity (83.0%; CI 76.0%–88.0%) for the identification of SBI (Table 3). The other tools had higher sensitivity (>80%) but lower specificity (≤60%). The IBI Score and AAP CPG criteria did not misclassify any hypothermic infants with a bacterial infection as low risk. Although the Boston and Philadelphia tools were only applicable to a small cohort given the large number of infants excluded from these tools because of age, they also did not misclassify any infants with a bacterial infection. The modified Rochester criterion misclassified 5 of 7 (71.4%) infants with SBI as low-risk. Of these, 1 patient had a UTI, 3 had bacteremia, and 1 had bacterial meningitis. The PECARN criteria misclassified 1 (1/6; 16.7%) infant with bacteremia as low-risk. Cases of SBI misclassified by the modified Rochester and PECARN criteria are summarized in Table 4. All febrile infant decision tools had similar negative predictive values (≥96.0%). Results from the main analysis of the AAP CPG and the AAP CPG without procalcitonin revealed minimal differences in diagnostic performance, although removing procalcitonin as a predictor only added 1 patient to the cohort for analysis.

The modified Rochester criteria had the lowest sensitivity (20.0%; CI 1.0%–72%) but the highest specificity (82.0%; CI 75.0%–88.0%) for the identification of IBI (Table 5). The other tools had higher sensitivity (≥67%) but lower specificity (≤60%). All febrile infant decision tools had a similar negative predictive value (≥97%). Again, Boston, Philadelphia, IBI Score, AAP CPG, and AAP CPG without procalcitonin did not misclassify infants. The modified Rochester criteria misclassified 3 cases of bacteremia along with 1 case of bacterial meningitis, and the PECARN criteria misclassified 1 case of bacteremia.

To the best of our knowledge, this is the first study to evaluate the performance of febrile infant decision tools to predict bacterial infections in hypothermic infants in a large multicenter cohort of these infants. We found that most febrile infant decision tools we tested had relatively high sensitivities and negative predictive values for detecting SBI and IBI, indicating few missed cases of bacterial infections. However, many infants are misclassified as higher-risk, as highlighted by the low specificity of all tools, potentially resulting in invasive procedures, unnecessary hospitalization, and exposure to antimicrobial therapy. In addition, only a small subset of hypothermic infants in the cohort met the inclusion criteria for these tools, supporting the need to develop decision tools specific to the unique characteristics of hypothermic infants.

The results are overall consistent with those of a previous single-center study,³  in which the authors retrospectively applied the Pittsburgh low-risk febrile infant criteria³²  to hypothermic infants. They reported that 26% (25/95) of patients without SBI were stratified as low-risk. All 9 patients with SBI were appropriately classified as higher-risk. Although the application of the Pittsburgh criteria suggests that hypothermic infants categorized as low-risk could be managed with observation alone, most infants were misclassified as higher-risk. We did not examine the diagnostic accuracy of the Pittsburgh criteria because it requires enhanced urinalysis, which was not routinely performed. Although our results align with this study, we took a novel approach with our study by exploring how more recent febrile infant decision tools performed when applied to hypothermic infants and included infants cared for at multiple sites.

One major driver of misclassification or exclusion of hypothermic infants in the febrile infant decision tools used in our study was patient age. Our study cohort had a median age of 5 days at the time of presentation, with >95% of infants presenting within the first month of life. Half of the febrile infant tools we studied automatically excluded (Boston, Philadelphia) or classified these infants as higher-risk on the basis of age (AAP CPG). In febrile infants, the rate of SBI is inversely proportional to age,¹⁵  but this is less well-studied in hypothermic infants. In 1 study using the Pediatric Health Information System database, rates of SBI in hypothermic infants were similar between age groups; however, the authors explored 30-day groupings (0–30, 31–60, and 61–90 days), which lacks granularity to explore differences in the first weeks of life.⁶  The authors of several other studies have reported an association between older age and SBI.^3–5  Many physical and physiologic characteristics of newborns, especially those with low birth weight or prematurity, may place them at increased risk for hypothermia, including a large surface area in relation to their body weight, decreased amount of fat for insulation and heat production, thin skin, and immature thermogenesis.³³  Therefore, temperature instability can lead to admission in young infants who may otherwise be healthy. In contrast, older infants have developed fat stores and may be more resistant to environmental hypothermia, thus making infections more likely to be an etiology of hypothermia. The relationship between patient age and risk of infection in hypothermic infants requires greater exploration.

Similarly, the relationship between gestational age and risk of bacterial infection in hypothermic infants needs further investigation. Premature infants have made up between 25% and 75% of hypothermic infant populations previously studied.^2,5,7,22  This was reflected in our study because 352 of the 783 (44.7%) well-appearing hypothermic infants who initially met inclusion criteria were ultimately excluded because of prematurity. A previous study also suggests that hypothermic infants are more likely to have been born prematurely compared with febrile infants.²²  Ramgopal et al examined a cohort of hypothermic infants ≤60 days old (46.9% born before 36 weeks’ gestation), and prematurity was not found to be associated with SBI in their analysis.⁵  Specific tools for hypothermic infants that include a wide range of patient chronologic and gestational ages are needed.

Specific risk-stratification tools for hypothermic infants could help identify hypothermic infants who may be managed with observation alone and avoid more invasive measures such as lumbar puncture or empirical intravenous antibiotics. The authors of previous studies of hypothermic infants have explored factors associated with SBI and IBI. Ramgopal et al found that higher ANC and lower platelet count were associated with SBI in their cohort of infants ≤60 days old.⁵  In the primary analysis from the Hypothermic Young Infant Collaborative, Raffaele et al found significant associations between IBI in infants ≤90 days of age and repeated temperature instability, abnormal WBC count, and thrombocytopenia.⁴  Notably, repeated temperature instability and thrombocytopenia are not part of the criteria for any of the febrile decision tools but may be useful in the risk stratification of hypothermic infants. The role of IMs in detecting inflammation and infection in hypothermic infants is not clear. Hypothermia may be the result of an altered immune response to infection^34–36  which may affect IM results. Therefore, reliance on IMs in risk stratification of hypothermic infants may potentially lead to missed detection of infection. These preliminary studies further support the need for risk stratification tools specific to hypothermic infants.

Our study has several limitations. First, our cohort was limited to hypothermic infants who had a laboratory evaluation performed. This potential selection bias was necessary to allow for the application of febrile infant decision tools but could impact the rate of infections reported and the performance of the febrile decision tools. Similarly, almost all infants in the cohort had at least a partial sepsis evaluation performed (93%) or were hospitalized (97%). Variation in the evaluation and hospitalization of hypothermic infants has been described^8,9  but the differences between our study and previous single-center studies should be noted. For example, Kasmire et al found that in their cohort of 116 infants ≤60 days of age with hypothermia, 72% did not have any bacterial cultures drawn, but 67% were hospitalized.²  Likewise, Perry et al found that 75% of infants in their cohort did not have any bacterial cultures obtained, although they used a different definition of hypothermia (<36.5C).³  The higher rates of evaluation and hospitalization in our cohort are likely due to substantial practice variation among hospitals; however, it could be the result of a sicker patient population, which is less likely because only well-appearing infants were included. In addition, the tools studied each had differing requirements for stratification, which also limited the study population. The results of our analysis could have been different if the necessary testing to apply every tool had been performed on each infant. Second, our study had a relatively small sample size of infants with bacterial infections, thereby limiting the precision of the point estimates for each component. Third, although febrile infant decision tools have primarily been studied in the ED setting, our study incorporated infants who were also directly admitted to the hospital. Although this was a different clinical environment, only a minority of infants were directly admitted; they were all well-appearing, and pathways incorporated in the ED were also used in the hospital. Fourth, we did not capture infants who re-presented to a different hospital with an infection, and we included infants without bacterial cultures obtained from all sites (urine, blood, and CSF), making it possible that we might not have identified some infants with an infection. We reviewed subsequent encounters to ensure that infections were not missed, and most infants in the cohort were hospitalized (97.4%), making it less likely that infections would not be identified clinically or on culture during the index visit or found on re-presentation to another facility. Finally, although the decision tools had a high negative predictive value (>96%), this was likely driven by the high prevalence of infants without a bacterial infection.

The majority of febrile infant guidelines have a high sensitivity and negative predictive value but low specificity and positive predictive value when used for the risk stratification of infants ≤90 days presenting with hypothermia. The application of guidelines designed for infants presenting with fever may lead to unnecessary invasive testing and increased medical costs if used to guide decisions for hypothermic infants. The creation of decision tools and eventually practice guidelines specific to the management of infants presenting with hypothermia could improve high-value care for this population.

We wish to acknowledge efforts of the additional members of our Hypothermic Young Infant Research Collaborative, including Meenu Sharma, Stephanie Prudencio, Evan Ingram, Sanford Williams, Meghan Gray, Noah Hellermann, Kira Molas-Torreblanca, Elizabeth Rinaldi, Melissa Burns, Amber Domako, Madhuri Prasad, and Rachael Mullin.