Performance of AAP Clinical Practice Guideline for Febrile Infants at One Pediatric Hospital Open Access

We performed a secondary data analysis using electronic health records from 2011 to 2018 at 1 pediatric emergency department, collected for a separate study.⁵  The institutional review board approved the study.

We used manual data abstraction techniques with high interrater reliability (κ = 0.96–0.99) to identify all infants ≤90 days old with temperatures of ≥38°C who had a blood culture performed.^5,6  For the main analysis, and to align with the CPG,¹  we excluded infants aged <8 days or >60 days and those with ill appearance, gestational age at birth of <37 weeks (ie, preterm), bronchiolitis, complicated neonatal courses, focal bacterial infections, predisposition to infection (eg, immune dysfunction) or congenital/chromosomal abnormalities, or recent immunizations (ie, within 48 hours before the encounter).¹  We classified infants as ill-appearing if they were described as toxic, limp, inconsolable, ill-appearing, listless, lethargic, irritable, or unresponsive.^5–7  The outcome was IBI, as defined in the CPG.¹ 

We applied the CPG to infants aged 8 to 21, 22 to 28, and 29 to 60 days to identify high-risk infants who may merit lumbar puncture performance, hospitalization, and parenteral antimicrobial therapy. For the main analysis, we considered all infants 8 to 21 days of age to be high-risk. For infants 22 to 60 days old, the CPG recommends clinicians use the following inflammatory markers and thresholds to denote high-risk designation in the absence of procalcitonin: temperature >38.5°C, C-reactive protein >20mg/L, and ANC >4000/mm³.¹  At this institution, clinicians routinely obtain temperature and ANC, but it is not standard practice to perform C-reactive protein for all febrile infants. For the main analysis, because infants for whom clinicians do not obtain inflammatory markers likely represent a group of infants with lower IBI risk relative to other infants, we considered missing laboratory values to be normal.^5,8  Thus, we included infants without C-reactive protein results. In accordance with the CPG,¹  we did not consider urinalysis results for purposes of IBI risk stratification. We assessed CPG performance in detecting IBIs by calculating area-under-the-receiver operating characteristic curve (AUC), sensitivity, specificity, positive/negative predictive values, and positive/negative likelihood ratios. We also evaluated CPG performance for infants aged 22 to 60 days old.

The CPG allows for a wide array of interpretations, clinical discretion, and shared decision-making, which may result in varying performance characteristics. To capture this variability and to demonstrate the performance of different applications of the CPG, we performed 3 sensitivity analyses. First, rather than risk stratifying infants 22 to 28 days old with CPG inflammatory markers and thresholds, we a priori designated infants in this age group as high-risk (Sensitivity #1). Second, we estimated CPG performance with a higher ANC threshold of 5200/mm³, as described in the CPG (Sensitivity #2).¹  Third, because the inclusion of infants with missing data may bias results, we estimated CPG performance when we excluded low-risk infants 22 to 60 days old without C-reactive protein (Sensitivity #3). We included infants in this analysis if they were missing C-reactive protein results, but temperature or ANC were elevated, denoting high-risk status. Using the electronic health record, which also captures data from the other major regional health system, we tracked infants for missed IBIs within 7 days from the initial encounter. We performed analyses using Stata (version 16.1, College Station, TX).

Of 1551 febrile infants ≤90 days old, 507 infants met the inclusion criteria for the main analysis (Fig 1). Nine infants had an IBI. Subject characteristics are shown in Table 1. Results from the main analysis revealed an AUC of 0.673 (95% confidence interval 0.652–0.694), sensitivity of 100% (64.4%–100%), and specificity of 34.5% (30.4%–38.9%; Table 2). For infants 22 to 60 days old, we observed a sensitivity of 100% (54.1%–100%) and specificity of 43.0% (38.1%–48.0%). Sensitivity analysis #2 (ie, higher ANC threshold of 5200/mm³) revealed the highest performance with an AUC of 0.691 (0.669–0.712), sensitivity of 100% (0.664–1), and specificity of 38.2% (33.9%–42.6%). See Table 2 for the results of the remaining analyses.

Temperature and ANC results were available for all infants. Seventy-two infants received C-reactive protein and 296 had an elevated inflammatory marker placing them at high risk for IBI. There were 139 infants missing C-reactive protein who we included in the main analysis and excluded for Sensitivity Analysis #3. No low-risk infants without C-reactive protein had an IBI. No discharged infants had an IBI within 7 days of the initial encounter.

In the absence of procalcitonin, the CPG revealed high sensitivity in detecting IBIs. This is critical to avoid missed IBIs and aligns with findings from Burstein et al.⁴  Clinicians should feel reassured that missed IBIs appear unlikely with CPG adoption, even without procalcitonin. However, we observed specificities substantially lower (9%–38%) relative to results from validation studies for the Step-by-Step (47%) prediction model,⁹  the Pediatric Emergency Care Applied Research Network (60%) prediction model,^10,11  and Burstein’s single-site study of CPG performance (46%)⁴  but similar to the Aronson model (26%).¹² 

Our findings suggest CPG implementation in the absence of procalcitonin may result in hospitalization and treatment of a high proportion of febrile infants (∼65%) without IBIs. As a retrospective, single-site study with a relatively small sample size, nonuniversal collection of inflammatory markers, and an uncommon IBI outcome, the observed low specificity may be related to the study design. Shared decision-making may also affect findings, but we would expect consideration of caregiver input and preferences to attenuate specificities within the observed ranges shown here, rather than to systematically improve specificity above and beyond that which we observed. The observed low specificity may also be due in part to the inflammatory markers and threshold values selected, the exact combination and sequence of which had not been investigated before CPG publication. Other researchers or clinicians may reasonably choose different combinations, each with different performance characteristics. This finding highlights a central challenge for the CPG in which its flexibility may enhance clinical adoption but may not improve measures of variation in care, or performance, to the extent that may be desired. Our findings should be considered preliminary and merit further prospective confirmation.

The CPG represents an important consensus-based approach to evaluating and managing febrile infants, but it excluded >40% of febrile infants 8 to 60 days old with other characteristics (eg, preterm). CPG specificity is likely lower in clinical practice when clinicians must decide how to evaluate and manage all febrile infants, exposing a large proportion of infants to unnecessary hospitalizations and treatments. Future studies should investigate the extent to which exclusion characteristics contribute to IBI risk, thereby enhancing guidance for clinicians and possibly permitting a more personalized approach for infants to safely avoid hospitalization and treatment.

This study has several limitations. First, the sample size is small and from a single site. Second, this retrospective study may be subject to documentation and/or reporting errors. We sought to minimize this potential source of bias by using a rigorous data abstraction process with high interrater reliability.⁵  Third, a substantial proportion of infants were missing C-reactive protein. However, this limitation did not seem to affect results, as sensitivity remained 100%. In addition, no infants without C-reactive protein had an IBI, suggesting clinicians may have considered other factors not present in the CPG when stratifying IBI risk. The observed specificity of 35% may be an overestimate because it is likely that C-reactive protein would have been elevated for a subset of infants without C-reactive protein. Further evaluation is needed to understand CPG performance when all recommended studies are routinely obtained and the extent to which additional risk factors should be considered to stratify IBI risk. Fourth, although we performed several sensitivity analyses to investigate CPG performance, there may be alternative CPG applications that outperform the approaches in this study. A prospective investigation of the CPG is needed to understand how performance may change in real-world settings, especially when clinicians engage with caregivers in shared decision-making.

Findings suggest the CPG can be highly sensitive in the absence of procalcitonin, minimizing missed IBIs, but specificity may be lower than previously reported.^4,9,11,12  Future studies should prospectively investigate CPG performance without procalcitonin in larger, multisite samples. Further investigation is needed to provide guidance for infants excluded from the CPG.

We would like to thank Dr Lauren Solan, MD, MEd, Associate Professor of Pediatrics, University of Rochester, (Rochester, NY) for her thoughtful suggestions and revisions.