Video Abstract
In 2021, the American Academy of Pediatrics (AAP) published the Clinical Practice Guideline (CPG) for management of well-appearing, febrile infants 8 to 60 days old. For older infants, the guideline relies on several inflammatory markers, including tests not rapidly available in many settings like C-reactive protein (CRP) and procalcitonin (PCT). This study describes the performance of the AAP CPG for detecting invasive bacterial infections (IBI) without using CRP and PCT.
This retrospective cohort study included infants aged 8 to 60 days old presenting to Kaiser Permanente Northern California emergency departments between 2010 and 2019 with temperatures ≥38°C who met AAP CPG inclusion criteria and underwent complete blood counts, blood cultures, and urinalyses. Performance characteristics for detecting IBI were calculated for each age group.
Among 1433 eligible infants, there were 57 (4.0%) bacteremia and 9 (0.6%) bacterial meningitis cases. Using absolute neutrophil count >5200/mm3 and temperature >38.5°C as inflammatory markers, 3 (5%) infants with IBI were misidentified. Sensitivities and specificities for detecting infants with IBIs in each age group were: 8 to 21 days: 100% (95% confidence interval [CI] 83.9%–100%) and 0% (95% CI 0%–1.4%); 22 to 28 days: 88.9% (95% CI 51.8%–99.7%) and 40.4% (95% CI 33.2%– 48.1%); and 29 to 60 days: 93.3% (95% CI 77.9%–99.2%) and 32.1% (95% CI 29.1%– 35.3%). Invasive interventions were recommended for 100% of infants aged 8 to 21 days; 58% to 100% of infants aged 22 to 28 days; and 0% to 69% of infants aged 29 to 60 days.
When CRP and PCT are not available, the AAP CPG detected IBI in young, febrile infants with high sensitivity but low specificity.
The 2021 American Academy of Pediatrics Clinical Practice Guideline provides guidance for the management of febrile infants aged 8 to 60 days. It relies on C-reactive protein and procalcitonin for infants aged >21 days, tests which are not rapidly available in many clinical settings.
Our study found that the American Academy of Pediatrics Clinical Practice Guideline algorithms detect invasive bacterial infection in young, febrile infants with high sensitivity and low specificity when applied without C-reactive protein and procalcitonin. Its sensitivity is comparable to existing protocols, whereas its specificity is significantly lower.
In August 2021, the American Academy of Pediatrics (AAP) published its first Clinical Practice Guideline (CPG) on the management of the well-appearing, febrile infant, consisting of 21 key action statements and 3 algorithms summarizing recommended management for infants aged 8 to 21, 22 to 28, and 29 to 60 days.1 The AAP CPG algorithm recommends obtaining blood, urine, and cerebrospinal fluid cultures, administration of empirical parenteral antibiotics, and hospitalization for all febrile infants aged 8 to 21 days old. The algorithms for the 2 older age groups are notable for providing management pathways with multiple decision points, allowing for shared decision-making and recommendations for common clinical scenarios such as traumatic lumbar punctures. Unlike most existing protocols,2–7 they do not recommend management on the basis of a binary high-risk or low-risk categorization; instead, the management of febrile infants aged >21 days is largely guided by the presence or absence of “positive inflammatory markers (IMs),” defined as an absolute neutrophil count (ANC) >4000/mm3, C-reactive protein (CRP) >20 mg/L, and procalcitonin (PCT) >0.5 ng/mL. In the absence of PCT, a higher threshold of ANC (ANC >5200/mm3), CRP >20 mg/L, and temperature >38.5°C are recommended as IMs instead. The lower ANC of 4000/m3 was identified as the ideal cutoff in the Pediatric Emergency Care Applied Research Network febrile infant rule which included PCT (N. Kuppermann, personal correspondence, 2021), whereas the higher ANC of 5200/mm3 was identified by the Febrile Young Infant Research Collaborative study as the best cutoff in a prediction rule which did not have PCT available.7
Although each laboratory criterion is evidence-based, the test characteristics for the detection of invasive bacterial infection ([IBI]; bacteremia and/or bacterial meningitis) have not been studied for this combination of IMs. Additionally, the guideline does not address what the best approaches are when CRP and PCT results are not accessible in a clinically relevant time frame, as is the case in many community hospitals and most medical offices. A recent study published in March 2022 found the AAP CPG, when applied without PCT, detected IBI with a sensitivity of 100% (95% confidence interval [CI] 87.2%–100%).9 However, this study was performed with data from a single Canadian center over a 4-year period, with a cohort of 957 infants, of whom 27 (2.8%) had IBIs. Our study aims to describe the test characteristics of the AAP CPG without the use of CRP and PCT in a much larger and more diverse population.
Methods
This study was approved by the Kaiser Permanente Northern California (KPNC) institutional review board. Informed consent was waived.
Study Design
This study retrospectively examined the performance of the AAP CPG in 2527 febrile infants aged 8 to 60 days evaluated in KPNC emergency departments (EDs) from 2010 to 2019. Using the cohort of infants meeting the AAP CPG inclusion criteria (Supplemental Table 6), the sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio for IBI were calculated for each age group (with Wilson score 95% CIs), along with the percentage of infants recommended to undergo invasive interventions which include lumbar puncture (LP), parenteral antibiotics, and hospitalization. These parameters were then compared with those of the Roseville, Rochester, Philadelphia, and Boston protocols, as published in Nguyen et al 2021, whose patient cohort overlapped significantly with the current study.7 Parameters for the Philadelphia and Boston protocols for infants aged <28 days were calculated using data from the current study cohort. These 4 protocols do not use CRP and PCT for their risk stratification. Although the AAP CPG includes separate recommendations for infants aged 8 to 21 days and 22 to 28 days, for comparability across protocols, these groups were combined and comparisons were made across 2 groups: infants aged <28 days and infants aged 29 to 60 days.
We recognize that the AAP deliberately allows for the exercise of clinical judgement with regard to medical interventions and hospitalizations. Where the AAP CPG allows clinical discretion (eg, “may” perform an intervention), the data were analyzed twice, as if the most conservative choices were selected and as if the least conservative choices were selected, and the results of both analyses reported as a range. Where the AAP CPG allows clinical discretion (eg, “need not” perform an intervention), the data were analyzed to include the least conservative option. Most conservative was defined as pursuing the most invasive interventions and least conservative was defined as pursuing the fewest invasive interventions. Most clinicians and families seek to minimize invasive interventions not considered absolutely necessary. Statistical analyses were performed using SAS 9.4 (SAS Institute, Inc, Cary, NC).
Within the KPNC system, obtaining a rapid CRP result has only recently become possible in the EDs, but is still not available for outpatient medical clinics. PCT rapid result is currently not available in any KPNC ED or outpatient medical clinic. Therefore, most infants in this retrospective cohort lacked these test results. In accordance with the CPG’s recommendation, for infants aged 22 to 60 days without PCT, IMs were temperature >38.5°C and ANC >5200/mm3. A subset of febrile infants who had CRP results was analyzed separately. In addition, a subanalysis of the AAP CPG was performed using ANC >5200/mm3 as the sole IM.
Study Setting
KPNC is a large, integrated care system, with >4 million members receiving care in >40 pediatric clinics, 21 EDs, and 10 pediatric hospital wards in Northern California. As a closed health care system, most members receive all their care internally. KPNC uses an Epic-based complete electronic health record (EHR) fully deployed in 2010. This EHR captures encounters from a comprehensive network of inpatient, ED, and ambulatory care, as well as laboratory, pharmacy, and imaging records and data.
Cohort Identification (Fig 1)
Infants born within KPNC presenting to a KPNC ED between 8 and 60 days of age were included if they had temperature ≥38°C measured by any method within 4 hours of arrival to an ED or measured by any method at a medical clinic visit within 12 hours preceding ED arrival, and had a complete blood count (CBC), blood culture, and urinalysis (UA) obtained for that episode of illness. Although the AAP CPG includes infants with documented fever at home, we included only infants with fever documented in a medical setting. We chose to use only infants with fever documented in a medical setting given the heterogeneity of fever documentation and our previous finding that 9.3% of infants with a chief complaint or final diagnosis of fever did not have a documented fever.10 For patients who had multiple ED visits for the same febrile illness, only the first episodes were captured, whereas the subsequent visits were excluded. Patients were also excluded from the final cohort using the AAP CPG exclusion criteria (Supplemental Table 6).
Definitions
See Supplemental Table 7.
Data Collection
Data were electronically extracted from the EHR and associated databases, and abstracted from the medical record via clinician review assisted by natural language processing. Data extracted included age in days and gestational age in weeks, maternal and neonatal infection history, comorbidity flag,11 maximum temperature in the first 4 hours in the ED or in clinic visit within the 12 hours preceding arrival to ED, immunizations within 48 hours of ED visit, and laboratory test results during and before the ED visit (CBC with differential, UA, cultures, and CRP and PCT when available). A minority of patients had a CRP obtained and a very minimal number had PCT, so the PCT was excluded from further analysis. IBIs were defined by culture results (Supplemental Table 7). When necessary, medical record review was used to confirm the diagnosis of bacteremia and/or bacterial meningitis. Data manually abstracted by medical record review included well versus ill appearance, presence of a clinically identifiable focal bacterial infection, Herpes simplex virus infection, or clinical bronchiolitis, as documented in the treating physician’s examination and assessment.
Results
Patient Population
There were 2527 infants who met enrollment criteria and who presented to a KPNC ED between 8 and 60 days of age with measured temperatures ≥38°C within 4 hours of ED arrival or measured in clinic within the 12 hours preceding arrival. Of these, 1937 had at least a CBC, blood culture, and UA obtained, and 1433 infants met the remaining AAP CPG inclusion criteria and comprised the final analytic cohort. There were 504 infants excluded using the AAP CPG exclusion criteria. See Table 1 for demographic information and Supplemental Table 6 for a breakdown of excluded infants by age and criterion. There were 60 (4.2%) cases of IBIs, 57 (4.0%) with bacteremia, and 9 (0.6%) with bacterial meningitis.
Demographics
. | 8–21 D Old . | 22–28 D Old . | 29–60 D Old . | Overall . |
---|---|---|---|---|
Total infants, n | 292 | 187 | 954 | 1433 |
Age, mean (SD) | 14.9 (3.8) | 25.1 (2.0) | 44.9 (9.0) | 36.2 (14.7) |
Sex, female, n (%) | 110 (37.7) | 78 (41.7) | 418 (43.8) | 606 (42.3) |
Gestational age in wk, mean (SD) | 39.2 (1.2) | 39.3 (1.1) | 39.2 (1.1) | 39.2 (1.1) |
Maternal race/ethnicity, n (%) | ||||
Asian/Pacific Islander | 68 (23.3) | 41 (21.9) | 244 (25.6) | 353 (24.6) |
Black | 10 (3.4) | 18 (9.6) | 73 (7.7) | 101 (7.1) |
Hispanic | 88 (30.1) | 51 (27.3) | 289 (30.3) | 428 (29.9) |
White | 113 (38.7) | 73 (39.0) | 313 (32.8) | 499 (34.8) |
Other/unknown | 13 (4.5) | 4 (2.1) | 35 (3.7) | 52 (3.6) |
Culture results, n (%) | ||||
UTI alone | 72 (24.7) | 30 (16.0) | 188 (19.7) | 290 (20.2) |
Bacteremia alone | 5 (1.7) | 4 (2.1) | 8 (0.8) | 17 (1.2) |
Meningitis alone | 0 | 2 (1.1) | 1 (0.1) | 3 (0.2) |
UTI + bacteremia | 13 (4.5) | 3 (1.6) | 18 (1.9) | 34 (2.4) |
UTI + meningitis | 0 | 0 | 0 | 0 |
Bacteremia + meningitis | 2 (0.7) | 0 | 2 (0.2) | 4 (0.3) |
UTI + bacteremia + meningitis | 1 (0.3) | 0 | 1 (0.1) | 2 (0.1) |
. | 8–21 D Old . | 22–28 D Old . | 29–60 D Old . | Overall . |
---|---|---|---|---|
Total infants, n | 292 | 187 | 954 | 1433 |
Age, mean (SD) | 14.9 (3.8) | 25.1 (2.0) | 44.9 (9.0) | 36.2 (14.7) |
Sex, female, n (%) | 110 (37.7) | 78 (41.7) | 418 (43.8) | 606 (42.3) |
Gestational age in wk, mean (SD) | 39.2 (1.2) | 39.3 (1.1) | 39.2 (1.1) | 39.2 (1.1) |
Maternal race/ethnicity, n (%) | ||||
Asian/Pacific Islander | 68 (23.3) | 41 (21.9) | 244 (25.6) | 353 (24.6) |
Black | 10 (3.4) | 18 (9.6) | 73 (7.7) | 101 (7.1) |
Hispanic | 88 (30.1) | 51 (27.3) | 289 (30.3) | 428 (29.9) |
White | 113 (38.7) | 73 (39.0) | 313 (32.8) | 499 (34.8) |
Other/unknown | 13 (4.5) | 4 (2.1) | 35 (3.7) | 52 (3.6) |
Culture results, n (%) | ||||
UTI alone | 72 (24.7) | 30 (16.0) | 188 (19.7) | 290 (20.2) |
Bacteremia alone | 5 (1.7) | 4 (2.1) | 8 (0.8) | 17 (1.2) |
Meningitis alone | 0 | 2 (1.1) | 1 (0.1) | 3 (0.2) |
UTI + bacteremia | 13 (4.5) | 3 (1.6) | 18 (1.9) | 34 (2.4) |
UTI + meningitis | 0 | 0 | 0 | 0 |
Bacteremia + meningitis | 2 (0.7) | 0 | 2 (0.2) | 4 (0.3) |
UTI + bacteremia + meningitis | 1 (0.3) | 0 | 1 (0.1) | 2 (0.1) |
UTI, urinary tract infection.
Test Characteristics of the AAP CPG Without CRP and PCT
The sensitivity of the AAP CPG was 100% (95% CI 83.9%–100%) and the specificity was 0% (95% CI 0%–1.4%) among infants aged 8 to 21 days, all of whom are recommended to undergo complete evaluations which include LP, empirical parenteral antibiotics, and hospitalization.
Using the combination of ANC >5200/mm3 and temperature >38.5°C as positive IMs, among infants aged 22 to 28 days, the AAP CPG performed with a sensitivity of 88.9% (95% CI 51.8%–99.7%) and specificity of 40.4% (95% CI 33.2%–48.1%). Among infants aged 29 to 60 days, it performed with a sensitivity of 93.3% (95% CI 77.9%–99.2%) and specificity of 32.1% (29.1%–35.3%). (Table 2).
Performance Characteristics of the AAP CPG, Using ANC >5200/mm3 and Temperature >38.5°C as IMs
. | 8–21 D Old . | 22–28 D Old . | 29–60 D Old . |
---|---|---|---|
Total infants, n | 292 | 187 | 954 |
True positive | 21 | 8 | 28 |
False positive | 271 | 106 | 627 |
False negative | 0 | 1 | 2 |
True negative | 0 | 72 | 297 |
Sensitivity, % (95% CI) | 100.0 (83.9–100) | 88.9 (51.8–99.7) | 93.3 (77.9–99.2) |
Specificity, % (95% CI) | 0.0 (0–1.4) | 40.4 (33.2–48.1) | 32.1 (29.1–35.3) |
PPV % (95% CI) | 7.2 (7.2–7.2) | 7.0 (5.5–8.9) | 4.3 (3.9–4.7) |
NPV % (95% CI) | NA | 98.6 (91.8–99.8) | 99.3 (97.5–99.8) |
PLR (95% CI) | 1.00 (1.00–1.00) | 1.49 (1.15–1.94) | 1.38 (1.24–1.53) |
NLR (95% CI) | NA | 0.27 (0.04–1.76) | 0.21 (0.05–0.79) |
. | 8–21 D Old . | 22–28 D Old . | 29–60 D Old . |
---|---|---|---|
Total infants, n | 292 | 187 | 954 |
True positive | 21 | 8 | 28 |
False positive | 271 | 106 | 627 |
False negative | 0 | 1 | 2 |
True negative | 0 | 72 | 297 |
Sensitivity, % (95% CI) | 100.0 (83.9–100) | 88.9 (51.8–99.7) | 93.3 (77.9–99.2) |
Specificity, % (95% CI) | 0.0 (0–1.4) | 40.4 (33.2–48.1) | 32.1 (29.1–35.3) |
PPV % (95% CI) | 7.2 (7.2–7.2) | 7.0 (5.5–8.9) | 4.3 (3.9–4.7) |
NPV % (95% CI) | NA | 98.6 (91.8–99.8) | 99.3 (97.5–99.8) |
PLR (95% CI) | 1.00 (1.00–1.00) | 1.49 (1.15–1.94) | 1.38 (1.24–1.53) |
NLR (95% CI) | NA | 0.27 (0.04–1.76) | 0.21 (0.05–0.79) |
NA, not applicable; NLR, negative likelihood ratio; NPV, negative predictive value; PLR, positive likelihood ratio; PPV, positive predictive value.
Out of 1433 infants with a total of 60 cases of IBIs, 3 (5%, 95% CI 1.04%–13.9%) cases of IBI were misidentified. All 3 misidentified IBI cases were infants with bacteremia. Case 1 was a 7-week-old infant with concurrent Influenza B and Escherichia coli bacteremia. Case 2 was a 27-day-old infant with group B Streptococcus agalactiae (GBS) bacteremia. Both cases had negative urine cultures. Case 3 was a 7-week-old infant with blood and urine cultures positive for GBS. No infants with meningitis were misidentified.
Interventions Recommended by the AAP CPG Without CRP and PCT
The AAP CPG recommends LP, empirical parenteral antibiotics, and hospitalization for all febrile infants aged 8 to 21 days. Ranges are reported because of the multiple management options built into the CPG. Using ANC >5200/mm3 and temperature >38.5°C as IMs, the AAP CPG recommends LP for 61% to 100% of infants aged 22 to 28 days and 0% to 69% of infants aged 29 to 60 days; parenteral antibiotics for 58% to 100% of infants aged 22 to 28 days and 56% to 69% of infants aged 29 to 60 days; and hospitalization for 65% to 100% of infants aged 22 to 28 days and 56% to 69% of infants aged 29 to 60 days (Table 3).
Interventions Recommended by AAP CPG, using ANC >5200/mm3 and Temperature >38.5°C as IMs
. | 8–21 D Old, N = 292, n (%) . | 22–28 D Old, N = 187, n (%) . | 29–60 D Old, N = 954, n (%) . |
---|---|---|---|
LP | 292 (100) | 114–187a (61–100) | 0–655a (0–69) |
Parenteral antibiotics | 292 (100) | 108–187a (58–100) | 539–655a (56–69) |
Hospitalization | 292 (100) | 121–187a (65–100) | 539–655a (56–69)a |
. | 8–21 D Old, N = 292, n (%) . | 22–28 D Old, N = 187, n (%) . | 29–60 D Old, N = 954, n (%) . |
---|---|---|---|
LP | 292 (100) | 114–187a (61–100) | 0–655a (0–69) |
Parenteral antibiotics | 292 (100) | 108–187a (58–100) | 539–655a (56–69) |
Hospitalization | 292 (100) | 121–187a (65–100) | 539–655a (56–69)a |
Where the AAP CPG allows clinical discretion (eg, may perform an intervention), the data were analyzed twice, as if the most conservative choices were selected and as if the least conservative choices were selected, and the results of both analyses were reported as a range. Where the AAP CPG allows clinical discretion (eg, need not perform an intervention), the data were analyzed to include the least conservative option.
CRP Subanalysis
Within the cohort of 1433 infants, 435 infants (30.3%) had a CRP and only 14 (0.01%) had a PCT. Among those who had CRP done, there were 26 (6%) cases of IBI, with 332 (76.3%) having either ANC >5200/mm3 or temperature >38.5°C, or both. When the AAP CPG was applied to the 435 infants who had a CRP using only ANC and temperature as IMs, 1 case of IBI (a 27 day old infant with GBS bacteremia) was misclassified. With the addition of CRP >20 mg/L as IM, 10 additional infants were considered to have positive IMs, but the single case of IBI was still missed (Table 4).
Added Value of CRP >20 mg/L to ANC >5200/mm3 and Temperature >38.5°C as IMs
. | 8–21 D Old (n = 93) . | 22–28 D Old (n = 56) . | 29–60 D Old (n = 286) . |
---|---|---|---|
IM = ANC and temperature | |||
Cases with positive IM | 93 (9 TP) | 39 (2 TP) | 200 (14 TP) |
Missed IBI | 0 (84 FP) | 1 (37 FP) | 0 (186 FP) |
IM = ANC, temperature, and CRP | |||
Cases with positive IM | 93 (9 TP) | 40 (2 TP) | 209 (14 TP) |
Missed IBI | 0 (84 FP) | 1 (38 FP) | 0 (195 FP) |
. | 8–21 D Old (n = 93) . | 22–28 D Old (n = 56) . | 29–60 D Old (n = 286) . |
---|---|---|---|
IM = ANC and temperature | |||
Cases with positive IM | 93 (9 TP) | 39 (2 TP) | 200 (14 TP) |
Missed IBI | 0 (84 FP) | 1 (37 FP) | 0 (186 FP) |
IM = ANC, temperature, and CRP | |||
Cases with positive IM | 93 (9 TP) | 40 (2 TP) | 209 (14 TP) |
Missed IBI | 0 (84 FP) | 1 (38 FP) | 0 (195 FP) |
FP, false positive; TP, true positive.
Temperature Subanalysis
Removing temperature >38.5°C and using ANC >5200/mm3 as the only positive IM led to a significant reduction in sensitivity: from 88.9% to 66.7% (95% CI 29.9%–92.5%) among infants aged 22 to 28 days (2 additional false negatives) and from 93.3% to 60.0% (95% CI 40.6%–77.3%) among infants aged 29 to 60 days (10 additional false negatives). There were 15 missed cases of IBI, 12 more than if temperature >38.5°C was also included as a positive IM. Removing temperature did lead to a notable improvement in specificity, to 73% (95% CI 65.9%–79.4%) among infants aged 22 to 28 days and 66.5% (95% CI 63.3%–69.5%) among infants aged 29 to 60 days (Supplemental Table 8).
Comparison of AAP CPG Versus Other Published Protocols
Table 5 lists the performance characteristics and recommended interventions by the AAP CPG without CRP and PCT compared with the Roseville, Rochester, Philadelphia, and Boston protocols. For infants aged <28 days, the AAP CPG had a sensitivity of 96.7% (95% CI 82.8%–99.9%), which does not differ statistically from other protocols, whereas the specificity of 16.0% (95% CI 12.8%–19.8%) is better than the Philadelphia and Boston protocols, but worse than the Roseville and Rochester protocols. The AAP CPG recommends invasive interventions for 84% to 100% of infants aged <28 days compared with 100% of infants for the Philadelphia and Boston protocols, 56% to 70% for Roseville, and 51% for Rochester. Among infants aged 29 to 60 days, the AAP CPG performs with comparable sensitivity (93.3%, 95% CI 77.9%–99.2%) to the Roseville, Rochester, Philadelphia, and Boston protocols. The specificity of the AAP CPG, 32.1% (95% CI 29.1%–35.3%) is statistically inferior to all 4 of these protocols (95% CIs 47.9%–67.8%). The AAP CPG recommends invasive interventions for 0% to 69% of infants aged 29 to 60 days, compared with 51% to 100% for Philadelphia, 36% to 100% for Boston, 0% to 26% for Roseville, and 44% for Rochester.
Comparison of Performance Characteristics and Recommended Interventions Between AAP CPG (Using ANC >5200/mm3 and Temperature >38.5°C as IMs) and Other Published Protocols
. | 7–28 D Old . | 29–60 D Old . | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
. | AAPa . | Roseville . | Rochester . | Philadelphia . | Boston . | AAP . | Roseville . | Rochester . | Philadelphia . | Boston . |
Sensitivity, % (95% CI) | 96.7 (82.8–99.9) | 96.7 (82.8–99.9) | 90.0 (73.5–97.9) | 100 (88.4–100) | 100 (88.4–100) | 93.3 (77.9–99.2) | 91.4 (76.9–98.2) | 93.8 (79.2–99.2) | 92.3 (79.1–98.4) | 86.1 (70.5–95.3) |
Specificity, % (95% CI) | 16.0 (12.8–19.8) | 31.7 (27.9–35.6) | 50.7 (46.7–54.8) | 0.0 (0.0–0.8) | 0.0 (0.0–0.8) | 32.1 (29.1–35.3) | 58.5 (55.6–61.4) | 56.9 (54.0–59.8) | 50.6 (47.9–53.2) | 65.2 (62.6–67.8) |
PPV, % (95% CI) | 4.8 (4.46–5.17) | 6.6 (6.1–7.2) | 8.3 (7.2–9.4) | 6.3 (6.3–6.3) | 6.3 (6.3–6.3) | 4.3 (3.9–4.7) | 6.3 (5.7–7.1) | 5.6 (5.0–6.2) | 4.9 (4.5–5.4) | 6.3 (5.5–7.2) |
NPV % (95% CI) | 99.1 (94.0–99.9) | 99.5 (96.5–99.9) | 99.0 (97.2–99.7) | NA | NA | 99.3 (97.5–99.8) | 99.6 (98.7–99.9) | 99.7 (98.9–99.9) | 99.6 (98.8–99.9) | 99.4 (98.7–99.7) |
LP % | 85–100 | 70 | 51 | 100 | 100 | 0–69 | 0–26 | 44 | 100 | 100 |
Parenteral antibiotics (%) | 84–100 | 56 | 51 | 100 | 100 | 56–69 | 26 | 44 | 51 | 100 |
Hospitalization (%) | 86–100 | 70 | 51 | 100 | 36 | 56–69 | 0–26 | 44 | 51 | 36 |
. | 7–28 D Old . | 29–60 D Old . | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
. | AAPa . | Roseville . | Rochester . | Philadelphia . | Boston . | AAP . | Roseville . | Rochester . | Philadelphia . | Boston . |
Sensitivity, % (95% CI) | 96.7 (82.8–99.9) | 96.7 (82.8–99.9) | 90.0 (73.5–97.9) | 100 (88.4–100) | 100 (88.4–100) | 93.3 (77.9–99.2) | 91.4 (76.9–98.2) | 93.8 (79.2–99.2) | 92.3 (79.1–98.4) | 86.1 (70.5–95.3) |
Specificity, % (95% CI) | 16.0 (12.8–19.8) | 31.7 (27.9–35.6) | 50.7 (46.7–54.8) | 0.0 (0.0–0.8) | 0.0 (0.0–0.8) | 32.1 (29.1–35.3) | 58.5 (55.6–61.4) | 56.9 (54.0–59.8) | 50.6 (47.9–53.2) | 65.2 (62.6–67.8) |
PPV, % (95% CI) | 4.8 (4.46–5.17) | 6.6 (6.1–7.2) | 8.3 (7.2–9.4) | 6.3 (6.3–6.3) | 6.3 (6.3–6.3) | 4.3 (3.9–4.7) | 6.3 (5.7–7.1) | 5.6 (5.0–6.2) | 4.9 (4.5–5.4) | 6.3 (5.5–7.2) |
NPV % (95% CI) | 99.1 (94.0–99.9) | 99.5 (96.5–99.9) | 99.0 (97.2–99.7) | NA | NA | 99.3 (97.5–99.8) | 99.6 (98.7–99.9) | 99.7 (98.9–99.9) | 99.6 (98.8–99.9) | 99.4 (98.7–99.7) |
LP % | 85–100 | 70 | 51 | 100 | 100 | 0–69 | 0–26 | 44 | 100 | 100 |
Parenteral antibiotics (%) | 84–100 | 56 | 51 | 100 | 100 | 56–69 | 26 | 44 | 51 | 100 |
Hospitalization (%) | 86–100 | 70 | 51 | 100 | 36 | 56–69 | 0–26 | 44 | 51 | 36 |
NA, not applicable; NPV, negative predictive value; PPV, positive predictive value.
AAP: includes 8 to 28 days old.
Discussion
The AAP CPG for the “Evaluation and Management of Well-Appearing Febrile Infants 8 to 60 Days Old” is an evidence-based set of recommendations and algorithms for a widely studied clinical scenario. Although it addresses a variety of common clinical scenarios and provides guidance for the application of clinician judgment and shared decision-making, it does not address its application to clinical settings where CRP and PCT are not available with rapid turnaround. This study demonstrates that, using only the combination of ANC >5200/mm3 and temperature >38.5°C as IMs, the AAP CPG detects IBI with a high sensitivity but low specificity. Furthermore, whereas recent studies have downplayed the utility of temperature in favor of newer IMs such as PCT,5,6 in this study, temperature was demonstrated to improve sensitivity when these markers were not available.
Without the new biomarkers of CRP and PCT, the AAP CPG increases the rate of potentially unneeded interventions without increasing sensitivity. The 5 protocols compared in this study (Roseville, Rochester, Philadelphia, Boston, and the AAP CPG) demonstrated statistically equivalent sensitivity, with overlapping CIs ranging from 73.5% to 99.9% (Fig 2). The specificity of the AAP CPG is statistically inferior in both age groups to comparison protocols, except for the Philadelphia and Boston protocols for infants aged <29 days. Compared with the Philadelphia and Boston protocols, the AAP CPG spares up to 42% of interventions in infants aged 22 to 28 days by using IMs for risk stratification instead of universally recommending LP, antibiotics, and hospitalization; for older infants, the AAP CPG recommends fewer LPs but more hospital admissions. Compared with the Rochester and Roseville protocols, the AAP CPG recommends more interventions across all age groups.
Comparison of performance characteristics between AAP CPG (using ANC >5200/mm3 and temperature >38.5°C as IMs) and other published protocols.
Comparison of performance characteristics between AAP CPG (using ANC >5200/mm3 and temperature >38.5°C as IMs) and other published protocols.
The risks of testing, hospitalization, and treatment are relatively small, but occur at high frequency. A study by Baker in 1993 reported an 18.9% risk of infiltration of an intravenous line and a 0.4% risk of drug-related rash.3 DeAngelis in 1983 reported complications in 19.5% of hospitalizations, ranging from intravenous infiltrates, fluid overload, accidental antibiotic overdose, and iatrogenic fever (high isolete temperature) to “distraught mother.”12 DeAngelis’ study also reported a significant number of contaminated cerebrospinal fluid cultures, contaminated blood cultures, repeated attempts at lumbar puncture, and other “diagnostic misadventures,” all familiar to emergency physicians and pediatric hospitalists. Byington in 2012 found the average cost of hospitalization for a febrile infant at 1 US academic institution to be $7178 in 2007.13 The sum of iatrogenic events, complications, and financial costs are not trivial, increasing the burden on an already overtaxed medical system, especially in the era of the coronavirus disease 2019 pandemic. In addition, such interventions undoubtedly heighten anxiety and pose a financial burden for families.
KPNC is a large, integrated health care system with a robust EHR extending back a decade, providing a large and consistently documented trove of clinical data. The major limitation of this study is that it was not adequately powered to address the contribution of CRP and PCT; however, we are not disputing the use of either PCT or CRP in the approach to the febrile infant, because both have been studied in thousands of febrile infants and the specificity was 60% (Pediatric Emergency Care Applied Research Network)6 and 46.9% (Step-by-Step),5 respectively, in those studies. Rather, our aim was to describe the use of the AAP CPG in the absence of CRP and PCT. In addition, we raise awareness of using IMs in settings which have not been validated in varying infant populations.
Although the AAP CPG contributes an extensive set of evidence-based recommendations for the management of the well-appearing, febrile, young infant, the application of its algorithms in clinical settings without ready availability of CRP or PCT may lead to increased interventions without improved sensitivity for detecting IBI. The authors of the CPG point out that “the recommendations in this guideline do not indicate an exclusive course of treatment or serve as a standard of medical care. Variations, taking into account individual circumstances, may be appropriate.” Until these tests become more widely available, community settings without readily available CRP or PCT may experience limited benefit from the AAP CPG.
Conclusions
When CRP and PCT testing are not available, the AAP CPG algorithms performed with high sensitivity but low specificity for detecting IBI in 8- to 60-day old febrile infants. Compared with some other protocols not utilizing CRP and PCT, the AAP CPG may recommend more invasive interventions while detecting IBI with similar sensitivity. Further studies may inform best practices to reduce unnecessary interventions.
Acknowledgment
We thank Dr Nathan Kuppermann for his insightful review of the manuscript.
Drs Nguyen and Young conceptualized and designed the study, collected data, drafted the initial manuscript, and reviewed and revised the manuscript; Drs Ballard, Greenhow, and Vinson conceptualized and designed the study, collected data, and critically reviewed and revised the manuscript; Drs Mark, Van Winkle, and Sharp conceptualized and designed the study, and critically reviewed and revised the manuscript; Ms Alabaster conceptualized and designed the study, designed the data collection instruments, coordinated data collection, conducted the initial analyses, and reviewed and revised the manuscript; Ms Shan and Ms Rauchwerger assisted in conceptualizing and designing the study, coordinated data collection, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: Supported by Garfield Memorial Fund.
CONFLICT OF INTEREST DISCLAIMER: The authors have indicated they have no conflicts of interest relevant to this article to disclose.
- AAP
American Academy of Pediatrics
- ANC
absolute neutrophil counts
- CBC
complete blood count
- CI
confidence interval
- CPG
Clinical Practice Guideline
- CRP
C-reactive protein
- ED
emergency department
- EHR
electronic health record
- GBS
group B Streptococcus agalactiae
- IBI
invasive bacterial infection
- IM
inflammatory marker
- KPNC
Kaiser Permanente Northern California
- LP
lumbar puncture
- PCT
procalcitonin
- UA
urinalysis
Comments
Response from AAP Guideline Authors
Our goal was to develop recommendations that allowed clinicians to safely do less while permitting variation based on clinical circumstances. We also needed to err on the safe side given the different levels of knowledge and experience of clinicians caring for febrile infants. A major departure from many prevailing models, suggesting complete sepsis evaluations for infants younger than 28-30 days, was to create a new 22-28d group, in which, according to the Nguyen study, 39% of infants would have been spared lumbar punctures and 35% hospitalizations.
There are several points worthy of clarification. The authors claim the CPG does not address applications where PCT is not available. This is not accurate and was subsequently elucidated in an article by one of the CPG authors.(3) Also, the AAP CPG includes urinalysis (UA) as a predictor in its algorithms; Nguyen et al provide no discussion in the methods or results as to if or how UA was used, an important consideration with 36/60 of their IBI cases having UTI + bacteremia. Further, for 29-60 d infants, the AAP CPG only recommends hospitalization for infants with abnormal cerebrospinal fluid CSF analysis. The 56%-69% expected hospitalization rate reported is inconsistent with the published prevalence of abnormal LP results. This misapplication by Nguyen et al makes the AAP CPG seem excessively likely to prompt hospitalizations, which it does not.
A stated concern regarding the 29-60d algorithm is the specificity for detecting invasive bacterial infections (IBI) of 32.4%%, lower than the comparison models, potentially driving unnecessary interventions. A difference in these strategies is in white cell counts. The comparison models used arbitrarily assigned WBC values <5,000 mm3, >15,000 mm3 to indicate high risk. Our absolute neutrophil count (ANC) cutoff was statistically derived for a scoring system in a retrospective study of hospitalized infants where an ANC of 5,200 mm3(2 points) had a specificity of 31.5%. This study, with the largest published number of IBI cases (181) had a respectable 98.8% sensitivity. It also highlighted the fact that reporting on IBI obscures data on bacterial meningitis, where 2 of 26 cases with scores > 2 were missed, prompting the authors to comment that with respect to performing lumbar punctures “clinicians should weigh the full clinical picture and consider the use of shared decision-making with parents.” (4)
An optimally derived ANC might improve theoretical algorithm accuracy compared with the current value used in the CPG or other prevailing 3 or 4 variable models. But in real clinical encounters, clinicians have considerably more information available to guide decision making, such as presence of other febrile illnesses at home, experience/ confidence of parents to recognize clinical changes, ability to timely access follow-up care. The incorporation of these factors in clinical decision-making is a basis for the well-known phenomenon of clinicians not adhering to published protocols.
What we provided is a guideline, not a prediction rule. As such, we have provided appropriate options with clear guardrails within the limits of the data available at the time. The upcoming completion of the AAP implementation study, REVISE 2, will likely provide a valid assessment of the CPG’s value and new insights for guideline revision. Our advice to clinicians viewing Nguyen’s new data would be to consider this discussion, and follow the CPG advice, “The guidance in this report does not indicate an exclusive course of action or serve as a standard of medical care. Variations, taking into account individual circumstances, may be appropriate.”
Robert H Pantell, Kenneth B Roberts, and Nathan Kuppermann for the Writing Group of the AAP Febrile Infant CPG
1. Nguyen THP, Young BR, Alabaster A, et al Using AAP Guidelines for Managing Febrile Infants Without C-Reactive Protein and Procalcitonin. Pediatrics 2023:151(1):e2022058495
2. Pantell RH, Roberts KB, Adams WG,et al. Subcommittee on Febrile Infants.Evaluation and management of well-appearing febrile infants 8 to 60 days old.Pediatrics. 2021;148(2):e2021052228
3. Kuppermann N, Mahajan P, Dayan PS. Fever, Absolute Neutrophil Count, Procalcitonin, and the AAP Febrile Infant Guidelines Pediatrics. 2023:151(2):e2022059862. doi: 10.1542/peds.2022-059862.
4. Aronson PL, Shabanova V, Shapiro ED,et al; Febrile Young Infant ResearchCollaborative. A Prediction model to identify febrile infants #60 days atlow risk of invasive bacterial infection.Pediatrics. 2019;144(1):e20183604