Comparison of Early-Onset Sepsis Risk-Stratification Algorithms in Neonates in a Kenyan Nursery

Tenwek Hospital is a teaching and regional referral center for high-risk obstetrics and neonates in Bomet, Kenya with 3000 deliveries per year. It has a NICU with ∼1500 admissions (including 300–400 referred out-of-hospital births) annually.

The hospital uses an electronic health record and has a quality improvement and research infrastructure, which has fostered a successful intervention to reduce perinatal event frequency and morbidity.²² 

The majority of mothers who deliver at Tenwek receive 2 of 4 Ministry of Health recommended comprehensive prenatal visits. Kenyan guidelines do not include Group B streptococcal (GBS) screening because of a lack of affordable and reliable access to testing. Reported GBS colonization rates in urban Kenyan settings are 12% to 20%.^23,24  In comparison, worldwide colonization is estimated at 18% and, in North America, is 22% to 23%.²⁵  Ampicillin is given to mothers with prolonged rupture of membranes (PROM) >18 hours or signs of chorioamnionitis (foul-smelling fluid, fever, signs of maternal illness during labor). Adequate antibiotics before delivery are defined as at least 1 dose >4 hours before delivery. Notably, when blood cultures from septic neonates have been studied in Sub-Saharan African settings, GBS represents a far lower percentage of isolates than in HIC.^26,27 

A simplified comparison of the 4 RSAs is available in Table 1.

Before December 2019, any neonate admitted to the NICU for concern for sepsis either by maternal illness, neonatal signs, or maternal antibiotic treatment received 7 to 14 days of antibiotics. The neonatal care team developed an evidence-based algorithm utilizing literature review and assessing local clinical capacity and cost to stratify EOS risk and guide antibiotic use. The algorithm consists of 2 steps: clinical assessment and evaluation of maternal risk factors followed by adding a complete blood count (CBC). Full implementation of the algorithm was scheduled for 2020 but was delayed by the coronavirus pandemic. For the purposes of our study, we used the term local RSA to refer to the application of both steps and simple RSA to refer to the clinical step only.

The American Academy of Pediatrics endorses 3 methods of identifying neonates at risk for sepsis based on risk factors, including a categorical RSA using primarily clinical maternal and infant risk factors.²⁸ 

For the purposes of our study, we used the clinical presentation portion of a well-known multivariate RSA,¹⁰  which includes only clinical infant examination and maternal factors (CE-M RSA).^20,29  We chose this clinical RSA rather than a multivariate RSA because neither maternal GBS colonization nor local EOS incidence was readily available in our setting.

The World Health Organization Integrated Management of Childhood Illness (IMCI) guideline^11,21,30  identifies signs of presumed serious bacterial infection in neonates, including EOS, without the use of laboratory values. It was designed primarily for community settings, without attention to capacity variation, to help recognize when to seek a higher level of care. The IMCI guideline alone is not an RSA; however, it is the most commonly applied guideline in LMIC for the identification of EOS. The local RSA was developed with the IMCI guideline in mind but with attention to their increased capacity as a referral hospital (oxygen, continuous positive airway pressure, monitors, etc).

This study was approved by the institutional review boards of both the Kenyan regional referral hospital and the partner academic children’s hospital in the United States. Cases were identified in the local electronic health record (Kranium) by evaluating daily NICU censuses and investigating all patients with >35 weeks’ gestation who were on antibiotics and/or had sepsis, pneumonia, meningitis, or bacteremia in their problem list. Once identified, the charts were reviewed for demographics, infant physical examination findings, laboratory, and maternal factors from the maternal and neonatal medical records. We defined a case as an infant ≥35 weeks’ gestation admitted to the regional referral hospital NICU for sepsis or risk of sepsis from 2018 to 2020. We excluded infants with <35 weeks’ gestation, infants >5 days of age at transfer to the nursery, out-of-hospital births, and those admitted for transient tachypnea of the newborn (isolated tachypnea) and hyperbilirubinemia (isolated jaundice) not treated with antibiotics, known severe birth asphyxia, and those with obvious underlying congenital genetic anomalies. All data were entered into a database by using REDCap.^31,32  We manually applied each of the 4 RSAs to stratify each infant into low, medium (CE-M RSA and local RSAs only), or high EOS risk (Table 1). An initial review was conducted by 1 of 3 investigators (L.F., T.W., A.G.), followed by a final review of all records by 1 investigator (AR). Documentation or context questions were resolved through meetings with Kenyan coinvestigators (C.S., C.R.). Differences in stratification were marked and discussed at biweekly reviewer meetings to reach a classification consensus.

The magnitude of agreement between any 2 RSAs was evaluated by using к coefficients. We determined the direction of disagreement using n × n frequency tables between RSAs and tested for significance using McNemar’s (for 2 × 2 tables) or Bowker’s (for larger tables) tests of symmetry. For the IMCI RSA, we compared the high risk to the combination of high and intermediate risk categories in other RSAs. We used the method local RSA employed to assign expected antibiotic duration (AD) by risk level (high risk = 10 days, intermediate risk = 7 days, low risk = 0 days) for all RSAs and assumed predicted LOS was equal to AD because a hospital stay is predominantly for antibiotic administration. The predicted AD and LOS and the difference between observed and predicted values were summarized by using median and interquartile ranges. The Wilcoxon rank test was used to evaluate if the difference was equal to 0. We excluded the IMCI RSA from AD and LOS comparisons because, with only 2 classes (high and low), we could not set an expected AD. For all tests, a P value of <.05 was considered statistically significant. All analyses were performed by using SAS statistical software version 9.4 (SAS Institute, Cary, NC).

We identified 330 infants with ≥35 weeks’ gestation admitted to the local NICU for sepsis or risk of sepsis from 2018 to 2020. We excluded 68 for failing to meet the criteria. Of the remaining 262 infants, 228 (87%) were ≥37 weeks’ gestation whereas 30 (11.5%) were 35–36 weeks’ gestation. Four infants (1.5%) had an unknown gestational age but were deemed to be term by Ballard score and weight.

The most common admitting diagnosis (186 infants [71%]) was “risk of sepsis secondary to PROM”. Forty-three infants (16%) were admitted for fever. Fifteen (6%) were admitted for “risk of sepsis in the setting of maternal chorioamnionitis.” Eighteen infants (7%) had other diagnoses, including hypoglycemia, hyperbilirubinemia, and birth asphyxia, but were treated as a sepsis rule out with antibiotics and were included (Fig 1).

Agreement between RSAs is summarized in Table 2. CE-M RSA and IMCI agreed highly with each other (к = 0.92). Local RSA and simple RSA agreed with each other only moderately (к = 0.55), and each agreed poorly with CE-M RSA (к = 0.23 and 0.11, respectively). Local RSA agreed slightly better with IMCI (к = 0.41) than did simple RSA (к = 0.20). For both local and simple RSA, >90% of the disagreements with CE-M RSA and/or IMCI were due to overestimating EOS risk. For example, when compared with CE-M RSA, local RSA overestimated EOS in 93% of cases (P <.0001), and simple RSA did so in 99% of cases (P <.0001). Simple RSA overestimated EOS risk when compared with local RSA in 95% of cases (P <.0001; Table 3). A detailed distribution of risk factors is available in Supplemental Tables 5 and 6. Notably, 52% of infants categorized as high risk by the simple RSA and low risk by the CE-M RSA were born to mothers with chorioamnionitis. Seventy-eight percent of infants classified as high risk by the simple RSA but intermediate risk by the CE-M RSA were categorized as such because of fever. Finally, 93% of infants classified as intermediate risk by the simple RSA but low risk by the CE-M RSA were born with PROM.

There was no statistical difference in predicted versus observed median AD when using the local RSA. Use of the CE-M RSA predicted 2 fewer days of antibiotic usage, and the simple RSA predicted 2 more days of antibiotic usage than observed (P <.0001; Table 4). Use of the local RSA predicted a 2-day shorter stay, and the CE-M RSA predicted a 4 day shorter stay than observed (P <.0001; Table 4).

We found that using an RSA in an LMIC nursery has the potential to decrease both AD and LOS. Despite the low agreement between the different RSAs tested, the use of clinical and available laboratory data revealed significant reductions that would benefit patients, facilities, and antibiotic stewardship efforts.

Although most HIC settings have blood culture capacity and local GBS colonization and EOS rates readily available, our findings are applicable to both LMIC and HIC settings. The use of clinical assessment without epidemiologic data has been applied successfully in HIC settings with low GBS colonization.²⁰  Colonization rates in urban Kenyan settings have been reported to be 12% to 21%,^23,24  but studies of blood culture data reveal GBS is not common among Sub-Saharan infants with EOS.^26,29,33  The worldwide colonization rate has been estimated to be 18% and, in North America, 22% to 23%.²⁵  It is therefore likely that colonization in our rural Kenyan setting is similar to if not lower than in many United States settings. Serial clinical assessment is one of the strategies recommended by the American Academy of Pediatrics to manage EOS²⁸  and has been implemented in HIC to determine the need for testing and antibiotics.^34,35  Because resources may be scarce in community settings in both HIC and LMIC, EOS risk stratification is likely to rely heavily on clinical assessment. Finally, antimicrobial resistance and its threat to child morbidity and mortality is a global problem and does not discriminate between high- and low-income settings.⁶  A goal of RSAs is to guide care using appropriate resources and reserving potentially harmful interventions, including the wide use of broad-spectrum antibiotics. Given the inequitable health care systems and resources known to exist in the United States and elsewhere,³⁶  looking to LMIC could help partners in HIC negotiate the challenge of implementing new guidelines and antibiotic stewardship while balancing high-value care.

Nonetheless, including population-based metrics would both enhance RSA performance and drive efforts to reduce harm. Pooling regional or country-level data to better understand incidence may be an important first step to demonstrating metric utility and driving standardized measurement and reporting. Improving local documentation practices and standardizing examination checklists and local hospital statistics may also improve hospital-level surveillance. Future studies should consider bundling EOS RSA in this setting with local and regional surveillance data, capacity building, and sustainability.³ 

We hypothesized that the use of a CBC would add little to local RSA and its cost would outweigh its value.^{4,7,19,20,34,37}  However, our results revealed that local RSA out-performed simple RSA in LOS, suggesting that the addition of 1 or more clinical laboratory values could enhance the utility and accuracy of an RSA in LMICs. In HICs, studies including CBC, blood culture, serum C-reactive protein, procalcitonin, or other biomarkers are often coupled with a clinical assessment to stratify risk and observe patients off antibiotics.³⁸  Blood cultures are the test of choice to rule out or confirm EOS,²⁸  but the cost, equipment, space, and capacity needed for reliable results is beyond the means of many centers. CBCs may be a more viable alternative, with considerably lower cost, existing infrastructure, and the predictive reliability of serial-age-dependent neutrophil counts.³⁹  Using C-reactive protein and procalcitonin to diagnose EOS in HIC has variable sensitivity and requires serial laboratories.^12,31,40  Serum biomarkers have not been tested in LMIC and may be more feasible to implement than blood cultures. Additional studies are needed to explore additional serum marker and blood culture capacity building and implementation to develop sustainable EOS RSAs in LMICs and track antimicrobial resistance reliably.^3,13,14,41 

Both local and simple RSAs overestimated EOS risk compared with both CE-M RSA and IMCI and could lead to more children being on antibiotics for longer. However, erring on the side of caution may be prudent in an LMIC setting because the incidence of EOS is believed to be much higher than in HIC settings,²  and underestimating risk could be catastrophic. The distribution of risk factors (Supplemental Tables 5 and 6) suggests that the overestimation of EOS risk compared with the CE-M RSA primarily occurred in children born to mothers with chorioamnionitis or PROM and infants with fever. These findings support our assertion that overestimation is likely prudent in a setting that lacks reliable prenatal prophylactic protocols and blood culture availability. The higher incidence of EOS in LMIC is likely multifactorial and related to limited access to prenatal care, crowding, malnutrition, lack of safe water and sanitation sources, certain cultural practices (such as covering the umbilical stump with animal feces), home births without a skilled attendant, and a lack of routine GBS screening.^1,3  We attempted to mitigate these factors by excluding out-of-hospital births, and roughly estimate from our experience with chart review in this study that fewer than one-third of the 300 to 400 neonates with >35 week’s gestation admitted annually showed signs of severe illness (data not shown). Additional study is needed to determine the consequence of shorter versus longer antibiotic courses.

Our study is subject to a number of limitations. First, as a retrospective study, it relied on accurate and consistent documentation of vital signs and assessments. For example, criteria from the local RSA, such as “significant resuscitation” and “ill appearing infant” may not have been explicitly specified in the record but, rather, inferred from data. Future studies could implement standardized documentation, either through an electronic health record or a printed checklist, that when completed in real-time may allow more accurate assessments of incidence and outcomes. Second, IMCI indicators are not a true RSA, but rather a set of nonspecific clinical indicators of neonatal illness designed for community health settings. Application in a hospital setting with more resources for monitoring may have introduced classification bias causing underestimation of risk, thus biasing comparison with local RSA away from the null. In the absence of a true LMIC standard RSA, additional study is needed to quantify the extent of this potential effect.

Third, using the CE-M RSA classification of infant presentation without epidemiologic data may have affected its validity. It is reasonable to expect that local GBS colonization rates are comparable to HIC settings,^23,24  thus not affecting RSA determination significantly. Local EOS rates could be higher than HIC²  but may be driven primarily by out-of-hospital births, which we excluded from our study. The lack of reliable population-based data is a reality for many LMICs and challenges standard RSA development.^1,3,13,14  By mimicking a real-world setting, our modification may have been a more accurate portrayal of its use in an LMIC, thus enhancing our findings’ validity. Fourth, our study is unable to determine which infants truly had EOS and which were infected with resistant organisms because blood cultures are not available. Anecdotally, surgical providers at the hospital report that when cultures have been done in the past, they most commonly grew gram-negative, multidrug-resistant bacteria (data not shown). Future work is planned to identify ways to overcome this barrier, a challenge common to LMIC settings. Finally, expected LOS was estimated by using antibiotic duration but could be influenced by factors unrelated to infection or antibiotics, such as feeding difficulties, hyperbilirubinemia treatment, or time needed for a family to pay the hospital bill. Future studies should be designed to account for confounders to assess the true effect of RSA use.

Accurate, timely diagnosis and treatment of neonatal sepsis have the potential to save newborn lives. However, careful attention needs to be paid to minimizing antibiotic resistance while optimizing infant risk stratification. Developing RSAs for LMIC nurseries is an important consideration in balancing this tension. We demonstrated that RSAs can be modified and applied in an LMIC context to stratify neonatal sepsis patients retrospectively. If applied in real-time, RSAs have the potential to decrease LOS and AD. Finally, although this study was done in an LMIC community hospital context, it highlights many shared lessons and barriers to improving care in community settings in both HIC and LMIC. Future studies will need to focus on RSA implementation, accurate outcome measurement including adverse events, and the utilization of diagnostic tools such as blood cultures and serum biomarkers.

The utilization of the REDCap web application was hosted by the University of Cincinnati Center for Clinical and Translational Science and Training supported by grant 2UL1TR001425-05A Early-Onset Sepsis Risk Calculators in Kenya Hospital.