Support for the Use of a New Cutoff to Define a Positive Urine Culture in Young Children

Between June 2019 and May 2020, we enrolled consecutive children presenting to the emergency department at the Children’s Hospital of Pittsburgh who had remaining urine after all clinical tests had been completed. The study was approved by the Institutional Review Board of the University of Pittsburgh. Children were included if they were between 1 month and 2 years and 11 months of age, had fever (documented temperature of ≥38°C in the emergency department or by parental report) within 24 hours of presentation, and had a urine sample collected via catheter to rule out UTI. We excluded neonates because this study is part of a larger study examining biomarkers for pyelonephritis. The diagnosis of pyelonephritis is accomplished by performing a renal scan, which is difficult to perform in neonates.

Children were excluded if they had received systemic antibiotics or corticosteroids within 3 days before enrollment, had other concurrent systemic bacterial infections, were immunodeficient, had a neurogenic bladder, or had major genitourinary abnormalities (eg, spina bifida, dysplastic kidneys, grade IV or IV vesicoureteral reflux). Conventional urine culture was performed at the hospital’s clinical microbiological laboratory using standard microbiological methods and the number of colony forming units (CFU)/mL were reported as <10 000, 10 000 to 49 000, 50 000 to 99 000, and ≥100 000.

An aliquot of residual urine was used for 16S sequencing. Aliquoting generally occurred within 1 hour of collection; however, if delays were anticipated, samples were kept refrigerated. The aliquot was frozen in a cryovial at −80°C without preservatives. Before shipping, we added 70 µLs of urine conditioning buffer (Zymo, D3061-1-8) to every 1 mL of frozen urine sample. Samples were shipped overnight in a cold pack to Pangea Laboratory, Tustin, CA, USA, for 16S sequencing analysis using the PrecisionBIOME NGS Microbial Test.

DNA was extracted from the urine specimen using the ZymoBIOMICS DNA Miniprep Kit according to the manufacturer’s instructions (Zymo Research Corporation, Irvine, CA). The extracted DNA was prepared for microbiome analysis following the PrecisionBIOME workflow, which included library preparation using the Quick-16S NGS library prep kit (V1-3 regions, Zymo Research Corporation, Irvine, CA), sequencing of barcoded amplicons with the MiSeq sequencing platform (Illumina, San Diego, CA), and bioinformatics analysis using a the PrecisionBIOME bioinformatics pipeline capable of producing species-level resolution of bacterial and fungal sequences (data on fungi present and cytokines will be reported separately). Negative controls (transport medium alone and unused swabs) were included. To control for contamination, we also included cell and DNA mock communities as positive controls (ZymoBIOMICS microbial community standard, catalog Nos. D6300 and D6305; Zymo Research Corp). Potential sequencing errors and chimeric sequences were removed with the DADA2 pipeline.

We used Uclust to perform taxonomic classifications using a PrecisionBIOME custom database. We computed phylotypes as percent proportions based on the total number of sequences in each sample. Species level resolution of this sequencing approach was previously confirmed by shotgun sequencing.⁷  We excluded samples with <1000 sequences per sample.

In a previous study conducted between 2011 and 2017,⁶  we found a high concordance rate between conventional urine culture and 16S sequencing in the identification of bacteria in the urine. This established, we felt it was now appropriate in the context of this exploratory study, to substitute 16S sequencing instead of urine culture in diagnosis of UTI (the target condition). In clinical practice, addition to presence of bacteriuria on culture, we require elevation of marker(s) of inflammation to diagnose UTI.³  Accordingly, to diagnose UTI in the context of this proof-of-concept study, in addition to requiring that ≥80% of sequences belonged to a single taxon (ie, relative abundance of any taxa ≥80%), we also required elevation of urinary markers of inflammation. In constructing our reference standard, we chose a cutoff of 80% (for our primary analysis) because we felt that as clinicians we would be compelled to diagnose UTI in a febrile child with urinary markers of inflammation whose urine had this level of bacteriuria. The remainder of the children were categorized as having “no UTI”. As a sensitivity analysis, we also examined results using relative abundance cutoffs of 50% and 90%.

Each child had a microscopic urinalysis in which white blood cells (WBCs) were enumerated (per cubic millimeter or per high powered field) and a dipstick test in which the leukocyte esterase test was reported semiquantitatively (none, trace, 1+, 2+, 3+). Because the sensitivity of both the WBC and leukocyte esterase tests are generally low,⁸  we also measured, using methods used in our previous studies,⁹  neutrophil gelatinase-associated lipocalin (NGAL) in an aliquot of residual urine. NGAL is an inflammatory marker released by neutrophils or by intercalated cells in the kidney in response to UTI.^9,10  Although use of urinary NGAL for the diagnosis of UTI is a relatively recent development, given the robust evidence supporting its use to diagnose UTI,¹⁰  we decided, a priori, to categorize a child as having evidence of inflammation if any of the following was present: ≥10 WBC/mm³, ≥5 WBC per high power field (Hpf), ≥trace leukocyte esterase, or NGAL level above 39.9 ng/mL.¹⁰  We conducted a sensitivity analysis examining results had we not considered NGAL as a marker of inflammation.

We defined the results of conventional culture as positive if the urine culture exhibited growth of at least 1 organism with a count of at least 10 000 CFU/mL and at least 1 of the urinary markers of inflammation was elevated. Other cutoffs evaluated were 50 000 CFU/mL and 100 000 CFU/mL.

For the primary analysis, we calculated sensitivity and specificity of urine culture at detecting UTI (using a relative abundance cutoff of 90% in a child with urinary markers of inflammation), along with corresponding 95% Wald confidence intervals. We then repeated the analyses using different definitions for UTI (ie, using relative abundance cutoffs of 50% and 80% instead of 90%). We summarized and analyzed the data using SAS version 9.4 (SAS Institute Inc).

Overall, 341 children were included in the study (Fig 1). Table 1 describes the demographic characteristics of these children. Most children enrolled were female (74%), white (67%), and most (64%) had a documented temperature of 39°C or more. The mean age of children at diagnosis was 12.5 months and the mean temperature at presentation was 39.3°C. In children with urinary markers of inflammation but without at least 1 organism with a count of 10 000 CFU/mL or more on culture, the median relative abundance of the predominant identifiable organism was 15% (interquartile range: 7%–32%).

Using a relative abundance cutoff of 80%, 46 children in our sample had a UTI. Of these, 41 (89%) had E. coli as the predominant pathogen (Fig 2). When using a cutoff of ≥10 000 to define a positive urine culture, among 46 children with UTI, 45 were correctly identified by conventional urine culture (sensitivity of 98%, confidence interval [CI]: 93% to 100%, Table 2). The 1 child that was missed was a 4-month-old with 3+ leukocyte esterase on dipstick, a WBC count of 73 per Hpf, an elevated NGAL of 319, in whom >99% sequences identified by 16S sequencing mapped to Klebsiella species. On culture, this child had <10 000 CFU/mL of Gram-negative rods and <10 000 CFUs of gram positive cocci; neither organism was further identified because of the low colony counts. Of the 295 children without UTI, 291 were correctly identified as such by conventional culture (specificity of 99%, CI: 97% to 100%).

Using a cutoff of ≥50 000 CFU/mL decreased the sensitivity of urine culture to 80% (95% CI: 68%–93%, Table 2). Change in cutoff to ≥50 000 had a negligible effect on specificity (ie, specificity remained at 99%, CI: 98%–100%, Table 2). Characteristics of the 8 extra children with UTI that would have been missed if a cutoff of 50 000 had been used are listed in Table 3; all children with missed UTIs had organisms currently considered as uropathogens, and, by definition, all were symptomatic and had elevated urinary markers of inflammation.

Use of a cutoff of 100 000 CFU/mL would have reduced sensitivity to 70% (95% CI: 55%–84%, Table 2). Changes in the definition of the reference standard had little effect on estimates of accuracy (Table 2).

As can be seen in Supplemental Table 4, sensitivity and specificity estimates would have been similar had we not included NGAL as a marker of inflammation.

Previous studies have attempted to understand urine culture results by comparing colony counts in extreme states (ie, pyelonephritis versus asymptomatic)¹  or by comparing characteristics of subjects above and below certain colony count cutoffs.²  Such study designs, although necessary during the early exploratory phases of research on diagnostic testing, cannot provide a true assessment of the accuracy of a test in clinical practice, nor can they be used to identify cutoffs that optimize sensitivity and specificity.⁴  Here, using 16S sequencing as the reference standard for the diagnosis of UTI, we were able to assess the diagnostic accuracy of urine culture at various colony count cutoffs. We found that, in febrile children under 3 years of age undergoing bladder catheterization to rule out UTI, using a colony count of 10 000 CFU/mL would have led to fewer missed cases of UTI than using a colony count of 50 000 CFU/mL (missed 20% of cases) or 100 000 CFU/mL (missed 30% of cases). Per study design, all children with missed UTI had elevated urinary markers of inflammation and were febrile. Accordingly, our data suggest that a cutoff of 10 000 CFU/mL best differentiates young children with and without a bona fide UTI.

There are many reasons why 16S sequencing, especially when paired with urinary markers of inflammation, is a suitable reference standard for the diagnosis of UTI. First, unlike urine culture, which is optimized to detect E. coli,¹¹  16S sequencing analysis provides a largely unbiased assessment of bacteria present in the urine. Indeed, a growing body of evidence^12,13  suggests that the results of 16S sequencing analysis align well with results of Expanded Quantitative Urine Culture – a more sensitive variation of the urine culture that uses additional culture media, larger volumes of the urine used to inoculate culture plates, longer incubation times, and a variety of atmospheric conditions.^14,15  However, although the data on the higher sensitivity of 16S sequencing compared with urine culture seems undisputed, its increased sensitivity could theoretically come at the cost of lower specificity, especially because of the required amplification step. Thus, one might imagine a scenario in which many nonclinically relevant organisms could be identified by 16S sequencing, leading to a large number of inappropriately diagnosed UTIs. In our study, however, use of 16S sequencing (paired with markers of inflammation) uncovered only 1 child in which UTI was missed. This suggests that 16S sequencing analysis (at a relative abundance threshold of 80%) paired with inflammatory markers of inflammation was a suitable reference standard to use for this study.

When we started this study, we were concerned about using a relative measure of abundance as our reference standard. However, our results (in this study and in the previous one) show that, for the diagnosis of UTI, relative abundance from 16S sequencing and absolute culture results were surprisingly similar. We hypothesize that for a pathogen like E. coli to account for 80% or more of all sample reads on 16S sequencing, the number of copies of its gene have to exceed the combined counts¹⁶  of all the other bacteria normally present as part of the urobiome by fourfold. Thus, the 80% relative abundance threshold will only be met when the absolute abundance of a uropathogen is quite high.

Our results support the continued use of urine culture (at a cutoff of 10 000 CFU/mL) in the population studied. However, because of continuing technological advances in the accuracy, speed, and cost of 16S sequencing, it may soon become available in clinical centers. If this occurs, identification of organisms could be accomplished in a matter of hours instead of days. Thus, it is important that comparative studies are conducted to establish the pros and cons of using this technology in the clinical setting.

Several limitations of this study need to be considered. We used a definition of UTI that required the presence of symptoms, elevated urinary markers of inflammation, and a high relative abundance of organisms on 16S sequencing analysis. Using a less stringent definition (eg, not requiring urinary markers of inflammation) might have led to a higher rate of missed UTIs, but we would have been less certain that all these UTIs represented bona fide UTIs. Our choice of 80% for the relative abundance threshold required to define UTI, although based on data from our previous study, was nevertheless, to an extent, arbitrary. We acknowledge that less abundant organisms may also be capable of causing significant disease. Ultimately, as borne out in the sensitivity analysis, the cutoff chosen for the primary analysis had little influence on our conclusions. This is likely because, in most cases, samples were either dominated by sequences from a single known uropathogen (Fig 2) or had very low abundance of a variety of organisms that had, in earlier studies, been detected in urine samples from asymptomatic individuals.¹⁴  Our findings only directly apply to children who have bladder catheterization performed to rule out UTI; a higher cutoff for urine culture may be needed if more contamination, for example related to the method of collection, is expected. We did not use preservatives before freezing the samples; however, samples were never left at room temperature, and our results do not suggest bias because of the approach we used. Our clinical laboratory did not report exact colony counts for the conventional urine cultures; this might have been helpful in pinpointing the colony count threshold less than 50 000 CFU/mL that best optimized the sensitivity and specificity of the urine culture. Most UTIs in our sample were caused by E. coli. Larger studies are needed to further examine the accuracy of colony count thresholds for less common uropathogens. Finally, 16S sequencing, albeit less biased than urine culture, can itself be subject to certain biases, especially when the biomass in samples is relatively low¹⁷ ; however, these biases are unlikely to change the pattern observed from one in which many organisms are present at relatively low abundances to one in which the sample is dominated by 1 organism. Strengths of the study included consecutive enrollment of symptomatic children with suspected UTI, use of a reference standard based on prior data, performance of the index test and reference standard on all included children, the use of a priori clinically based definitions, use of positive and negative controls during extraction and sequencing, and the use of a validated sequencing approach.

Inevitably, lowering the cutoff for the diagnosis of UTI, from 50 000 CFU/mL to 10 000 CFU/mL, will increase the number of children falsely labeled as having UTI. Nevertheless, this is appropriate because (1) the number children falsely labeled as having a UTI will be less than the number of children with missed UTIs that will be uncovered (Table 2), and (2) the negative consequences of missing a febrile UTI usually outweigh that of prescribing an additional course of antimicrobial agents.

In conclusion, using 16S sequencing as the reference standard, we found preliminary empirical evidence to support use of a cutoff of 10 000 CFU/mL for the diagnosis of UTI in young febrile children.

We thank Dr Wendy Ullmer for her help with the 16S rRNA gene amplicon sequencing analysis.

CFU

colony forming unit

Hpf

high power field

NGAL

neutrophil gelatinase-associated lipocalin

rRNA

ribosomal ribonucleic acid

UTI

urinary tract infection

WBC

white blood cell