Diagnostic Accuracy of the Urinalysis for Urinary Tract Infection in Infants <3 Months of Age

We would like to express our thanks to Dr. Iqbal and colleagues for their interest in our research and for their questions about our methodology. The primary intent of our investigation was to describe the sensitivity of the urinalysis in a population of young infants with incontrovertible infection. Because thresholds for a "positive" UA vary across studies, the high sensitivity found in our study may have been partially attributable to a low threshold for positivity. For this reason, we opted to calculate specificity estimates on similarly-aged populations of infants being worked up for UTI but who had negative urine cultures, with the understanding that if we had used thresholds that were overly sensitive, specificity would suffer. We agree that sampling infants with negative urine cultures from all of the participating sites would have made our specificity estimates more generalizable, but the remaining sites were recruited because of their existing databases of young bacteremic young infants which did not contain subjects without bacterial infections. We are confident that the specificity estimates reported in our study our similar to estimates from prior studies.

There was variability in the reporting of UA results across (and often even within) the participating institutions, with some reports containing raw values for WBC (e.g. "17 wbc/hpf") and others containing categories (e.g. "15-19 wbc/hpf or "10-25 wbc/hpf"). In order to create at least some standardization of the results, we created the categories a priori as described in our methods section, using the middle values of the reported range to guide placement into the appropriate category. We hope that this transparent and logical approach, as well as our use of various thresholds for sensitivity and specificity calculations (e.g. >3 wbc/hpf and >10 wbc/hpf), mitigate concerns about measurement error or bias.

We agree that ROC curves are useful for the evaluation of diagnostic tests. However, as we state in our methods section: "Given that the sensitivity and specificity of the UA were calculated in separate samples of patients, likelihood ratios and predictive values would be misleading and were not calculated, and ROC curves were not created." Had our patients included all possible urine culture results (not just infants with bacteremic UTI and infants with negative urine cultures), an ROC curve would have been appropriate.

Finally, we thank the authors for mentioning the earlier study by Roman et al from the study's central site, Santa Clara Valley Medical Center. We agree that clinicians should be familiar with the entity of bacteremic UTI, but as we suggest in that paper, it is not clear that infants with bacteremic UTI are clinically sicker than infants with non-bacteremic UTI.

Sincerely,
Alan R. Schroeder, MD

Conflict of Interest:

None declared

Dear Editor,

We read the article by Schroeder, et al published in your June 2015 edition titled "Diagnostic accuracy of the urinalysis for urinary tract infection in infants <3 months of age" 1. We hope the authors will help readers understand some of our observations regarding the study methodology.

Case controls studies are usually retrospective (as this study was) and choice of control include the following: patients in the same hospital but with unrelated disease, patients matched one to one match to control for key prognostic factors such as age and sex, and a random sample of the population from which the cases came. In case control studies, the sample size of the cases is usually the same as those of the control to give the greatest statistical power (the best possibility of detecting a true effect) this was not the case here. When the number of available cases are limited, the power could be increased by choosing more controls2. Why was it beneficial to use a single site control when the study was a multi-site study? Was it logistically difficult to do a site matched case control study be as this could have added additional power to the study?

To standardize measures across the sites, authors grouped urinalysis findings in terms of white blood cells per high-powered field (WBC/hpf) into groups of 0-3, 4-10, 11-20, 21-50, and > 50 WBC/hpf. Infants with WBC/hpf that did not fall into these groups were assigned using the middle values of the WBC count leading to the possibility of measurement bias? Could authors explain how measurement bias was accounted for in the analysis?

Receiver operator curves (ROC) are used for the interpretation of diagnostic tests, since there is usually a trade-off between sensitivity and specificity. The area under the ROC curve (AUC) is used as an indicator of diagnostic performance. The AUC is equal to the probability that the observer will correctly identify the positive case when presented with a randomly chosen pair of cases in which one case is positive and one case is negative.3 A previous study using ROC curve analysis demonstrated that the CRP, ESR, and standard UA were powerful but imperfect tools with which to discriminate for UTI in potentially infected neonates4. We wonder whether authors considered plotting ROC for the different sensitivities and specificities calculated for the different UA groups described above.

A review of recent literature described the prevalence of bacteremia as 4.1% (95% confidence interval 3.1%-5.3%) for all Urinary Tract Infection (UTI), and 8% (95% confidence interval 6.1%-10.2%) for UTIs in which blood culture was obtained 5. Even though the low prevalence implies a low burden of disease in infants less than 3 months of age; the sequale of serious bacterial illnesses makes the UTI combined with bacteremia a disease one clinicians should still be conversant with.

References

1. Schroeder AR, Chang PW, Shen MW, Biondi EA, Greenhow TL. Diagnostic accuracy of the urinalysis for urinary tract infection in infants <3 months of age. Pediatrics. 2015;135(6):965-971.
2. Janet Peacock PP. Oxford Handbook of Medical Statistics Oxford: Oxford University Press 2010.
3. Eng J. Receiver operating characteristic analysis: a primer. Acad Radiol. 2005;12(7):909-916.
4. Lin DS, Huang SH, Lin CC, Tung YC, Huang TT, Chiu NC, et al. Urinary tract infection in febrile infants younger than eight weeks of Age. Pediatrics. 2000;105(2):E20.
5. Roman HK, Chang PW, Schroeder AR. Diagnosis and management of bacteremic urinary tract infection in infants. Hosp Pediatr. 2015;5(1):1- 8.

Conflict of Interest:

None declared

We appreciate the thoughtful comments by Dr. Tsujimoto and colleagues, who are concerned that our sensitivity estimates may mislead practitioners to assume that a negative UA rules out bacteremic UTI (or UTI in general). The intent of our study was not to assess whether a UA rules in or rules out bacteremic UTI, but rather to analyze the UA in a population of infants with definitive infection. While Tsujimoto et al are concerned that the UA sensitivity is overestimated in our study, we are similarly concerned that all prior studies have underestimated the UA sensitivity by using the urine culture as a gold standard when in fact many "positive" urine cultures reflect contamination or asymptomatic bacteruria. Whether or not bacteremic UTI represents UTI in general or represents a more severe form of UTI is debatable. Most investigations comparing infants with bacteremic vs non-bacteremic UTI have demonstrated few if any clinical differences upon presentation, and even these few differences may be explained by inclusion of infants with asymptomatic bacteruria or contamination in the non-bacteremic UTI groups. However, few would disagree that bacteremic UTI - isolation of the same pathogenic organism from the blood and urine - represents true infection. Because of the entities of asymptomatic bacteruria and contamination, the lack of a reliable gold standard for the diagnosis of UTI will be a perpetual problem,1 and we believe that bacteremic UTI offers a unique way to assess the performance of the UA in an infant who has a true infection.

Tsujimoto et al are also concerned that a positive UA might trigger practitioners to obtain a blood culture, and suggest that this might create "review bias". We agree that this bias, which is also called "verification bias" or "confirmation bias", tends to falsely elevate sensitivity and falsely lower specificity, because subjects with negative index tests are under-represented.2 However, we are confident that this potential bias is not a limitation of our study. Infants < 1 months of age routinely undergo a full workup for serious bacterial infection independent of the preliminary results, and the sensitivity estimates were the same in this age group as they were in the 1-3 month age group.

How our study findings will impact practitioners' interpretation of the combined findings of a negative UA and a positive urine culture, or their willingness to rely on the UA as a screening test, may depend in part on how practitioners weigh the risks of undertreatment vs overtreatment of UTI. Traditional practice patterns have erred towards avoidance of undertreatment by interpreting all positive urine cultures in young infants as true UTI irrespective of the UA, and in so doing have led to long courses of antibiotics, hospitalizations, and invasive urinary imaging that may have been unnecessary. We believe that our new findings should force us to rethink this practice, and that, as stated in the accompanying commentary by Ken Roberts, the absence of inflammation (pyuria and/or leukocyte esterase)on the UA "should create great doubt about the presence of a UTI."1

1. Roberts KB. The Diagnosis of UTI: Liquid Gold and the Problem of Gold Standards. Pediatrics 2015;135(6):1126-7. 2. Newman TB, Kohn MA. Evidence-Based Diagnosis: Chapter 5. Cambridge Medicine, 2009.

Conflict of Interest:

None declared

We read an article by Alan R. Schroeder with great interest, and appreciate the authors' efforts to assess diagnostic accuracy of urinalysis for urinary tract infection (UTI) with blood stream infection (1). However, we would like to point out three concerns.

First, we are concerned that the excellent sensitivity may lead to confusion and misinterpretation for a clinician as if a negative urinalysis would perfectly rule out bacteremic UTI. The authors did not prespecify the cut-off points of urinalysis. The authors only reported the sensitivity of urinalysis for bacteremic UTI. Although they calculated specificity of urinalysis for UTI in general with or without bacteremia, it is a comparison of values for different conditions. It is not simply explained by spectrum bias. There is a serious concern that clinicians who are not familiar with observational studies for diagnostic test accuracy may misinterpret the results.

Secondly, heterogeneity of the spectrum of UTI should be considered. They reported excellent sensitivity of urinalysis for bacteremic UTI. However, UTI with positive urine and blood culture is not representative of all cases of UTI. The heterogeneity of UTI (e.g. lower UTI and obstructive UTI) may need different treatments and have different prognosis.

Lastly, bias may be introduced if a positive urinalysis led to the clinicians obtaining a blood culture. (e.g. review bias) (2).

Hence, we are concerned about the risk of misinterpretation of the reported diagnostic accuracy. In addition, introduced bias might affect the reported test accuracy of the study.

(1) Schroeder AR, Chang PW, Shen MW, Biondi EA, Greenhow TL. Diagnostic Accuracy of the Urinalysis for Urinary Tract Infection in Infants , 3 Months of Age. 2015;135(6).

(2) Whiting P. Sources of Variation and Bias in Studies of Diagnostic Accuracy. Ann Intern Med. 2004 Feb 3;140(3):189.

Conflict of Interest:

None declared