Diagnosis Code and Health Care Utilization Patterns Associated With Diagnostic Uncertainty

This case-control study was conducted at the Cincinnati Children’s Hospital Liberty Campus, a satellite campus with an emergency department (ED) and 42-bed inpatient unit. The inpatient unit has multiple primary inpatient teams in addition to PHM, including adolescent medicine, pulmonology, pediatric surgery, and oncology teams. This campus has onsite operating rooms, laboratory, imaging, and pharmacy services, but does not have an ICU. Not all subspecialty consult services are available on site but all are accessible via telemedicine. The PHM team at the satellite campus includes hospital medicine attending physicians and fellows, advanced practice provides, as well as pediatric and family medicine residents. All study patients were admitted to the PHM service between November 1, 2017, and June 15, 2019. This study was reviewed and approved by our institutional review board.

Lack of communication around diagnostic uncertainty was identified as a key driver in safety events at our institution. Subsequent quality improvement work led to the development of an uncertain diagnosis (UD) label in the EHR.¹³  The UD label was built on established safety infrastructure, and the resulting communication processes are designed to improve shared recognition of patients with diagnostic uncertainty among the health care team. The UD label was assigned to 5% to 10% of patients on the hospital medicine team during our study period. The operational definition for the UD label as used by clinicians during this study period was “you wouldn’t be surprised if the patient had a different diagnosis that required a change in management.” As previously mentioned, there is no widely used operational definition for diagnostic uncertainty in clinical practice. This definition was developed and refined as part of a quality improvement project using iterative feedback from clinicians using the UD labeling process. For clinicians using this label in real time,¹³  it was important to focus on diagnostic uncertainty in which subsequent changes in management may be needed during their admission, rather than instances of diagnostic uncertainty with a clear evaluation and management plan. For example, the clinician assigning the UD label would not be surprised if a change in management, such as obtaining additional diagnostic testing, was needed because of the patient developing new symptoms or not responding as expected to the initial treatment plan. Conversely, the UD label was not intended for febrile neonates younger than 28 days old who get a very standardized diagnostic evaluation and empirical antimicrobial therapy, despite not knowing the etiology of fever at time of admission. Cases were defined as patients who clinicians labeled as having a UD at any time during their hospitalization and who were admitted to the PHM service during the study period. All cases were identified using a structured electronic database query.

All patients who were admitted to the PHM service at the satellite campus during the study period who did not receive the UD label were included in a dataset of potential control patients. A matching algorithm selected up to 4 controls for each UD case.¹⁴  Once an individual patient was selected as a control, they were removed from the pool of potential control patients. Our match criteria were age strata, biological sex, and date of admission (within 30 days); we selected these criteria to adjust for differences in diagnostic considerations based on patient characteristics and seasonality. Age strata were adapted from the National Institute of Child Health and Human Development and delineated the following categories: neonates/infant 0 to 12 months, toddlers 13 months to 2 years, early childhood 2 to 5 years, middle childhood 6 to 11 years, and adolescence/young adulthood 12 to 21 years.¹⁵ 

Several study team members, including 2 PHM attending physicians and 2 pediatric residents (T.L.M., C.S., D.W., P.O.), confirmed through chart review that the UD cases and matched controls met the inclusion criteria. A small number of patients were excluded from the study because they were not admitted to the PHM service or because they had missing or incomplete diagnostic billing code data.

Diagnosis code outcomes examined the primary, or first-listed, International Classification of Diseases, 10th revision (ICD-10), diagnosis codes on admission and discharge for cases and controls. For our dataset, the primary ICD-10 diagnosis code was extracted from EHR data and was determined by the billing provider for that date of service, typically the PHM attending physician. We did not use claims data for ICD-10 diagnosis codes because these are sometimes modified for billing purposes, adding additional variability in coding practices. Admission was defined as the first billing date on the PHM service and discharge was the last billing date on the PHM service. These typically coincided with dates of hospital admission and discharge, but occasionally patients were transferred to or from other inpatient services. This approach was selected because the UD label was developed on the PHM service and was not consistently used by other inpatient services within our study period. We assessed differences in the type of diagnosis code used (ie, disease or nondisease based) on admission as well as change in primary diagnosis code from admission to discharge from the hospital medicine service.

Although ICD-10 codes are divided into major categories of diagnoses (ie, codes K00-K93 are diseases of the gastrointestinal tract), there is not a standardized approach to categorizing ICD-10 diagnosis codes as disease or nondisease based. To address this, we drafted operational definitions for disease- and nondisease-based codes. Next, we created a dataset of all the unique ICD-10 codes for our cohort of patients. Study team members, including 3 PHM attending physicians (T.L.M., P.A.H., P.W.B.) and 1 medical student (C.M.), then categorized a random test set of 100 ICD-10 codes. Based on this test set, we refined our operational definitions (Table 1) and added a third distinct set for health status or risk factors diagnosis codes (eg, gastrostomy tube present). The remaining ICD codes were then independently categorized by 2 of these study team members. In cases where the 2 reviewers did not agree, the team met as a group to adjudicate and assign a final categorization to the ICD-10 diagnosis codes. The full dataset of ICD-10 diagnosis codes was then reviewed for any unintentional variation in coding practices (ie, similar codes with discrepant categorizations), with identified discrepancies subsequently reviewed and resolved by the team to generate our final categorizations.

Health care utilization outcomes included hospital LOS, transfer to our main hospital campus, specialist consultations, and 30-day health care reutilization. We also assessed measures that may indicate higher clinical acuity such as rapid response team (RRT) activation and escalation of care to an ICU. For all health care utilization outcomes, we included resource utilization at both our satellite campus and main campus for those requiring transfer. Hospital LOS was defined as admission to the hospital until discharge to home, an external health care facility, or death. The 30-day health care reutilization included both subsequent visits to the emergency department and readmissions to the hospital on any inpatient service within 30 days of discharge. Both hospital LOS and 30-day health care reutilization data were obtained using a structured electronic database query. However, only emergency department visits and readmissions to our health care system were captured by our query. All other health care utilization outcomes were obtained using structured chart review. A specialty consultation was only counted if a physician consult note appeared in the medical record. We also excluded consultations by services such as physical therapy or lactation because these are typically nondiagnostic consultations and are likely not associated with diagnostic uncertainty. Data for cases and controls were entered into a secure Research Electronic Data Capture database.¹⁶ 

Descriptive statistics for demographic and hospital variables were produced using means and standard deviations for continuous variables and frequencies and percentages for categorical variables. Comparisons between groups were done using t tests and χ² tests. To assess for differences in health care utilization outcomes for UD cases versus matched controls we used χ² tests. We used conditional logistic regression to determine the odds of UD cases having a nondisease-based ICD-10 diagnosis code on admission compared with the controls and the odds of UD cases having a change in ICD-10 diagnosis code from admission to discharge. Our models adjusted for both biological sex and age. Additionally, for change in diagnosis from admission to discharge, we performed 1 analysis that included any change in the ICD-10 diagnosis code, including potentially minor changes, and a second analysis that included only changes in the ICD-10 major diagnostic category (Fig 1). For the purposes of analysis, health status and risk factor codes were included with disease-based codes because both categories were thought to imply low levels of diagnostic uncertainty. We observed that some patients with short lengths of stay only had 1 date of service with diagnosis code data. For our analysis, these patients were still included and were considered to have no change in ICD-10 code from admission to discharge. Analyses were performed using SAS software, 2021 (SAS Institute Inc, Cary, NC, USA).

Our final cohort contained 240 UD cases and 911 matched control patients (Table 2). Following validation of inclusion criteria via chart review and assessing for completeness of billing diagnosis code data, 3 UD case patients and 33 matched control patients were excluded. The diagnosis code analyses found significant differences in ICD-10 code use in UD cases versus controls. The odds of UD cases receiving a nondisease-based diagnosis code were 8.0 times (95% confidence interval [CI], 5.7–11.2) higher compared with matched controls. Our logistic regression model also found that the odds of receiving a nondisease-based diagnosis code decreased as patient age increased (ie, younger patients were more likely than older patients to receive a nondisease-based code). Biological sex was not associated with differences in type of ICD-10 diagnosis code a patient received.

The odds of UD cases receiving any change, including minor changes, in their primary ICD-10 diagnosis code from admission to discharge was 2.5 times (95% CI, 1.9–3.4) higher than matched controls. The odds of UD cases having a change in their ICD-10 major diagnostic category from admission to discharge was 3.5 times (95% CI, 2.5–5.0) higher than for controls. Biological sex and age were not associated with changes in ICD-10 diagnosis codes in either analysis.

Health care utilization outcomes are summarized in Table 3. Compared with the matched controls, the UD cases had a longer average hospital LOS and increased transfers rates to our main campus, specialty consultations, and 30-day readmissions. The most common reason for transferring UD case patients to our main hospital campus was better access to specialty consult services. There were no significant differences in RRT utilization or escalation to intensive care between UD cases and matched controls.

Our study, which expands on prior descriptive work, found meaningfully different patterns in both ICD-10 diagnosis code use and health care utilization for patients with uncertain diagnoses compared with matched controls.¹⁷  Advantages of our study design include the prospective labeling process for the UD case patients, rather than relying on proxy measures or retrospective identification, as well as a matched control group. Our team also took a stepwise and nuanced approach to identifying ICD-10 codes that are likely to signal diagnostic uncertainty. ICD-10 diagnostic codes are grouped into broad categories, including a category for “Symptoms, signs, and abnormal clinical and laboratory findings, not elsewhere classified.” However, nonspecific codes exist across all major ICD-10 categories. For example, eosinophilia, a nonspecific laboratory abnormality, is classified with “Diseases of blood and blood-forming organs.” We manually categorized all the ICD-10 diagnosis codes used in our patient cohort as nondisease-based, disease-based or health status or risk factor codes. This process enabled us to categorize ICD-10 diagnosis codes more accurately as definitive diagnoses or nonspecific signs or symptoms.

Notably, our findings contradict 1 previous study that concluded that ICD-9 and ICD-10 coding mechanisms were not successful in capturing diagnostic uncertainty.¹⁸  The study by Bhise et al evaluated a cohort (n = 389) of adult ambulatory care encounters, derived from a previous study evaluating for diagnostic errors, and used structured retrospective chart review of the documentation from a clinic visit to identify a subset of patient encounters with diagnostic uncertainty (n = 156). They observed that 37% of the patients with diagnostic uncertainty had ICD-9 codes in the “Signs, symptoms, and ill-defined conditions” major diagnostic category and an additional 23% had “other symptomatic codes.” However, because many of these patients still had disease specific ICD-9 codes, the authors ultimately concluded that ICD diagnosis codes did not reliably reflect uncertainty. However, this study was limited by its small size, retrospective approach to identifying diagnostic uncertainty, and lack of a comparison group.

Patients identified as having uncertain diagnoses had significantly longer hospital LOS and higher rates of specialty consultation, transfers to main campus, and 30-day readmissions. These increases in health care utilization were consistent with previous studies using proxy measures of uncertainty, such as physician risk tolerance.^7–10  Interestingly, our study did not find greater utilization for our surrogate markers of higher clinical acuity, such as RRT activation and escalation to intensive care. The lack of significant differences in these outcomes suggests that diagnostic uncertainty and severity of illness are likely separate constructs with potentially quite different mitigation strategies, highlighting the importance of evaluating diagnostic uncertainty as its own entity. For this study, we did not evaluate if increased resource utilization affected diagnostic outcomes or resulted in resolution of uncertainty. Future work assessing resource utilization in the context of uncertainty could focus on ensuring that resources used add value to patient care.

This study demonstrates that it may be possible to identify patients with diagnostic uncertainty using structured data readily available in the EHR. There is likely not 1 distinguishing feature that reliably predicts the presence of diagnostic uncertainty because not all patients with diagnostic uncertainty will have nondisease-based codes or multiple specialty consults. Additionally, some patient populations with established diagnoses (eg, children with medical complexity or chronic conditions) will display some of these features, such as multiple consults to assist with inpatient management or longer than average LOS. However, this study represents an important first step in identifying variables that could be used to build EHR-based tools to assist in the detection of diagnostic uncertainty in clinical practice. Increasing our ability to detect diagnostic uncertainty is critical for better understanding the impact of diagnostic uncertainty on diagnostic and clinical outcomes and for developing and targeting interventions aimed at mitigating associated negative consequences; this includes supporting earlier detection and discussion of diagnostic uncertainty among the health care team to avoid premature diagnostic closure and potentially prevent diagnostic errors.

Our study has several limitations to consider. This is a single-center study performed at our satellite campus, which does have some limitations on in-person consultation for some subspecialty services and could have affected our health utilization outcomes. Additionally, our PHM service uses a unique process to prospectively label hospitalized children with diagnostic uncertainty. This process is dependent on the health care team recognizing that there is uncertainty with a patient’s diagnosis and following the appropriate situation awareness labeling process. Therefore, the UD label likely does not capture all patients with diagnostic uncertainty, which could have resulted in miscategorization in our control cohort. However, because diagnostic uncertainty was likely underidentified, this biased our findings toward the null. Additionally, it is possible that use of the UD label affected how clinicians approached the clinical care of these patients (ie, additional specialty consults or prolonged inpatient observation) and could have influenced how providers selected ICD-10 diagnosis codes used for billing purposes.

Several studies have shown that the accuracy of ICD-10 billing codes in reflecting patient diagnoses are limited.^19,20  Furthermore, our approach to defining disease versus nondisease-based diagnosis codes would be time intensive to replicate across the entire spectrum of ICD-10 codes. Nonetheless, diagnosis codes are readily available data that are often used to identify target populations for clinical and health services interventions and research.

This study demonstrates that patients identified as having uncertain diagnoses have meaningfully different patterns of diagnosis codes and health care utilization compared with matched controls. Diagnostic uncertainty remains difficult to detect in clinical practice, but this study suggests how we might use structured data to begin building EHR-based tools to detect diagnostic uncertainty and ultimately increase our understanding of its impact on diagnostic outcomes and provide opportunities for mitigation.