Promoting Choosing Wisely Thyroid Function Test Guidelines in a Large Pediatric Hospital System Free

This QI initiative was conducted over a 44-month period between June 2019 and February 2023 at a large free-standing children’s hospital with 2 hospital locations, 400+ beds, and a primary care network of 37 primary care locations across 15 counties. The hospital system used computerized physician order entry via the electronic health record (EHR) (Epic Systems Corporation). All inpatient and outpatient sites were included in the project. All TSH, fT4, and reflex fT4 orders were included except for those placed by endocrinology providers. Ordering modes targeted for interventions included outpatient and inpatient order sets and system preference lists. Order sets consist of preconfigured sets of orders (eg, laboratory testing, imaging, medication) that are commonly ordered together and can be configured based on certain diagnoses. Preference lists are collections of individual orders that allow users to easily look up and select entries they most commonly use. Preference lists can be built at a system level to simplify (or restrict) options that are available to a group of users or they can be created by individual users who wish to have their own personalized list of frequently used orders. Panels can be created to group orders and/or medications for quick ordering and can also include clinical decision support.

This improvement project was initiated by the Laboratory Formulary Committee, a multidisciplinary group of physicians, nurses, and laboratory staff whose purpose is to promote best practices within laboratory medicine by ensuring high-quality, evidence-based, and cost-effective testing. In addition to several committee members, additional subject matter expert physicians, data analysts, nurses, and QI specialists were involved in this QI project. This QI project does not meet the definition of human subject research per our institutional review board and therefore does not fall under its purview.

Automated data reports were generated monthly from the EHR and included the number of TSH/fT4 co-orders, TSH, fT4, and reflex fT4 tests ordered using test definitions described in Table 2. The automated data reports also included the patient's name, medical record number, ordering mode (inpatient or outpatient), ordering provider, ordering department as well as the order set or preference list used to place the order. Percentages of TSH/fT4 co-orders and percentages of reflex fT4 test usage were calculated monthly between June 2019 and February 2023 using the formulas defined in Table 2. Two primary outcomes were measured: (1) the change in percentage of TSH/fT4 co-order usage and (2) the change in percentage of reflex fT4 test usage (Table 2). Automated monthly run charts were created using Qlik Sense Analytics Software (Qlik Technologies Inc., King of Prussia, PA) and statistical process control charts, namely p-charts, were created using Excel Macros from Cincinnati Children’s Hospital. Charts were used to track and display the 2 key outcome measures. Two secondary outcomes were measured: (1) yearly and cumulative direct cost savings as a result of performing fewer fT4 tests and (2) the fT4 utilization rate as measured by the ratio of fT4 to TSH tests performed (Table 2). The number of fT4 and TSH tests performed were determined by searching the EHR and Strata²⁴  for CPT codes 84439 (fT4) and 84443 (TSH). fT4 costs were used from the hospitals’ cost accounting system from 2022. TSH tests performed as part of the Ohio newborn state screen were excluded.

After a review of the literature and best practice recommendations for thyroid function test use,^8–16  ordering practices of nonendocrinology providers were analyzed by determining the percentage of TSH/fT4 co-orders, TSH standalone orders, and fT4 standalone orders over a 9-month period (June 1, 2019–January 31, 2020) using the calculations described in Table 2 under baseline analysis. Potential inappropriate fT4 testing was defined as co-ordering TSH/fT4 when TSH was within the laboratory’s reference range and was calculated using the formula in Table 2 under baseline analysis. As a balancing measure, during the baseline period, the percent of abnormal fT4 results in the setting of a normal TSH result from TSH/fT4 co-orders was calculated (Table 2).

The Model for Improvement Framework²⁵  was used and consisted of a project aim, tracking metrics monthly pre- and postinterventions, identifying change concepts/ideas, and conducting Plan Do Study Act cycles. A Key Driver Diagram was created to track the project work focusing on key points related to possible overutilization of fT4 testing (Fig 1). Educating providers on Choosing Wisely guidelines pertaining to thyroid function testing and EHR support were identified as key drivers. Pareto Charts were generated from pre- and postintervention data to identify and prioritize those changes that would be of greatest benefit for improvements in appropriate TSH and fT4 test utilization.

Baseline analysis over a 9-month period revealed 94% of TSH/fT4 co-orders had normal TSH results. The first intervention identified was to implement a reflex fT4 test order option where fT4 was automatically performed if TSH was outside the normal reference range. The reflex fT4 test was implemented on February 25, 2020. Downstream effects in the laboratory that resulted from implementation of the reflex fT4 test included: (1) requirements to create and validate a new test to ensure the fT4 test was triggered when TSH was outside the normal reference range; (2) delay in resulting of TSH when the fT4 test was triggered because the fT4 test had to be completed before resulting TSH; and (3) potential for increased work for laboratory personnel to manually retrieve samples if the reflex fT4 test was triggered and samples had been removed from the analyzer. Additionally, an increase in minimum test volume for the reflex fT4 test (1.5 mL blood) versus TSH only (0.5 mL blood) was required.

Concurrent with the implementation of the reflex fT4 test and after review of outpatient order sets, inpatient order sets, and system preference lists with subject matter experts, order entry modifications were made in the EHR and are detailed in Table 3. As seen, the reflex fT4 test replaced individual TSH and fT4 tests or was added to existing inpatient order sets, outpatient order sets, and preference lists. Either TSH or the reflex fT4 test is appropriate in the initial evaluation of a patient with suspected nonneoplastic thyroid disease (primary hypothyroidism and hyperthyroidism).^8,9  Benefits of the reflex fT4 test over the TSH standalone test are avoidance of a second blood draw and faster fT4 results should TSH be abnormal. fT4 as a co-order with TSH is indicated as a follow-up to an abnormal newborn TSH screen as well as in suspected central (secondary or tertiary) hypothyroidism, hyperthyroidism, thyroid hormone resistance, TSH resistance syndrome, medication-induced hypothyroidism, and management of thyroid cancer. fT4 as an isolated test is indicated in assessing the adequacy of thyroid hormone replacement in patients with known central hypothyroidism.^11–16 

All medical staff were informed of the Choosing Wisely recommendation⁸  and new reflex fT4 test 1 week before the reflex fT4 test going live via an SBAR (situation, background, assessment, recommendation) e-mail communication. A subsequent SBAR was distributed to medical staff 14 months after implementation of the reflex fT4 test that reported a >30% reduction of TSH/fT4 co-orders among nonendocrinology providers. The SBAR also communicated that the top departments co-ordering TSH and fT4 were pediatrics and pediatric gastroenterology and reminded providers to order TSH or the reflex fT4 test in the initial evaluation of a patient with suspected nonneoplastic thyroid disease. Because many outpatient providers were still co-ordering TSH and fT4 from preference lists in situations where the reflex fT4 test would be appropriate, the next intervention was the implementation of a clinical decision support thyroid order panel on outpatient system preference lists at the time of electronic order on May 17, 2022 (Fig 2). TSH, fT4, and reflex fT4 were added as panel synonyms so the panel would display if searching for individual thyroid function tests. In addition to the initial Choosing Wisely guidance,⁸  the panel included pediatric-specific outpatient order guidance that states not to routinely measure thyroid function in children with obesity,¹⁰  during acute illness,^13,26–28  and to order both TSH and fT4 after an abnormal newborn screen for primary congenital hypothyroidism^{11,14–16,28}  (Table 1). The reflex fT4 test was prechecked as the recommended test for suspected thyroid disease with the option to uncheck and instead order TSH and/or fT4. The thyroid order panel replaced TSH and fT4 on all outpatient system preference lists except for neonatology, endocrinology, and hematology/oncology where co-ordering was more likely to be appropriate. The rationale for interventions implemented using various ordering modes is detailed in Table 3. All medical staff were informed of the new outpatient thyroid order panel 1 week before it going live via an SBAR e-mail communication. Data were reviewed and minor changes to order sets and preference list content were made when the panel was added.

Baseline analysis of TSH/fT4 co-orders and TSH orders placed by nonendocrinology providers over a 9-month period revealed that of 8258 orders, 5510 (67%) had TSH/fT4 co-orders and 2748 (33%) had TSH orders only. During this period, there were 83 standalone fT4 orders only. A total of 94% (5195/5510) of TSH/fT4 co-orders had normal TSH during this time. Among the 5510 TSH/fT4 co-orders, fT4 was abnormal and TSH was normal in 229 cases (4.2%).

The percent reduction in TSH/fT4 co-orders and the percent increase in reflex fT4 tests among outpatients and inpatients from tests ordered by nonendocrinology providers between June 1, 2019 and February 28, 2023, along with dates of implementation of the reflex fT4 test, an outpatient thyroid order panel, and associated communication to providers, are shown in the p-charts in Figs 3 and 4. After the first intervention (new reflex fT4 test and provider education), the percentage of TSH/fT4 co-order usage decreased from 67% to 29% (Fig 3), whereas the percentage of reflex fT4 test usage increased from 0% to 57% (Fig 4). After the second intervention (provider performance update), TSH/fT4 co-order usage decreased further to 23% (Fig 3), whereas reflex fT4 test usage increased to 66% (Fig 4). After the final intervention (new thyroid order panel and provider education), TSH/fT4 co-order usage decreased further to 15% (Fig 3) and reflex fT4 test usage increased to 77% (Fig 4), both of which were sustained for more than 8 months.

Among nonendocrinology providers, 5179 fewer fT4 tests were performed between 2019 and 2022, which resulted in a total 3-year direct cost reduction of $45 800. This reduction in fT4 testing yielded direct cost benefits of $25 693, $17 385, and $2722 in years 1, 2, and 3, respectively. Decreased year-over-year cost benefits were due to performing fewer fT4 tests yearly: The number of fT4 tests performed in 2019, 2020, 2021, and 2022 were 8160, 4972, 3601, and 2981, respectively, which translated to 3188, 1371, and 620 fewer tests performed yearly when compared with the previous year and an overall reduction of fT4 tests of 63% (5179/8160). Most of the cost savings were a result of implementation of the reflex fT4 test on February 25, 2020, because the thyroid order panel was not implemented until May 17, 2022. The fT4 utilization rate decreased yearly from 0.64 (8160 fT4/12 666 TSH tests performed) in 2019 to 0.22 (2981 fT4/13 486 TSH tests performed) in 2022.

Following the Model for Improvement Framework²⁵  and continuous monitoring of data over 44 months, this project demonstrated successful implementation of interventions to improve thyroid function test use. TSH/fT4 co-orders were reduced from 67% to 15%, whereas usage of the reflex fT4 test increased from 0% to 77% among nonendocrinology providers and exceeded Smart Aim goals of 30% (TSH/fT4 co-order usage) and 50% (reflex fT4 test usage). These improvements were sustained for more than 8 months after implementation of a reflex fT4 test, an outpatient thyroid order panel, provider education, and necessary changes in EHR test builds. The reflex fT4 system was the biggest contributor to the reduction in fT4 testing, likely because the fT4 test could be performed when needed without a second blood draw. When TSH was abnormal, fT4 results were available sooner. Providers’ acceptance and reliance on this system was likely enhanced by simultaneous education, revisions to order sets, and clinical decision support provided at order entry in outpatient preference lists. Tailoring EHR interventions to the appropriate clinical setting was key in reducing potential inappropriate TSH/fT4 co-ordering and was the rationale for revising separate TSH and fT4 orders in some but not all order sets and preference lists. Additionally, because the balancing measure performed at baseline revealed 4.2% of TSH/fT4 co-orders had a normal TSH and abnormal fT4 and because we wanted to minimize unintended consequences of missing these cases using a TSH-centered strategy, it was important to identify when TSH/fT4 co-orders were appropriate. Because this was not easily identified from system-level data, a goal of 50% usage of the reflex fT4 test was set. This goal was exceeded, as demonstrated by a sustained reflex fT4 test usage of 77%.

Several previous studies have reported their findings following reflex fT4 test implementation^20–23  and clinical decision support.²³  Barry et al²⁰  reported significant (P < .001) reductions in TSH tests from 270 to 195/week and increases in reflex fT4 testing from 107 to 184/week after implementation of the reflex fT4 test but not a significant change in fT4 tests (95 to 91/week) (P = .386). It is unclear why fT4 volumes did not decrease and may have resulted from concurrent ordering reflex fT4 and fT4 tests. After reflex fT4 implementation, Taher et al,²¹  Gilmour et al,²²  and Abitbol²³  reported 39%, 34%, and 30% reductions in fT4 testing, respectively. It is likely our 64% reduction in fT4 use exceeded these studies because of our focus on changes among nonendocrinology providers only as well as variations in changes to order sets and preference lists, which varied from none reported²²  to removing fT4 from existing order sets^21,23  to optimizing preference lists.²⁰  Our approach was to replace individual TSH and fT4 tests with the reflex fT4 test or add the reflex fT4 test to existing order sets or preference lists. Abitbol et al²³  reported cost savings of $43 000 per year as a result of performing fewer fT4 tests. It is likely this cost savings is an overestimate because one would predict direct cost savings to decrease yearly because fewer fT4 tests would be performed year-over-year. Abitbol et al²³  also used clinical decision support at order entry for fT4 whereby providers were required to select an appropriate indication to order fT4. Our clinical decision support panel was upstream of fT4 ordering because we wanted to encourage reflex fT4 ordering, which our panel defaults to, but also allows ordering of TSH and/or fT4 if indicated. Although similarities existed between our study and previous studies, learning and improving on existing studies adds value in health care by focusing on continuous quality improvement.

The financial implications of this and other Choosing Wisely initiatives are important to understand. At its core, Choosing Wisely is designed to remove non–value-added care. In this instance, we demonstrated a reduction in unnecessary fT4 testing and reported a direct cost savings of $45 800 over 3 years at our single institution. A total of 5179 fewer fT4 tests were performed between 2019 and 2022 among nonendocrinology providers. After several rounds of provider education, the direct cost savings associated by avoiding inappropriate fT4 test orders was mainly attributed to performing the reflex fT4 test and the cost savings were most pronounced in years 1 ($25 693) and 2 ($17 385). These cost savings were associated with corresponding reductions in fT4 utilization per case as measured by the ratio of fT4 to TSH tests. Although some initial costs in terms of technologist time and information technology resources were required to implement the reflex fT4 test, these 1-time build and setup costs were excluded from the overall cost analysis. True cost savings can be difficult to quantify because direct cost savings do not consider potential downstream effects such as repeat and/or unnecessary additional testing as well as referrals to specialists for incidental findings. On the reimbursement side, historically, laboratory testing is reimbursed in outpatient settings on a fee-for-service basis, with payment provided for each test performed. This outpatient payment model provides little financial incentive to manage test use. In the inpatient setting, laboratory testing is reimbursed as part of a diagnosis-related group, in which reimbursement is fixed regardless of the number of tests performed. Therefore, reducing test vol reduces costs but not revenue so financial incentives exist for managing inpatient test use. It is likely that reimbursement models currently in place for tests performed on inpatients will, in the future, also encompass outpatients. Capitated care, managed care, and accountable care models are built on the same principle, which involves paying a predetermined per-patient amount to cover all services over a defined time. In these models, the impact on fT4 reduction is similarly favorable to inpatient bundled payments. To improve value-based care, our study focused on improving thyroid function testing in both the outpatient and inpatient setting. Although this has a negative impact on hospital outpatient revenue, it ensures evidence-based best practice.

There were some limitations to this improvement project. First, direct costs are calculated locally and although calculations would translate broadly to other organizations, the actual dollar values and thus scale of cost savings may differ. Second, systems with weaker provider alignment may not achieve similar improvements. Third, data analytics may pose challenges because of: (1) pulling data from different sources (ie, laboratory system versus EHR) and use of different analytics software; (2) defining how data are pulled (ie, units of service/billed tests versus cases/ordered tests); and (3) changes in testing methodology, reference range, and/or costs during a multiyear study.

In this study, orders placed by endocrinology providers were excluded from analysis based on the rationale that these individuals are experts who diagnose and treat complex thyroid disorders and are familiar with clinical practice guidelines and Choosing Wisely recommendations. This patient population may include those with secondary or tertiary (central) hypothyroidism, thyroid hormone or thyrotropin resistance syndromes, or medication-induced hypothyroidism, in which monitoring and screening of fT4 and other thyroid function tests may be necessary. Previous studies have reported test ordering patterns among endocrinologists vary with experience and those with the least experience having the highest proportion of potentially inappropriate test requests.^29,30  This suggests that future investigations into ordering patterns among endocrinology providers may provide an opportunity for further improvements in thyroid function test utilization. Additional future interventions may also target reducing redundant orders that occur proximate in time and ordered by different providers throughout the enterprise. With a recommended maximum TSH testing frequency endorsed by clinical practice guidelines of every 4 weeks, clinical decision support tools to define a minimum retesting interval of 4 weeks may be justified and has been used by other organizations.^29,31 

We thank Carlo Cornacchione and Shannon Niziolek for their assistance in implementing the reflex fT4 test in the laboratory.