Reducing Unnecessary Testing in Uncomplicated Skin and Soft Tissue Infections: A Quality Improvement Approach

The study took place in a large, urban, tertiary medical center and the only freestanding children’s hospital in the surrounding county. The hospital serves a total catchment area of >900 000 children and has >20 000 annual pediatric admissions and 96 000 emergency department (ED) visits. All general pediatric patients are admitted to the PHM service. Ordering providers for both the PHM and ED services include attending physicians, fellows, residents, and pediatric nurse practitioners.

Our institution has successfully used and sustained evidence-based pathway (“pathway”) order sets that include clinical decision-making elements for other common conditions.²⁴  The team used this existing framework, quality improvement (QI) tools, and the plan-do-study-act²⁵  methodology to understand barriers to improvement and implement interventions. We created an initial interdisciplinary study team of physicians (PHM and infectious disease), nurses, laboratory technicians, pharmacists, quality management (QM) staff, and information technology staff. We included 2 members from QM to help with definitions and data interpretation. The project lead met with these content experts in small and large group sessions over the course of 4 months.

The team reached a consensus on intervention choices using an iterative process that included discussion on the feasibility of proposed interventions. To identify barriers and potential solutions, we used Ishikawa and key driver diagrams (Figs 1 and 2), literature review, and local data. Using these standard QI processes as a targeted needs assessment, we identified the lack of guidelines and the local culture of obtaining laboratory evaluation as major motivations for laboratory testing (Fig 1). Key drivers of reducing unnecessary testing included education, provider buy-in, use of the electronic health record (EHR), and collaboration with the ED.

The team followed our local pathway program process to achieve group consensus on inclusion and exclusion criteria via iterative, in-person discussions of evidence-based literature and local data. Patients admitted to the PHM service with International Classification of Diseases, 10th Revision codes for cellulitis and/or abscess that did not meet exclusion criteria were eligible (Table 1). Exclusion criteria, including service (admitted to a non-PHM service or any ICU days during the encounter), age (<2 months), and coded comorbidities or complications, are noted in Table 1. All patient lists for the baseline year, intervention year, and sustainability period were obtained in the same manner from QM by using hospital billing data and filtered by the QM analyst using these criteria. Baseline data were obtained from review of a random sample (50 out of 169; 30%) of the January 2016 to December 2016 records. During the January 2017 to December 2017 intervention year, all records were reviewed by a member of the team and presented to the entire team for discussion. Any records with concern for a complicated SSTI were excluded by team consensus. We chose these exclusion criteria to avoid unintended use of the pathway order set for patients requiring additional testing and to avoid harms such as delays in care or need for alternate treatments. During the sustainability period from January 2018 to February 2019, records were reviewed until the end of the collaboration meetings in August 2018.

Ordering providers received education in sessions grouped by level of training (residents, fellows, and attending physicians). We chose these groups to allow participants to have open discussion and address clinical or process questions that could differ between groups. The education sessions reviewed evidence-based guidelines for the evaluation and management of uSSTIs, data on testing at our institution for the baseline study period, and rationale for creating a pathway order set. We used evidence-based education to challenge the local culture and to provide the opportunity for clinicians to voice concerns over changing their practice. They were also able to ask questions about the literature and review examples of patients who would meet inclusion criteria for use of the pathway order set. Those who were unable to attend received this information electronically, with the option to review the contents with the project lead.

The pathway order set was created by using evidence from the literature and subject-matter expert consensus. It contained guidance including avoidance of routine blood laboratory evaluation. The 4 key tests that were the focus of this project (BCx, CBC, CRP, and ESR) were intentionally not included in the electronic order set, thereby forcing the provider to order them separately, if desired. We chose a “go live” implementation date of March 17, 2017, after education and study team pilot testing by using mock patients in our “test” environment of the EHR were completed. As with our other pathways, the order set would show as “recommended” for providers if key terms were noted in the problem list or principal diagnosis.

We chose to start the project within our PHM service to “lead first” by holding ourselves accountable for reducing testing and standardizing care for these patients. The actions taken, data from, and commitment to the project by the PHM service were drivers that enabled the next cycle of improvement, the collaboration with the ED that began on July 1, 2017. This team of 12 PHM and ED physicians and other members met every 2 weeks to review records and interval and cumulative (run chart) data, identify actionable items, and assign members to deliver ordering provider–specific feedback.

We added the uSSTI order set as a “favorite” to the resident “environment” order set list within our EHR on September 13, 2017, to promote resident use, even if a trainee was unaware of the order set or had not received training.

Patient lists were obtained and records were reviewed as noted in the Defining Target Population and Data Source section. We used interrater reliability of record review to reconcile any discrepancies at each meeting. The team continued to meet during the sustainability period from January 2018 to August 2018 for record review only (no interventions). Data for September 2018 to February 2019 were obtained in the same manner noted above but did not undergo record review.

The primary outcome measure was the mean number of tests (BCx, CBC, CRP, and ESR) ordered in the ED or inpatient setting per patient encounter (rate) and was measured monthly. Additional measures included the proportion of eligible patients with appropriate uSSTI order set use and balancing measures of missed diagnoses, readmissions within 7 days, and unintended order set use, all measured monthly. Hospital chargemaster data for laboratory testing were obtained to assess financial impact.

We used statistical process control with our primary process measure of mean number of tests per patient encounter, displayed on a u-chart. We followed established rules for differentiating special versus common cause variation.²⁶  We used proportions to track balancing measures (counts) and a p-chart for order set use.

The institutional review board deemed this project as not human-subjects research and gave a QI exemption waiver.

The 169 patient encounters meeting our uSSTI inclusion criteria in the January 2016 to December 2016 baseline year had 573 tests (BCx, CBC, CRP, and ESR) performed. We calculated a centerline rate of test use of 3.4 tests per patient encounter in this baseline year. After our interventions, we noted special cause variation with 8 points below the centerline, starting in January 2017 (Fig 3). The new centerline of 2.2 tests per patient encounter (130 eligible patients with 286 tests), reflects a 35% reduction in testing in the January 2017 to December 2017 intervention year. Continued measurement demonstrated postintervention sustainability from January 2018 to February 2019 without further shifts in our centerline. Use of the pathway order set (from April 2017 to Dec 2017) was 80% (83 out of 104) and increased to 87% (86 out of 99) in the postintervention period. We noted special cause variation for pathway order set use in May 2018. This resulted in a change in the centerline that demonstrates 90% use after this shift (Fig 4).

There were no unplanned readmissions or missed or delayed diagnoses for the 130 intervention-period patients. Of the 75 patients meeting exclusion criteria during this period, 23 had unintended order set use. However, there were no patient harms (delays in diagnosis, errors in testing or treatment choices, or unplanned readmissions) from this use. During the intervention period, there were no other projects or system changes that would have affected our interventions or results. Although the project aim did not include reduction in charges, the decreased use of the 4 tests (272 fewer tests) resulted in a reduction in hospital charges of $49 413 over the 26-month intervention and sustainability period.

Using QI methodology and an active PHM-ED collaborative QI project team, we achieved our aim and successfully reduced unnecessary testing by 35% in patients with an uSSTI in our PHM service, with no patient harms identified. This study demonstrates how laboratory testing can be reduced by identifying the appropriate patient population in which to limit testing, addressing the current culture, and using the EHR to guide clinical decision-making. The widespread adoption of the EHR has facilitated the use of order sets and clinical decision support, which can be used to effectively promote evidence-based practice^27–29  and reduce overtesting.^30–32  Our reduced testing with stable balancing measures also mirrors outcomes from other QI studies that have shown that reducing laboratory testing does not result in increased readmission rates^27,33  or missed diagnoses.^34,35  By examining key drivers for testing and developing relevant targeted interventions, we effectively changed the laboratory ordering practices of providers and sustained this change without missing diagnoses or increasing unplanned readmissions.

Targeted education addressed our finding that some providers felt compelled to order laboratory tests even when it was not clinically indicated and did not change management. One study demonstrated that residents routinely ordered unnecessary testing because of a variety of reasons: practice habit, discomfort with diagnostic uncertainty, and lack of role modeling of cost-conscious care.³⁶  We found that trainees in our institution identified similar reasons for ordering tests and targeted our interventions to address these issues. Although we did not use formal Lean³⁷  process improvement, we did borrow from the concept of listening to the voice of the customer in our team meetings. We used the existing pathway order set structure and visual layout that was familiar to our institution to achieve our outcome.

The team-based creation of inclusion and exclusion criteria allowed for a larger discussion on the appropriate use of clinical guidelines and the importance of avoiding potential patient harms. In particular, team members shared thoughts on the value of testing when the diagnosis is not clear. This underscored that guidelines are adjuncts and not replacements for clinical decision-making. Clinical guidelines have been shown to improve quality of care³⁸  but need to be used in the appropriate clinical context because guidelines also have limitations.³⁹  Clear delineation of patients for whom this uSSTI pathway was appropriate allowed the team to have confidence that the reduction in testing was indeed in unnecessary testing in a low-risk population.

We leveraged our EHR to automate steps to increase reliability. The order set had prechecked orders, purposefully excluded orders for the 4 tests we targeted, and provided guidance to help promote thoughtful decision-making around laboratory testing. It was also critical for team members to voice their opinions on content, integrate evidence-based medicine, and see the order set in phases of creation so they could have confidence in the result, ultimately making it easy to do the right thing. Achieving a higher level of order set implementation, noted as special cause in May 2018, could have been a key reason for the sustained outcomes for our primary measure.

Our charge savings only reflect the decrease from testing and not the potential savings in possible decreased antibiotic use, subsequent laboratory testing to trend abnormal laboratories, or other possible impacts of reduced testing. Although these additional items were not calculated in our project, 1 study of evidence-based management of cellulitis in adult patients demonstrated reduced pharmacy and facility costs.²⁷  Discussion of the concept of value and health care costs are especially important because other studies have also shown that provider spending habits are often formed during residency and can have long-lasting effects in clinical practice.^40–42 

Yet our strength was in the development of our PHM-ED collaboration that encouraged transparency and open communication that we believe led to our positive outcomes, sustained gains, and, importantly, an ongoing positive relationship between these services. Collaboration was crucial because most admitted patients first present to our ED, where the majority of tests are obtained. Our collaborative meetings facilitated opportunities to discuss overtesting for uSSTIs and high-value care, which further strengthened the PHM-ED relationship. Record reviews and discussions also resulted in increased awareness of the value of peer feedback, which has been shown to be effective in other QI studies.^29,43  Meeting every 2 weeks created a high level of engagement and encouraged discussion between meetings. We discussed the value of reduced testing from the perspective of the patient and family experience (eg, pain from blood draw and confusion from possible false-positive culture results) and the system (eg, staff time to perform blood draw and charges to the family and health care system). Additionally, discussion with trainees across both ED and inpatient environments identified ways to incorporate interventions that fit into their workflow and provided education about high-value, cost-conscious care. This provided a framework for future projects to target other areas of overuse, with a new PHM-ED collaborative QI project already active.

Although we were able to achieve our aim, our outcomes were modest. We learned that there are limitations with automation and challenges to changing practice habits. We did not survey providers, but the team noted a number of perceived barriers to greater success: trainees or less-experienced faculty may feel the need to see tests to validate their clinical decision-making; trainees from different core institutions may have practices that differ from ours; ED and ward providers may have different testing habits in general; and providers may be concerned that not testing could be viewed by the patient as not doing anything. Studies have shown that providers are influenced by these factors, which may lead to increased testing or unnecessary treatment.^9,44  Future steps to address these barriers could include performing cycles of improvement for 1 test at a time and obtaining qualitative data from patients, families, and providers to further investigate these factors. Additionally, there is a need to identify the best method by which to efficiently and effectively inform families of the evidence behind judicious testing and engage them in shared decision-making when possible. Some studies have shown that these practices may lead to reduced testing⁴⁵  and use of resources⁴⁶  and that tests with low pretest probability provide little reassurance to patients.⁴⁷ 

Despite our success, our study had several limitations. The QI initiative took place at a single tertiary hospital with a long history of pathway order set use, which may limit generalizability. Not all records for the baseline period had record review performed to validate the inclusion and exclusion criteria applied by using codes, so it is possible that some patients meeting exclusion were included in the baseline. The baseline data were obtained from a random sample of 30%. Records were reviewed by several individuals; however, the PHM-ED team reviewed and reconciled by group consensus. The laboratory tests reflected testing done within our institution and did not capture tests that may have been performed at outside EDs. However, our referral patterns from outside facilities and the percentage of patients with studies performed at outside EDs remained stable during the baseline and intervention years. Finally, our sample size was small. It is possible that with a larger sample, balancing measure rates would change.

Using QI methods, guiding clinical decision-making in the EHR, and creating a PHM-ED collaborative QI project, our team reduced unnecessary laboratory testing in uSSTIs by 35% and sustained the gains. Our approach addressed both ED and PHM culture and used the EHR to increase reliability. This work is useful to other sites because it is focused on a common disease state seen nationally, and the interventions used are not unique to uSSTIs, making them applicable to other clinical conditions. Our next improvement steps for this project will focus on targeted interventions within the ED, using photographs to guide care.

We thank the members of the PHM-ED QI collaborative and specifically key members Cindy Lewis, RN, MS, CNS, CPN; Magali Chavez-Zavala; Rachel Stengone, PharmD; Theresa Mueller, CLS; Paige Anderson, PharmD; Michelle van der Pool CLS (ASCP), SM; Alice Pong, MD; Kristy Schwartz, MD; and Allyson Andrews, MPH, for their assistance in the creation of the order set and in the planning of this QI project.