Improving Guideline-Based Streptococcal Pharyngitis Testing: A Quality Improvement Initiative

The project was conducted at an ambulatory pediatrics practice in Kansas City, Missouri. The practice includes 12 board-certified pediatricians, 4 certified nurse practitioners, and 19 nurses. The practice cares for over 40 000 children annually at 2 practice locations and received Patient-Centered Medical Home Level 3 designation from the National Committee for Quality Assurance. The practice trains pediatric residents and medical students. Children’s Mercy Kansas City, a 354-bed children’s hospital, was the sponsoring institution for this quality improvement (QI) project. This project was approved for American Board of Pediatrics Maintenance of Certification (MOC) Part 4 credit.

The team was led by a pediatric infectious diseases fellow (L.E.N) and a pediatric infectious diseases attending (A.L.M) at Children’s Mercy Kansas City. The team leaders have formal training in QI methodology and provided content expertise. Children’s Mercy QI leaders provided coaching and feedback throughout the project. The Model for Improvement was used as a framework to guide this project. The Model for Improvement consists of (1) fundamental questions to understand the aim and expected results of improvement efforts and (2) Plan-Do-Study-Act (PDSA) cycle(s). The PDSA cycle is a means to test and implement change through iterations leading to increased knowledge of the system to improve.¹⁶ 

A fishbone diagram (Supplemental Fig 5) was constructed to determine factors leading to unnecessary testing for GAS pharyngitis. Meetings were held with stakeholders (providers and staff) at the practice site to identify key drivers and establish project aims (Fig 1). A Pareto chart, which is a graph displaying relative frequency of an occurrence, was used to determine which reasons for unnecessary testing should be targeted for improvement to have the greatest impact.¹⁶ 

Multiple interventions were used during the project (Table 1) including face-to-face meetings, provider and staff education, modification of office procedure, communication strategies, and patient and family education.

A Webinar was presented to educate providers and staff regarding incidence of GAS pharyngitis and GAS colonization, complications of GAS pharyngitis, antibiotic complications, and a review of the IDSA guideline for diagnosis and management of GAS pharyngitis. This Webinar was offered at the start of the project period and repeated 2 months into the project. Only 2 of 16 providers participated in these Webinars. Because of the lack of provider participation in the Webinars, the same information was presented at the first provider meeting (intervention 1). Subsequently, an e-mail update of the practice’s trend in unnecessary GAS testing was sent to providers with a published commentary on the changing value of testing for and treating GAS pharyngitis (intervention 2).¹⁷ 

The practice allowed a standing order for nursing staff to perform a RADT on any patient presenting with sore throat before provider evaluation of the patient and regardless of associated signs or symptoms. The practice eliminated this standing order and required provider evaluation of patients before testing.

Discussions were held with providers on communicating with families about recommendations for testing and management of pharyngitis (intervention 3). Providers reported concerns about parental expectations for testing even when it was not indicated and reported concerns about parental request for second-line antibiotics based on a perception that amoxicillin was ineffective. Strategies were provided to respond to these parental concerns (interventions 3 and 4). A 1-page educational handout (Supplemental Fig 6) was distributed to families at the providers’ discretion (intervention 5).

Outcome measures were assessed on statistical process control charts by using Excel QI Macros and R 3.3.2. A control chart is a line chart in which the measure of interest is plotted on the y-axis and the time series is plotted on the x-axis. The addition of a centerline and upper and lower control limits allows for designation between variation inherent to the system and variation that is a result of a special cause, such as the change implemented to improve the system.^16,18 Ongoing audit of outcome measures was performed throughout the intervention period with feedback provided to the practice at each face-to-face meeting.

Two outcomes were measured for this project. The first outcome measure was unnecessary GAS pharyngitis testing, which was defined as a patient meeting ≥1 of the following criteria: (1) age was <3 years and was without documentation of a household contact with GAS pharyngitis, (2) had a presence of ≥2 viral symptoms, (3) had an absence of sore throat, or (4) had an absence of any expected GAS pharyngitis examination findings. Viral symptoms included conjunctivitis, coryza, cough, diarrhea, hoarse voice, and viral exanthema. These criteria, including the specific viral symptoms, were chosen on the basis of recommendations from the IDSA guideline for the diagnosis and management of GAS pharyngitis. The decision to use ≥2 viral symptoms was made because providers involved in the project expressed a concern that documentation may not clearly reflect the timing, severity, or cause of symptoms (eg, lingering cough after resolved viral respiratory tract infection or a noninfectious condition such as allergies), and they suspected viral symptoms would be overrepresented in retrospective chart review. Expected examination findings included tender cervical lymph nodes, tonsillar or pharyngeal erythema, tonsillar exudate or swelling, palatal petechiae, and scarlatiniform rash.

The second outcome measure was appropriate antibiotic use for GAS pharyngitis, which was defined as use of an antibiotic that was clinically indicated (necessary GAS pharyngitis testing was performed and the test result was positive) with selection of a first-line antibiotic for the suitable duration (single dose for intramuscular penicillin, 5 days for azithromycin, and 10 days for any other antibiotic used). Penicillin and amoxicillin were classified as first-line antibiotics for nonallergic patients, and clindamycin, azithromycin, clarithromycin, cephalexin, and cefadroxil were classified as first-line antibiotics for patients with documented penicillin allergy or unknown penicillin allergy status (n = 8).

The project period was October 1, 2014, to October 31, 2016. A standardized abstraction form was used to collect information regarding demographics, symptoms and exposure history, examination findings, penicillin allergy status, antibiotic selection, and antibiotic duration from a clinic encounter note. Symptoms and examination findings were classified as absent if not documented. A convenience sample of 20 patients per month was selected from patients with a Current Procedural Terminology code of 87880 for streptococcal RADTs. The sample size was chosen on the basis of minimum subgroup sizes needed to have upper and lower control limits on the control chart by using estimated baseline levels for outcome measures.¹⁹ Office staff at the practice selected charts from all providers who ordered RADTs each month selecting ≥1 chart for each provider when available. Patient charts were excluded if RADT results were not documented. A current state analysis was conducted at the start of the project by using retrospective chart review of 20 charts per month for a 12-month period (October 2013–September 2014) to determine baseline levels for each outcome measure. The length of the baseline period was selected to identify variation in unnecessary testing that occurs during both high and low GAS pharyngitis incidence seasons, and the timing was designed to capture baseline provider familiarity with the IDSA guideline (published in September 2012). To optimize provider participation, the preintervention period was extended to June 2015 (when intervention 1 occurred).

The effectiveness of the interventions was determined by the presence of a shift (≥8 points above or below the centerline) indicating special cause variation or a nonrandom pattern in the data as a result of a sustained change.¹⁸ Using the selected sample size, we ensured that a change would be detectable when data were plotted in a statistical process control chart.^18,19 Data analysis was performed by using SAS 9.4 (SAS Institute, Inc, Cary, NC) and Stata 14.2 (Stata Corp, College Station, TX). P values of <.05 were considered to be statistically significant. Providers were surveyed at the end of the project to assess their perception of the impact of the project on their individual practice.

The project was considered a local QI project and not human subjects research by the Children’s Mercy Kansas City Office of Research Integrity. A data use agreement was executed between Children’s Mercy Kansas City and the practice involved in this project.

During the project period, the practice performed 3893 streptococcal RADTs. Fewer than 20 charts were available for review from the practice for 3 months of the project (March 2015, May 2015, and July 2015). A total of 734 patient charts were selected for this project with 25 charts (3.4%) excluded because of unknown RADT results. A total of 709 charts were reviewed (392 from the preintervention period and 317 from the intervention period). The average frequency of unnecessary GAS pharyngitis testing per month was 64% before intervention 1. Special cause signal was detected beginning December 2015 with 11 consecutive points below the baseline average, which resulted in a new average of 40.5% (Fig 2). The presence of viral symptoms was the most frequently observed criterion for unnecessary testing followed by the absence of documented sore throat. Testing of patients <3 years old decreased significantly during the project period (Fig 3).

The centerline for appropriate antibiotic use for GAS pharyngitis per month was 49.1% throughout the entire project, with a random pattern of data indicating common cause variation (Fig 4). Over the course of the project, including the baseline period, 140 inappropriate antibiotic prescriptions were given for pharyngitis (119 were unnecessary because of unnecessary testing or negative test results). Unnecessary GAS pharyngitis testing was the most frequently observed criterion for inappropriate antibiotic use throughout the preintervention and intervention periods (Table 2). Throughout the baseline period and the entirety of the project, the median proportion of appropriately selected (first-line) antibiotics was 87.5% (75%, 100%).

The provider survey response rate was 100% (16 of 16). All providers responded affirmatively when asked if this project was meaningful and if they would participate in a similar project again. Additionally, most providers (88%) believed their ability to communicate with families about the need for testing and about antibiotic use improved during the project. The majority of providers (94%) believed the project did not negatively impact clinic efficiency. Spontaneous responses to inquiry about what had the greatest impact on reducing unnecessary GAS pharyngitis testing in their practice included “limiting nurses from testing kids with URI [upper respiratory infection] symptoms,” “specific criteria for testing,” and “education on best practice.”

Injudicious use of GAS pharyngitis testing may lead to avoidable health care costs, unnecessary antibiotic exposure, and unintended harms, including adverse events¹⁷ and increased risk of antibiotic resistance. With our results, we suggest that an interprofessional collaboration using QI methods can successfully improve adherence to guideline-based GAS pharyngitis testing in ambulatory pediatric practice. Additionally, these QI methods were associated with provider perception of an improved ability to communicate with families about the need for testing and about antibiotic use. Despite a reduction in unnecessary testing, a significant improvement in appropriate antibiotic use for GAS pharyngitis was not observed during the project, underscoring the need for further efforts.

The majority of GAS pharyngitis testing in this practice was inconsistent with guideline recommendations for testing before intervention. The use of ≥2 viral symptoms as a criterion to determine unnecessary testing may even underestimate unnecessary testing. The decision to use this criterion is supported by a publication by Shapiro et al²⁰ in which they found an inverse relationship between the number of viral features and the prevalence of GAS pharyngitis demonstrating that patients with ≥2 viral features were >40% less likely to have GAS pharyngitis compared with patients with no viral features. The use of a conservative estimate of unnecessary testing would not be expected to affect the magnitude of absolute reduction that was observed, which was the focus of this project.

A number of local factors were key to reducing unnecessary testing. The practice was receptive to process change, including the modification of existing office procedure, which likely contributed to the observed improvement in guideline-based testing. Additionally, physicians received MOC Part 4 credit for their work on this project, which incentivized their engagement. Faculty engagement in an MOC project, along with other interventions, was previously used to successfully improve adolescent pregnancy screening rates.²¹ 

Despite local contextual factors, we did not observe the magnitude of absolute reduction in unnecessary GAS pharyngitis testing for which we aimed. Cognitive bias likely played a role in unnecessary GAS pharyngitis testing. Some providers reported fear of complications that could result from missed GAS pharyngitis diagnosis as a driver of their decision to perform GAS pharyngitis testing. We recognized that providers overestimated the risk for complications associated with GAS pharyngitis and underestimated risks associated with antibiotic use (ie, base rate neglect).²² 

Our project design relied on educational interventions. Previous studies reveal that passive educational interventions result in limited sustainable improvement.²³ Face-to-face educational outreach, involvement of local opinion leaders, and a combination of interventions deployed simultaneously have been used to successfully create QI.²⁴ We combined educational interventions with some of these methods to increase stakeholder buy-in and audit with feedback on performance in outcome measures to achieve greater improvement. Despite these efforts, variability in unnecessary GAS pharyngitis testing was observed. With our results, we underscore the challenges associated with QI initiatives that are aimed at modifying provider behavior and highlight the importance of provider buy-in to achieve improvement. Analyses for this project were performed at the practice level. Provider-level analysis of unnecessary GAS pharyngitis testing could be used to identify specific providers to target for individualized future interventions. Additionally, future projects in which researchers evaluate the use of higher reliability interventions, such as decision support tools embedded in electronic health records, are needed.

We aimed for improvement in appropriate antibiotic use for GAS pharyngitis after our interventions. Our definition encompassed clinical indication, antibiotic selection, and antibiotic duration. Adherence to guideline-based first-line antibiotic selection was higher in this practice (median 87.5%) than reported in published national data,¹ leaving little room for improvement in antibiotic selection. Adherence to guideline-based antibiotic duration was also high. Prescribing antibiotics to patients with unnecessary GAS pharyngitis testing remained the primary reason for inappropriate antibiotic use before and after intervention. With these data, we suggest that a larger reduction than we observed in unnecessary GAS pharyngitis testing is needed to significantly improve appropriate antibiotic use for GAS pharyngitis. Further efforts are needed to tackle the important national problem of antibiotic overuse for pediatric pharyngitis.

Our project has limitations. Multiple PDSA cycles were introduced in a 6-month period, making it difficult to know which intervention had the greatest impact. The QI infrastructure available for our project may not be available in smaller practices, which may limit generalizability of our results to smaller practices. The number of providers included in this project was small and the patient sample was selected from patients on the basis of GAS pharyngitis testing and not on the basis of antibiotic prescribing or clinical presentation. This sampling method resulted in small sample sizes of antibiotic prescriptions each month and excludes patients who received antibiotics for pharyngitis without a RADT; however, the majority of patients (>98%) who received an antibiotic prescription for an indication of pharyngitis or streptococcal pharyngitis in this practice during the project period had a streptococcal RADT performed. Individual provider-level variation in antibiotic prescribing for pharyngitis was not analyzed because of the small sample size. Fierro et al²⁵ found significant variation in off-guideline prescribing for GAS pharyngitis within pediatrics practices. Larger studies are needed to evaluate the impact of similar interventions on individual provider prescribing for GAS pharyngitis.

Our collaborative, interprofessional QI initiative was successful in reducing unnecessary GAS pharyngitis testing in a community pediatrics practice. Interventions used to achieve improvement included provider and patient and family education, modification of office procedure, and communication strategies with periodic feedback provided on performance in outcome measures. Important contextual factors included QI infrastructure, provider buy-in, and MOC status of the project. Additional work is needed to identify higher reliability interventions that reduce variability in testing practices and result in improved antibiotic use for GAS pharyngitis across diverse practice settings.

GAS

group A streptococcal

IDSA

Infectious Diseases Society of America

MOC

Maintenance of Certification

PDSA

Plan-Do-Study-Act

QI

quality improvement

RADT

rapid antigen detection test

We acknowledge the providers and staff at the practice for their work on this project and their dedication to practice improvement. We acknowledge Megan Bledsoe, MSc, CCRC, for her assistance with data collection. We also acknowledge the Medical Writing Center at Children’s Mercy Kansas City.