Improving HPV Vaccination Rates: A Stepped-Wedge Randomized Trial

Interventions took place in Massachusetts in the Boston metropolitan area and included 5 clinical sites serving primarily low-income and minority patients. One site was a safety-net hospital; the others were independent federally qualified health centers. Sites were selected before the intervention period on the basis of 2014 site-specific data indicating HPV vaccination rates below national targets.⁹  The DOSE HPV intervention is a provider-focused program that included 7 sessions, 1 hour each, performed at approximately monthly intervals over 6 to 8 months between 2016 and 2018 (Fig 1). Sessions were held during regular staff meetings to maximize attendance, and food was served. Intervention sessions were interprofessional, including clinic leadership, primary care providers, and nursing staff. The intervention educates providers and activates them to make tailored systems changes within their health care settings. The intervention included 3 core components: interprofessional education and communication training for health care providers, data assessment and feedback, and creation of an action plan that included both provider- and systems-level changes. All sessions were facilitated by the intervention team, and sessions at which site-specific action plans were developed were led by clinical champions and participating providers.

In session 1, the study team provided health center–specific preintervention data on HPV vaccine initiation and completion rates. For session 2, a physician expert provided education on HPV-associated cancers, effectiveness of vaccination, and evidence-based interventions. Sessions 3 and 4 were focused on motivational interviewing training.²⁸  During session 5, the study team used quality improvement methods to collaboratively identify barriers to HPV vaccination and create site-developed action plans, including systems- and provider-level changes to improve performance. Sessions 6 and 7 were facilitated discussions reviewing progress on implementing action plans, defining near-term action steps, and specifying task ownership to address obstacles and accelerate progress. Although the 5 sites developed action plans specific to their unique practice environments, all decided to routinely recommend HPV vaccination before age 11 (Fig 1). The data review motivated this change because clinic-level vaccination rates revealed that, although most patients received vaccination at well-child visits, a substantial proportion of their primary care populations remained unvaccinated primarily because of attrition in well-child care among older adolescents.

The trial used a stepped-wedge cluster randomized trial design: the study team randomized clinical sites (clusters) by generated number, and interventions occurred in order of their random number, staggered by ∼6 months during the 3 year study period.²⁹  Interventions began on April 7, 2016, and ended on March 15, 2018. Electronic medical record data were used to evaluate the intervention’s effect on HPV vaccination rates. The study population included adolescents aged 9 to 17 years with an assigned primary care provider and ≥1 visit to the primary care site at any time during the 3-year study period. The study period was divided into preintervention, intervention, and postintervention periods for each site. The intention for the stepped-wedge design was for all preintervention periods to begin on the same date (October 1, 2015), 6 months before the intervention start date at clinical site 1. However, because of changeover of electronic medical record systems, preintervention periods began later at sites 4 and 5 (November 1, 2015, and June 1, 2016, respectively; Fig 1). The intervention periods began on the date of the first session and ended on the date of the last session at each site. The postintervention periods began 1 day after the last intervention session at each site and ended on September 30, 2018, for all sites. Differential pre- and postintervention periods in the stepped-wedge design serve to control for secular trends and allow comparisons within sites over time as well as between sites.²⁹ 

All analyses accounted for the national change from a 3-dose to a 2-dose schedule; dose eligibility mirrored recommendations on the date of the clinic visit. Before December 16, 2016, all adolescents required 3 doses to complete the series; visit dates ≥2 months after the first dose and ≥4 months after the second dose were considered eligible. After December 2016, the 3-dose schedule was applied only to adolescents who initiated the series on or after their 15th birthday. Adolescents who initiated vaccination before their 15th birthday were considered complete after 2 doses, with visit dates ≥5 months after receipt of dose 1 considered the minimum interval conferring eligibility for dose 2, consistent with CDC recommendations. Analyses considering intervals of ≥6 months and ≥12 months for dose 2 eligibility yielded similar results.

Our primary outcome was the likelihood that a patient who was due for an HPV vaccine dose would receive vaccination at an eligible clinic visit. All visits to the primary care clinical site by an adolescent who was eligible to receive dose 1, 2, or 3 on the date of the clinic visit were eligible, including nurse-only and problem-focused visits. Visits to specialty departments or emergency departments were not eligible. We performed subset analyses focusing on 9- to 10-year-olds, 11- to 12-year-olds, and 13- to 17-year-olds to examine intervention effects on early, on-time, and catch-up vaccination. Our secondary outcome was rate of “on-time” completion, defined as completion before the 13th birthday, for children aged 12 and 13 years on the visit date. We also examined population-level vaccine initiation and completion rates estimated for patients who visited a study site at any time during the preintervention, intervention, or postintervention periods.

We used longitudinal generalized linear models to estimate the effect of the intervention on the primary and secondary outcomes.^29,30  Our main covariate of interest was a categorical indicator of the 3 study periods: preintervention (0), intervention (1), and postintervention (2). For the primary outcome analysis, the unit of observation was a patient outpatient visit, and we included all eligible patient visits during the study period. We regressed the indicator of receipt of an eligible vaccination (0 or 1) on the period indicator as well as potential confounding variables, including child age, sex, race and ethnicity; English as primary language; insurance coverage type; and indicator of receipt of tetanus or meningitis vaccination during the outpatient visit. We included dummy indicators of calendar time by quarter to capture secular temporal trends. We used a patient-level random-effects specification to capture the clustering of all visits and individual unobserved patient effects; because each patient is affiliated with a provider and clinic, patient-level random effects also capture systematic variation by provider and clinic. We obtained heteroscedasticity-consistent robust SEs.³¹  Similar regression models were estimated for the secondary outcomes by using the appropriate eligible cohort noted above. As a check on the robustness of our findings, we estimated the same regression models with a patient-level fixed-effects specification instead of random effects; with fixed effects, only within-patient variation across visits is used to obtain estimates, avoiding potential confounding from unobserved factors that may be correlated with patient-level variables.³¹  Fixed-effects models yielded similar results; random-effects models are reported in the text and tables. The Boston University Internal Review Board approved all intervention and evaluation protocols.

A total of 16 136 individuals were included in our analyses (Table 1). Patient demographics reflected the populations served by the clinical sites in the preintervention, intervention, and postintervention periods. Patients had a mean age of 12 years (range: 9–17 years), were equally divided between male and female individuals, and were racially diverse (12.0% to 15.3% white, 19.6% to 29.8% black, 37.1% to 55.0% Hispanic or Latino, 13.4% to 17.8% other). Approximately half (42.7% to 56.0%) spoke English as their primary language, and >80% had Medicaid or other subsidized insurance. Differences in demographics between the pre- and postintervention periods relate to the stepped-wedge design. Clinical sites 1, 2, and 3 had greater proportions of Hispanic and non-Hispanic black patients and longer postintervention periods, whereas clinical sites 4 and 5 had more non-Hispanic white patients and longer preintervention periods.

In Fig 1, we describe the stepped-wedge design in detail by calendar month. The action plan components enacted by each site are noted, as are whether these changes were continued in the postintervention period. The timing of preintervention, intervention, and postintervention phases is delineated for each site, along with the unadjusted vaccine initiation prevalence stratified by ages 9 to 10, 11 to 12, and 13 to 17 years. The monthly vaccine initiation prevalence is the proportion of adolescents with an eligible visit at the site in that month that initiated the HPV vaccine series either before or during the visit. Initiation rates for 9- and 10-year-olds generally increased during the intervention periods, generally from <20% to >50%. More gradual improvements were noted for older age groups.

At each intervention site, we evaluated unadjusted preintervention, intervention, and postintervention population coverage of HPV vaccine series initiation and completion, defined as the proportion of all 9- to 17-year-old patients with at least 1 visit to the clinical site in the preintervention, intervention, or postintervention periods with either 1 HPV vaccine dose or a complete HPV vaccine series, respectively. For all clinics combined, population-level series initiation coverage increased from 75% (preintervention) to 84% (intervention) to 90% (postintervention), and series completion increased from 60% (preintervention) to 63% (intervention) to 69% (postintervention) (P < .001 for all comparisons). Increases occurred at all clinical sites, with initiation rates exceeding 80% for all sites by the postintervention period (Fig 2 A and B). Because HPV vaccination is routinely recommended at ages 11 to 12, this group was analyzed separately. Series initiation increased from 83% (preintervention) to 89% (intervention) to 93% (postintervention), and series completion increased from 54% (preintervention) to 53% (intervention) to 69% (postintervention) (P < .001 for all comparisons except completion in the intervention period, P = .40); individual clinic performance is detailed in Fig 2 C and D.

Adjusted models that accounted for clustering, covariates of interest, and calendar time were used to further explore the effects of the intervention. Adjusted models indicated that the likelihood of receiving vaccination at an eligible visit for children aged 9 to 10 increased by 12.7 percentage points (intervention) and 27.3 percentage points (postintervention) compared with the preintervention period (P < .001 for all comparisons; Table 2). For ages 11 to 12, increases were 16.2 percentage points (intervention) and 17.8 percentage points (postintervention periods; P < .001 for all comparisons; Table 2). For ages 13 to 17, the likelihood of vaccination increased by 8.1 percentage points in the intervention period (P < .001; Table 2) but did not remain significantly elevated in the postintervention period. Hispanic and non-Hispanic black patients, patients whose primary language was not English, and those with subsidized insurance were more likely to receive vaccination than were non-Hispanic white, English-speaking, and privately insured patients. Receipt of tetanus and/or meningitis vaccination was strongly associated with receipt of HPV vaccination at the same visit. Over time, the likelihood of vaccination at an eligible visit continued to increase for ages 9 to 10 but decreased for older age groups. This may represent a ceiling effect, as >90% of patients aged 11 and older who attended clinic visits in the postintervention periods had already initiated the vaccine series (Fig 1). Visits appearing as “missed opportunities” in older age groups in later years may represent factors not amenable to clinical practice changes; for example, patients who either were ineligible or whose parents consistently refused vaccination. We found some indirect evidence for a ceiling effect: the proportion of eligible visits with a previous missed opportunity rose from 35% to 71% from the preintervention to the postintervention periods, and adjusted models indicated that patients with previous missed opportunities were less likely to be vaccinated at future visits (P < .001; data not shown).

A secondary outcome was on-time completion of the HPV vaccine series, by using the CDC definition of series completion before the 13th birthday and adjusted for visit date relative to national recommendations for a 2-dose schedule. Adjusted models indicate a 4.3 and 4.7 percentage point increase in the likelihood of on-time completion in both the intervention and postintervention periods, respectively, compared with the preintervention period (P < .001; Table 3). Male sex and Hispanic ethnicity were also positively associated with on-time series completion. Over time, the likelihood of an adolescent aged 12 to 13 completing vaccination at or before their eligible clinic visit increased (P < .05 for the third quarter of 2016 onward compared with the fourth quarter of 2015; Table 3).

This stepped-wedge randomized trial of DOSE HPV, a multilevel intervention, was shown to improve vaccination rates during the intervention period, and improvements were sustained during follow-up periods ranging from 6 to 18 months. Our findings contribute to the body of evidence indicating that multilevel interventions appear promising as a means of raising vaccination rates.^19–23  Because multilevel interventions require substantial investments of personnel and time in the short-term, demonstrating that intervention effects continue in the postintervention period is important when clinical and policy decision-makers consider upfront costs.

Maintaining gains in vaccination rates may depend on the sustainability of intervention components.³²  The DOSE HPV intervention activates providers to create tailored systems changes, which may allow improvements in HPV vaccination provision to be sustained after intervention completion. Health intervention literature indicates that systems-level changes require fewer ongoing resources and can be more sustainable than provider-focused efforts, especially as participating providers may leave the practice after intervention completion.^33,34  However, research also demonstrates benefits to educating providers³⁵  as well as developing interventions in partnership with stakeholders to ensure relevance and capacity.^33,34  Viewing the DOSE HPV intervention through this lens, educational sessions and communications training may have activated providers in the short-term, and allowing participants to develop tailored systems changes to address barriers may have promoted sustainability by building engagement and aligning efforts with existing clinical processes.^32,36,37  System-wide efforts to routinely recommend HPV vaccination at younger ages were considered to be easy and successful changes by providers³⁸  and appeared to be a key factor underlying population-level gains over time. Initiating vaccination at age 9 or 10 can increase the number of opportunities to complete vaccination on time.³⁹  High rates of vaccination among 9- and 10-year-olds in the postintervention period indicated that early vaccination was acceptable to parents. Providers attributed high parental acceptance of early vaccination to a preference for fewer shots per visit as well as an attenuated connection between vaccination and sexual activity at younger ages.³⁸  The ability of sites to implement and sustain this change is demonstrated by increases in early and on-time vaccine series initiation and completion that persisted through the postintervention periods. Researchers of other studies have also demonstrated that initiating vaccination before age 11 years can increase on-time vaccination rates; therefore, this may be a crucial component of future interventions.^40,41 

This study has several limitations, including a limited geographic sampling area and implementation in clinics serving primarily low-income, minority, urban populations. Although the clinical sites were all resource constrained, they did have relatively high HPV vaccine initiation rates at baseline (75%), indicating that providers and parents were already accustomed to vaccinating. Two of our sites transitioned to a new electronic medical record system, limiting our ability to identify patients who might receive primary care at the intervention sites but who had not presented for care in >12 months; this in turn could lead to overestimation of population-level vaccination rates. Also, we could not identify immunocompromised adolescents who required a 3-dose schedule.

HPV vaccination rates continued to increase nationally during the study period, likely facilitated by the release of the 2-dose schedule in December 2016. Although the stepped-wedge design is intended to control for secular changes, fully disentangling intervention effects from contextual factors is difficult. To be able to statistically estimate the effect of the intervention from that of the 2-dose schedule, we would need to separately estimate the DOSE HPV effect among those on 3-dose and 2-dose schedules separately. Because our intervention focused primarily on younger adolescents, all of whom were affected by the 2-dose schedule switch, we cannot perform this type of stratified analysis and primarily use the stepped-wedge design to assess for secular trends. The regression models indicate that the intervention increased the likelihood of vaccine receipt and series completion after controlling for calendar time. Despite these limitations, this trial is one of the first to indicate effectiveness of a multilevel intervention leading to improvements in HPV vaccination rates 6 to 18 months beyond completion of the intervention. If long-term gains are replicated in other populations, these types of interventions could be uniquely useful tools to improve HPV vaccination rates in the United States.

Multilevel interventions that include interprofessional provider education, data feedback, and tailored systems changes that focus on initiating HPV vaccination before age 11 may be successful in improving HPV vaccine series initiation and completion beyond the conclusion of the intervention period. Replication of these findings in settings with lower baseline vaccination rates, in other geographic regions, and among rural populations and those with private insurance is an important area for future research.

We thank the providers and clinic staff who participated in the intervention.

CDC

Centers for Disease Control and Prevention

DOSE HPV

Development of Systems and Education for Human Papillomavirus Vaccination

HPV

human papillomavirus

PI CME

Performance Improvement Continuing Medical Education