Nationally, 54.2% of youth are fully vaccinated for human papilloma virus (HPV) with persistent gender and racial/ethnic disparities. We used a quality improvement approach to improve completion of the HPV vaccine series by age 13 years. As a secondary aim, we examined racial/ethnic and gender differences in vaccine uptake.
The study setting included 2 pediatric, academic, primary care practices in Massachusetts. We designed a multilevel patient-, provider-, and systems-level intervention addressing parental hesitancy, provider communication, and clinical operations. Rates of HPV series completion by age 13 were monitored using a control p chart. Bivariate and multivariate analyses evaluated vaccine completion differences on the basis of clinic size, gender, and race/ethnicity.
Between July 1, 2014, and September 30, 2021, control p charts showed special cause variation with HPV vaccine initiation by age 9 years, increasing from 1% to 52%, and vaccine completion by 13 years, increasing from 37% to 77%. Compared with White and Black children, Hispanic children were more likely to initiate the HPV vaccine at age 9 (adjusted odds ratio [95% confidence interval] = (1.4–2.6)] and complete the series by age 13 (adjusted odds ratio [95% confidence interval] = 2.3 (1.7–3.0).
A multilevel intervention was associated with sustained HPV vaccine series completion by age 13 years. Hispanic children were more likely to be vaccinated. Qualitative family input was critical to intervention design. Provider communication training addressed vaccine hesitancy. Initiation of the vaccine at age 9 and clinicwide vaccine protocols were key to sustaining improvements.
Uptake of the human papilloma virus (HPV) vaccine has risen nationally but remains suboptimal. Multilevel interventions that impact patients, providers, and clinic systems show promise. There are significant racial/ethnic disparities. The impact of the coronavirus disease 2019 pandemic on HPV vaccination rates is unknown.
This study shows effects of a multilevel intervention to improve HPV vaccination completion rates for 13-year-old girls and boys, addresses sex and race/ethnicity disparities, and sustains changes over 7 years, including the first 18 months of the coronavirus disease 2019 pandemic.
Human Papillomavirus (HPV) remains the most common sexually transmitted infection in the United States.1 In 2018, the estimated incidence for HPV infection for 15 to 24-year-old youth was 7.1 million.2,3 HPV-associated cancers are higher in both non-Hispanic Black and Hispanic adults.3 The HPV vaccine was licensed in 2006 with a 3-dose schedule for females and in 2011 for males.4–7 A 2-dose schedule was recommended for children aged 9 to 14 years in 2016.6 Since 2006, HPV infections in adolescent females significantly decreased.7–10 By 2020, only 64.5% of United States youth aged 13 to 17 years and 48.4% of 13-year-olds were fully vaccinated for HPV.11 Black and Hispanic children were initially less likely to receive the HPV vaccine, but this disparity shifted by 2019 because Hispanic children had higher rates of HPV vaccine completion (58.1%) than Black children (54.3%) and White children (51.6%).11,12 Sex differences persisted with females more fully vaccinated for HPV than males (56.8% vs 51.8%).11
Previous primary care interventions to increase HPV vaccine uptake focused on providers, parents, systems improvements, and community outreach.13,14 Educational interventions and social marketing show varied success.15–17 Vaccine reminders with phone calls, letters, or texts increased receipt of vaccine.18–20 Successful provider interventions included education, performance feedback, and electronic decision support.17,20–25 Communication training emphasizes strong clear vaccine recommendations.26–28 Clinic systems-based interventions, including provider prompts, and initiating the HPV vaccine at age 9 rather than age 11 are effective and cost-effective.4,27,29–31 Multilevel interventions that build on previous research, combining improvement initiatives focused on families, providers, and clinic systems, show promise.32–34
Although HPV vaccination rates are increasing, they remain suboptimal. Few studies have applied multilevel interventions and measured improvements in HPV vaccination completion for both girls and boys over a sustained period. We sought to use a comprehensive group of quality improvement (QI) interventions to improve HPV vaccine completion for a diverse population of girls and boys. Interventions were targeted at improving family acceptance of the vaccine, provider knowledge and communication skills, and clinic systems. As a primary aim, we evaluated, over 7 years, the cumulative effects and sustainability of this intervention, aiming to improve the rate of HPV vaccine completion by age 13 years for girls and boys at 2 urban primary care practices. As a secondary aim, we examined racial, ethnic, and sex differences in vaccine uptake. We believe study findings may suggest strategies for improving uptake of all vaccines, including coronavirus disease 2019 (COVID-19).
Methods
Setting and Participants
The setting included an academic, hospital-based clinic and a community health center in Massachusetts. The hospital-based clinic serves 16 000 patients aged 0 to 21 years, 75% with Medicaid, with a staff of 29 pediatricians, 65 pediatric residents, 4 nurse practitioners, 20 nurses, 1 QI consultant (QIC), and 33 administrative staff. The community health center, nonfederally qualified, serves 6000 patients aged 0 to 25 years, 85% with Medicaid, with a staff of 6 pediatricians, 2 nurse practitioners, 12 pediatric resident physicians, 9 nurses, 1 QIC, and 15 administrative staff. Both practices have a majority Black and Hispanic patient population. They share clinical leadership, electronic health record (EHR), and have undergone practice transformation using the Model for Improvement,35 improving patient continuity, staff engagement, care teams, and QI processes.
Planning the Intervention
Our objective was to test evidence-based strategies for improving HPV vaccine completion. The improvement team included 6 physicians, 1 departmental QI director, 1 nurse manager, 2 QICs, and 4 evaluators. Key drivers and improvement initiatives were identified using literature review and by conducting focus groups and semistructured interviews with nonresident providers, parents/caregivers, and adolescents at both practices.13,15,16,18,36 From March 2014 to June 2014, the team used this qualitative information to identify individual-, provider-, and systems-level barriers. Individual-level themes included family vaccine mistrust, feelings that their child’s HPV risk was low, parents’ need for more information, and reluctance to return for series completion. Provider-level themes included discomfort discussing the HPV vaccine given its association with sexual activity and lack of time. Systems-level barriers were challenges with the dosage schedule and administration of multiple vaccines at the same time, especially at age 11 when other vaccines were required.37
Using qualitative data and literature review, the team developed process maps clarifying steps of HPV vaccination delivery, highlighting gaps in processes and vaccination barriers. The team proposed potential solutions. In a series of meetings, a final set of intervention components was identified on the basis of feasibility, acceptability, and potential impact, and a key driver diagram (Fig 1) was constructed. The institutional review board at Boston Children’s Hospital approved the study.
Intervention
The QICs maintained the patient-tracking registry, created run charts for staff feedback, and developed statistical control charts. The team collectively planned improvement efforts, obtained practice engagement, and tracked tests of change. Table 1 summarizes intervention components, implemented between July 1, 2014, and June 30, 2016, while the 3-dose HPV schedule was required. The clinics used the 2-dose schedule in September 2016, when the intervention entered a postintervention phase. During the pandemic phase, well-child care and vaccine access were limited as the clinics shifted to telemedicine.
Summary of Clinic System, Provider and Patient Intervention Components
Objective . | Intervention Component . | Implementation Date . |
---|---|---|
Level: Clinic System | ||
Increase access | Monthly evening vaccine clinic | July 2014 |
Decrease parent barriers | Start vaccine series at age 9 | July 2014 |
Decrease provider barriers | Clinic-wide protocol for HPV vaccine series in recommended talking points; focus starting series at age 9 y | July 2014 |
Follow-up visit improvement | Clinic flow: standing orders (nurses administer second and third doses without written physician orders) | September 2014 |
Scheduling: reminder cards with dosing schedule; EHR vaccine order set for subsequent doses | November 2014 | |
Attendance: automated patient tracking reports; nonautomated patient reminder calls; phone calls for missed appointments | January 2015 | |
Track progress | Implement quality measure; report data to clinic regarding progress | November 2014 |
Improve access for HPV vaccination | Urgent care patients HPV status checked; administrative outreach for doses 2 and 3 | August 2015 |
Track HPV gaps | HPV added to preventive bundle report | November 2015 |
CDC recommends change to 2-dose schedule | Clinics revised measures for 2 dose recommendations (new goals): | October 2016 |
1 HPV dose at age 9, second dose age 10 | January 2017 | |
Pay for performance | Medicaid Accountable Care Organization includes HPV in quality measure portfolio. | January 2018 |
Level: provider and nurse | ||
Objective | Intervention component | Implementation date |
Identify barriers and assess knowledge | Provider, staff and parent/adolescent focus groups | May 2014 |
Educate providers decrease parent barriers | Training on importance of HPV vaccine | May and July 2014 |
Use of standardized recommendation language | July 2014 | |
Motivational Interview training for providers | ||
Level: patient | ||
Objective | Intervention component | Implementation date |
Increase knowledge and empower parents | Visual reinforcement campaign with informational handouts in English and Spanish, posters in waiting and exam rooms | September 2014 |
Objective . | Intervention Component . | Implementation Date . |
---|---|---|
Level: Clinic System | ||
Increase access | Monthly evening vaccine clinic | July 2014 |
Decrease parent barriers | Start vaccine series at age 9 | July 2014 |
Decrease provider barriers | Clinic-wide protocol for HPV vaccine series in recommended talking points; focus starting series at age 9 y | July 2014 |
Follow-up visit improvement | Clinic flow: standing orders (nurses administer second and third doses without written physician orders) | September 2014 |
Scheduling: reminder cards with dosing schedule; EHR vaccine order set for subsequent doses | November 2014 | |
Attendance: automated patient tracking reports; nonautomated patient reminder calls; phone calls for missed appointments | January 2015 | |
Track progress | Implement quality measure; report data to clinic regarding progress | November 2014 |
Improve access for HPV vaccination | Urgent care patients HPV status checked; administrative outreach for doses 2 and 3 | August 2015 |
Track HPV gaps | HPV added to preventive bundle report | November 2015 |
CDC recommends change to 2-dose schedule | Clinics revised measures for 2 dose recommendations (new goals): | October 2016 |
1 HPV dose at age 9, second dose age 10 | January 2017 | |
Pay for performance | Medicaid Accountable Care Organization includes HPV in quality measure portfolio. | January 2018 |
Level: provider and nurse | ||
Objective | Intervention component | Implementation date |
Identify barriers and assess knowledge | Provider, staff and parent/adolescent focus groups | May 2014 |
Educate providers decrease parent barriers | Training on importance of HPV vaccine | May and July 2014 |
Use of standardized recommendation language | July 2014 | |
Motivational Interview training for providers | ||
Level: patient | ||
Objective | Intervention component | Implementation date |
Increase knowledge and empower parents | Visual reinforcement campaign with informational handouts in English and Spanish, posters in waiting and exam rooms | September 2014 |
At the systems level, we developed initiatives to improve both efficiency of vaccine administration and vaccine access. A nursing protocol allowed nurses to recommend and administer the vaccine before providers entered the room. The clinics increased vaccine access by creating a monthly evening shot clinic. To decrease parent barriers related to number of vaccines typically administered at ages 11 to 12, we standardized initiating the HPV vaccine at age 9. We developed reminder-based prompting by programming EHR checkboxes for ordering subsequent doses of the vaccine, gave patients dosing schedule cards with the suggested dates of future doses, and used a population management system for tracking, reminding, and rescheduling patients who needed the vaccine. We established standing orders for nurse administration of all HPV vaccine doses.
At the provider-level, we focused on improving providers’ communication skills, including standardized HPV vaccine recommendations for provider (medical doctor or nurse practitioner) or nurse “announcement” language, such as, “Your child is due for the HPV vaccine today,” rather than “Would you like your child to receive the HPV vaccine today?”27 All providers received a 45-minute training on motivation interviewing (MI), a nonconfrontational, nonjudgmental counseling style that explores and addresses ambivalence to overcome barriers to behavior change.38 The MI training encouraged provider reflection on their own concerns about HPV vaccination while increasing their confidence for engaging in difficult discussions with parents. At the patient/parent level, we posted informational posters in English and Spanish throughout the clinic, including on the back of examination room doors (for patients and parents to read while they waited), and developed informational handouts (Appendix 1 in Supplemental Information). Residents received education from their attending physicians. Vaccine rates were reviewed quarterly on run charts and distributed to the practices through monthly QI newsletters.
Methods of Evaluation
We compiled a data set from the EHR that contained records for every clinic patient who had any clinic visit during the previous year from the start of the intervention rollout period, July 1, 2014, to September 30, 2021.The data set contained dates for every HPV vaccine given in both practices. Doses of HPV vaccine obtained elsewhere were entered into the EHR manually or through the Massachusetts Immunization Information System, which includes vaccines given in all Massachusetts immunization sites.39 The data set included patient sociodemographic characteristics (birthdate, sex, race/ethnicity, and insurance status). Parents self-reported their child’s race and ethnicity data at registration for entry into standardized EHR fields.
Quality Measure
We assessed quarterly (3-month) rates of receiving 1 dose at age 9 and completion of 2 doses of HPV at age 13 for females, males, and both sexes combined. We used a visit within the last year to define the population. The denominator for each quarter was the number of patients with a birthday during that quarter who had 1 or more primary care visits within the last year in relation to the reporting quarter, and the numerator was the number of these patients who had received at least 1 dose (or all doses) of HPV vaccine. For example, in the 13th year analysis, the denominator of this measure is the number of patients who reached age 14 years during the reporting quarter with at least 1 visit during the previous 12 months. The numerator was the number of these patients who received all doses of HPV vaccine before age 14.
In September 2015 and February 2016, the team conducted 45-minute feedback sessions with clinic leadership and staff (n = 16 across all sessions) to assess the perceived acceptability (ie, positive and negative perceptions) of each intervention component. The sessions were audiotaped and transcribed; 2 independent coders used inductive content analysis to group data into themes.40
Analysis
We tracked the effects of our intervention over time using process (p) charts with control limits set at 3 α.41 Using ChartRunner expl Lean (version 3.0), we created charts for patients aged 9 and 13 years. We established a mean completion rate for each p-chart, illustrated as the center line. We established upper control limits and lower control limits. Special cause variation, defined by the standard control limits built into the Statistical quality Control chart software as 7 points beyond the center line, signaled an effect and a system change. All other statistical analyses used SAS version 9.4 software. χ2 tests were performed to examine relations between study time period and covariates (clinic size, race/ethnicity, sex). Logistic regression was used to determine which characteristics were associated with HPV vaccine adherence. For each outcome (1 dose at age 9 and series completion by age 13), unadjusted logistic regression models were used to examine the independent associations between study period, clinic size, sex, and race/ethnicity with HPV vaccine adherence. Additionally, for each outcome, a multivariate, logistic regression model adjusted for study period, clinic size, sex, and race/ethnicity was used to analyze the effects of these covariates on adherence.
Results
Outcomes
We identified 6779 children aged 10 years and 5491 children aged 14 years who had attended any visit between July 1, 2014, and March 30, 2020 (Table 2). Nearly half of the population was female (49%), with 38% non-Hispanic Black and 45% Hispanic. Most had public insurance (70%).
Population Demographics from July 2014 to March 2020
. | Patient Aged 9 Years . | Patient Aged 13 Years . | ||
---|---|---|---|---|
. | N . | % . | N . | % . |
Large clinic | 5137 | 75.8 | 3968 | 72.3 |
Small clinic | 1642 | 24.2 | 1523 | 27.7 |
Female | 3277 | 48.3 | 2724 | 49.6 |
Male | 3502 | 51.7 | 2767 | 50.4 |
Asian, non-Hispanic | 135 | 2.0 | 95 | 1.7 |
Black, non-Hispanic | 2309 | 34.1 | 2060 | 37.5 |
Hispanic | 3063 | 45.2 | 2463 | 44.8 |
White, non-Hispanic | 298 | 4.4 | 225 | 4.1 |
Other, non-Hispanic | 569 | 8.4 | 387 | 7.1 |
Multiracial, non-Hispanic | 89 | 1.3 | 45 | 0.8 |
Unknown race/ethnicity | 316 | 4.7 | 216 | 3.9 |
. | Patient Aged 9 Years . | Patient Aged 13 Years . | ||
---|---|---|---|---|
. | N . | % . | N . | % . |
Large clinic | 5137 | 75.8 | 3968 | 72.3 |
Small clinic | 1642 | 24.2 | 1523 | 27.7 |
Female | 3277 | 48.3 | 2724 | 49.6 |
Male | 3502 | 51.7 | 2767 | 50.4 |
Asian, non-Hispanic | 135 | 2.0 | 95 | 1.7 |
Black, non-Hispanic | 2309 | 34.1 | 2060 | 37.5 |
Hispanic | 3063 | 45.2 | 2463 | 44.8 |
White, non-Hispanic | 298 | 4.4 | 225 | 4.1 |
Other, non-Hispanic | 569 | 8.4 | 387 | 7.1 |
Multiracial, non-Hispanic | 89 | 1.3 | 45 | 0.8 |
Unknown race/ethnicity | 316 | 4.7 | 216 | 3.9 |
Figures 2 and 3 show p-charts for children receiving the first dose of HPV vaccine at age 9 and 2 doses of HPV vaccine by age 13 between July 1, 2014, and September 30, 2021. For all patients, receipt of the first HPV vaccine dose by age 9 increased from 1.1% to 51.8% but fell to 22.1% after March 30, 2020. Completion of the HPV vaccine series by age 13 increased from 37.4% to 77.2% between 2014 and 2021. Special cause variation was achieved by March 30, 2015, and again in September 30, 2016, signaling the postintervention phase.
Control chart for male and female receipt of HPV dose 1 at age 9 years.
Control chart for female and male completion of HPV 2-dose series by age 13 years.
Control chart for female and male completion of HPV 2-dose series by age 13 years.
Tables 3 and 4 summarizes bivariate analysis for 1 dose of HPV at age 9 and age 13 completion of the HPV series by clinic size, sex, and race/ethnicity at baseline and postintervention. Table 5 summarizes the multivariate analysis. In the postintervention phase, the small clinic was more successful at administering the first dose of HPV vaccine at age 9 years than the larger clinic (64.7% vs 47.5%; P < .001). This small clinic trend continued for completion of the HPV series by age 13 years (85.9% vs 73.6%; P < .001) and persisted in multivariate analysis (P < .001) at both age 9 and 13. By age 13, both bivariate and multivariate analysis showed significant sex differences at baseline, with females more likely to initiate the vaccine (58.2% vs 16.8%; P < .001). These differences disappeared in postintervention (female 77.7% versus male 76.2%; P = .38).
Bivariate Analysis for Receipt of First Dose of HPV Vaccine at Age 9 Years
. | Baseline Period (2013 Q1–2014 Q1), n = 1114 . | Postintervention Period (2017 Q1–2020 Q1), n = 3118 . | Postintervention Versus Baseline . | Difference in Proportion to Baseline . | Difference in Proportion to Postintervention . |
---|---|---|---|---|---|
Received First HPV Immunization at 9 years Old . | % (95% CI) . | % (95% CI) . | P . | P . | |
Overall | 1.1 (0.6–1.9) | 51.8 (50.0–53.5) | <.001 | — | — |
Clinic | |||||
Large | 1.1 (0.5–2.0) | 47.5 (45.4–49.5) | <.001 | .99 | <.001 |
Small | 1.1 (0.2–3.3) | 64.7 (61.3–68.1) | <.001 | ||
Sex | |||||
Female | 1.4 (0.6, 2.7) | 51.9 (49.3–54.4) | <.001 | .29 | .91 |
Male | 0.7 (0.2–1.9) | 51.7 (49.2–54.1) | <.001 | ||
Race/ethnicity | |||||
Asian American, non-Hispanic | 0.0 (0.0–14.9) | 56.1 (43.3–68.3) | <.001 | .25 | <.001 |
Black, non-Hispanic | 0.3 (0.0–1.5) | 46.0 (42.9–49.1) | <.001 | ||
Hispanic | 1.2 (0.2–2.1) | 58.5 (56.0–61.1) | <.001 | ||
White, non-Hispanic | 3.1 (0.0–7.3) | 40.6 (32.3–49.0) | <.001 | ||
Other, non-Hispanic | 2.7 (0.33–9.3) | 46.1 (40.2–52.1) | <.001 | ||
Multiracial, non-Hispanic | 0.0 (0.0–20.4) | 54.9 (40.3–68.9) | <.001 | ||
Unknown | 2.0 (0.1–10.9) | 44.8 (36.4–53.3) | <.001 |
. | Baseline Period (2013 Q1–2014 Q1), n = 1114 . | Postintervention Period (2017 Q1–2020 Q1), n = 3118 . | Postintervention Versus Baseline . | Difference in Proportion to Baseline . | Difference in Proportion to Postintervention . |
---|---|---|---|---|---|
Received First HPV Immunization at 9 years Old . | % (95% CI) . | % (95% CI) . | P . | P . | |
Overall | 1.1 (0.6–1.9) | 51.8 (50.0–53.5) | <.001 | — | — |
Clinic | |||||
Large | 1.1 (0.5–2.0) | 47.5 (45.4–49.5) | <.001 | .99 | <.001 |
Small | 1.1 (0.2–3.3) | 64.7 (61.3–68.1) | <.001 | ||
Sex | |||||
Female | 1.4 (0.6, 2.7) | 51.9 (49.3–54.4) | <.001 | .29 | .91 |
Male | 0.7 (0.2–1.9) | 51.7 (49.2–54.1) | <.001 | ||
Race/ethnicity | |||||
Asian American, non-Hispanic | 0.0 (0.0–14.9) | 56.1 (43.3–68.3) | <.001 | .25 | <.001 |
Black, non-Hispanic | 0.3 (0.0–1.5) | 46.0 (42.9–49.1) | <.001 | ||
Hispanic | 1.2 (0.2–2.1) | 58.5 (56.0–61.1) | <.001 | ||
White, non-Hispanic | 3.1 (0.0–7.3) | 40.6 (32.3–49.0) | <.001 | ||
Other, non-Hispanic | 2.7 (0.33–9.3) | 46.1 (40.2–52.1) | <.001 | ||
Multiracial, non-Hispanic | 0.0 (0.0–20.4) | 54.9 (40.3–68.9) | <.001 | ||
Unknown | 2.0 (0.1–10.9) | 44.8 (36.4–53.3) | <.001 |
Bivariate Analysis for Completion of HPV Series by Age 13 Years
. | Baseline period (2013 Q1–2014 Q1), n = 875 . | Maintenance period (2017 Q1–2020 Q1), n = 2529 . | Maintenance versus baseline . | Difference in proportion–baseline . | Difference in proportion–maintenance . |
---|---|---|---|---|---|
HPV series complete at 13 y old . | % (95% CI) . | % (95% CI) . | P . | P . | |
Overall | 37.4 (34.2–40.6) | 76.95 (75.3–78.6) | <.001 | — | — |
Clinic | |||||
Large | 30.5 (26.6–34.2) | 73.6 (71.6–75.6) | <.001 | <.001 | <.001 |
Small | 55.4 (48.9–61.7) | 85.86 (83.0–88.4) | <.001 | ||
Sex | |||||
Female | 58.2 (53.4–62.8) | 77.7 (75.3–79.9) | <.001 | <.001 | .38 |
Male | 16.8 (13.4–20.7) | 76.20 (73.7–78.5) | <.001 | ||
Race/ethnicity | |||||
Asian American, non-Hispanic | 23.1 (5.0–53.8) | 73.0 (55.9–86.2) | <.001 | <.001 | <.001 |
Black, non-Hispanic | 28.4 (23.7–33.4) | 73.11 (70.1–76.0) | <.001 | ||
Hispanic | 49.7 (44.7–54.8) | 83.5 (81.2–85.6) | <.001 | ||
White, non-Hispanic | 25.6 (13.5–41.2) | 58.9 (49.0–68.3) | <.001 | ||
Other, non-Hispanic | 22.2 (11.2–37.1) | 72.5 (65.7–78.7) | <.001 | ||
Multiracial, non-Hispanic | 42.9 (9.9–81.6) | 60.7 (40.6–78.5) | <.001 | ||
Unknown | 23.1 (8.97–43.65) | 72.1 (62.8–80.2) | <.001 |
. | Baseline period (2013 Q1–2014 Q1), n = 875 . | Maintenance period (2017 Q1–2020 Q1), n = 2529 . | Maintenance versus baseline . | Difference in proportion–baseline . | Difference in proportion–maintenance . |
---|---|---|---|---|---|
HPV series complete at 13 y old . | % (95% CI) . | % (95% CI) . | P . | P . | |
Overall | 37.4 (34.2–40.6) | 76.95 (75.3–78.6) | <.001 | — | — |
Clinic | |||||
Large | 30.5 (26.6–34.2) | 73.6 (71.6–75.6) | <.001 | <.001 | <.001 |
Small | 55.4 (48.9–61.7) | 85.86 (83.0–88.4) | <.001 | ||
Sex | |||||
Female | 58.2 (53.4–62.8) | 77.7 (75.3–79.9) | <.001 | <.001 | .38 |
Male | 16.8 (13.4–20.7) | 76.20 (73.7–78.5) | <.001 | ||
Race/ethnicity | |||||
Asian American, non-Hispanic | 23.1 (5.0–53.8) | 73.0 (55.9–86.2) | <.001 | <.001 | <.001 |
Black, non-Hispanic | 28.4 (23.7–33.4) | 73.11 (70.1–76.0) | <.001 | ||
Hispanic | 49.7 (44.7–54.8) | 83.5 (81.2–85.6) | <.001 | ||
White, non-Hispanic | 25.6 (13.5–41.2) | 58.9 (49.0–68.3) | <.001 | ||
Other, non-Hispanic | 22.2 (11.2–37.1) | 72.5 (65.7–78.7) | <.001 | ||
Multiracial, non-Hispanic | 42.9 (9.9–81.6) | 60.7 (40.6–78.5) | <.001 | ||
Unknown | 23.1 (8.97–43.65) | 72.1 (62.8–80.2) | <.001 |
CI, confidence interval.
Multivariate Analysis for First Dose of HPV Vaccine at Age 9 Years and Completion of HPV Series by Age 13 Years
Predictor . | OR . | 95% CI . | P . | OR . | 95% CI . | P . |
---|---|---|---|---|---|---|
. | . | Unadjusted . | . | . | Adjusted . | . |
First HPV immunization at 9 years old | ||||||
Study period | ||||||
Baseline | Reference | Reference | ||||
Intervention | 45.5 | 25.6–80.9 | <.0001 | 47.9 | 27.0–59.3 | <.0001 |
Postintervention | 98.5 | 55.6–174.8 | <.0001 | 105.4 | 59.3–187.1 | <.0001 |
Clinic | ||||||
Large | Reference | Reference | ||||
Small | 1.9 | 1.7–2.1 | <.0001 | 1.7 | 1.5–2.0 | <.0001 |
Sex | ||||||
Male | Reference | Reference | ||||
Female | 1 | 0.9–1.1 | .92 | 1 | 0.9–1.2 | .49 |
Race/ethnicity | ||||||
White, non-Hispanic | Reference | Reference | ||||
Asian American, non-Hispanic | 1.9 | 1.2–3.0 | .2 | 1.9 | 1.2–3.1 | .11 |
Black, non-Hispanic | 1.4 | 1.1–1.9 | .07 | 1.4 | 1.0–1.8 | .32 |
Hispanic | 2.3 | 1.7–3.0 | <.0001 | 1.9 | 1.4–2.6 | <.0001 |
Multiracial, non-Hispanic | 1.8 | 1.2–3.0 | .47 | 1.6 | 0.9–1.8 | .73 |
Other, non-Hispanic | 1.5 | 1.1–2.1 | .61 | 1.3 | 0.9–2.0 | .22 |
Unknown | 1.4 | 1.0–2.1 | .44 | 1.4 | 0.9–2.0 | .55 |
HPV series complete at 13 y old | ||||||
Study period | ||||||
Baseline | Reference | Reference | ||||
Intervention | 3.1 | 2.6–3.6 | <.0001 | 3.4 | 2.8–4.0 | <.0001 |
Maintenance | 5.6 | 4.7–6.6 | <.0001 | 6.4 | 5.4–7.6 | <.0001 |
Clinic | ||||||
Large | Reference | Reference | ||||
Small | 2.5 | 2.2–2.9 | <.0001 | 2.1 | 1.8–2.5 | <.0001 |
Sex | ||||||
Male | Reference | Reference | ||||
Female | 1.7 | 1.5–1.9 | <.0001 | 1.8 | 1.6–2.0 | <.0001 |
Race/ethnicity | ||||||
White, non-Hispanic | Reference | Reference | ||||
Asian American, non-Hispanic | 1.1 | 0.7–1.8 | .23 | 1 | 0.6–1.7 | .41 |
Black, non-Hispanic | 1.4 | 1.1–1.9 | .78 | 1.4 | 1.0–1.9 | .11 |
Hispanic | 2.8 | 2.2–3.8 | <.0001 | 2.3 | 1.7–3.0 | <.0001 |
Multiracial, non-Hispanic | 1.1 | 0.6–2.1 | .36 | 0.9 | 0.4–1.7 | .21 |
Other, non-Hispanic | 1.4 | 1.0–2.0 | .75 | 1.2 | 0.9–1.8 | .99 |
Unknown | 1.4 | 1.0–2.0 | .97 | 1.2 | 0.8–1.8 | .92 |
Predictor . | OR . | 95% CI . | P . | OR . | 95% CI . | P . |
---|---|---|---|---|---|---|
. | . | Unadjusted . | . | . | Adjusted . | . |
First HPV immunization at 9 years old | ||||||
Study period | ||||||
Baseline | Reference | Reference | ||||
Intervention | 45.5 | 25.6–80.9 | <.0001 | 47.9 | 27.0–59.3 | <.0001 |
Postintervention | 98.5 | 55.6–174.8 | <.0001 | 105.4 | 59.3–187.1 | <.0001 |
Clinic | ||||||
Large | Reference | Reference | ||||
Small | 1.9 | 1.7–2.1 | <.0001 | 1.7 | 1.5–2.0 | <.0001 |
Sex | ||||||
Male | Reference | Reference | ||||
Female | 1 | 0.9–1.1 | .92 | 1 | 0.9–1.2 | .49 |
Race/ethnicity | ||||||
White, non-Hispanic | Reference | Reference | ||||
Asian American, non-Hispanic | 1.9 | 1.2–3.0 | .2 | 1.9 | 1.2–3.1 | .11 |
Black, non-Hispanic | 1.4 | 1.1–1.9 | .07 | 1.4 | 1.0–1.8 | .32 |
Hispanic | 2.3 | 1.7–3.0 | <.0001 | 1.9 | 1.4–2.6 | <.0001 |
Multiracial, non-Hispanic | 1.8 | 1.2–3.0 | .47 | 1.6 | 0.9–1.8 | .73 |
Other, non-Hispanic | 1.5 | 1.1–2.1 | .61 | 1.3 | 0.9–2.0 | .22 |
Unknown | 1.4 | 1.0–2.1 | .44 | 1.4 | 0.9–2.0 | .55 |
HPV series complete at 13 y old | ||||||
Study period | ||||||
Baseline | Reference | Reference | ||||
Intervention | 3.1 | 2.6–3.6 | <.0001 | 3.4 | 2.8–4.0 | <.0001 |
Maintenance | 5.6 | 4.7–6.6 | <.0001 | 6.4 | 5.4–7.6 | <.0001 |
Clinic | ||||||
Large | Reference | Reference | ||||
Small | 2.5 | 2.2–2.9 | <.0001 | 2.1 | 1.8–2.5 | <.0001 |
Sex | ||||||
Male | Reference | Reference | ||||
Female | 1.7 | 1.5–1.9 | <.0001 | 1.8 | 1.6–2.0 | <.0001 |
Race/ethnicity | ||||||
White, non-Hispanic | Reference | Reference | ||||
Asian American, non-Hispanic | 1.1 | 0.7–1.8 | .23 | 1 | 0.6–1.7 | .41 |
Black, non-Hispanic | 1.4 | 1.1–1.9 | .78 | 1.4 | 1.0–1.9 | .11 |
Hispanic | 2.8 | 2.2–3.8 | <.0001 | 2.3 | 1.7–3.0 | <.0001 |
Multiracial, non-Hispanic | 1.1 | 0.6–2.1 | .36 | 0.9 | 0.4–1.7 | .21 |
Other, non-Hispanic | 1.4 | 1.0–2.0 | .75 | 1.2 | 0.9–1.8 | .99 |
Unknown | 1.4 | 1.0–2.0 | .97 | 1.2 | 0.8–1.8 | .92 |
CI, confidence interval; OR, odds ratio.
During postintervention for both initiation at age 9 and series completion by age 13, a significant difference among racial/ethnic groups was observed (P < .001). Hispanic children had highest rates of initiation at 9 years (58.5%) and highest rates for completing the series by 13 years in both baseline (49.7%) and postintervention (83.5%) periods. This trend persisted in multivariate analysis adjusting for clinic size, sex, and study period (P < .001).
Qualitative Staff Feedback
Staff feedback sessions (Supplemental Table 6) showed positive perceptions toward clinicwide protocols standardizing language recommending the vaccine to patients/parents and beginning the vaccine series at age 9. Staff endorsed standing orders for HPV vaccine. The evening shot clinic and phone reminders had more barriers than benefits, with neither perceived as increasing immunization rates. Providers had mixed feedback for some components with promise but needing improvement. They rated MI training positively, but received insufficient skills to use MI effectively for vaccine refusal. Providers suggested more intensive MI trainings. They noted logistical issues with the appointment reminder system. Using this feedback, we focused on nursing roles with standing orders, use of announcement language, and initiating the vaccine at age 9 in the postintervention period.
Discussion
In our multilevel intervention, we met our goal to improve rates of HPV vaccination for racially and ethnically diverse girls and boys over a sustained period of time. Our approach included patient-, provider-, and systems-level changes at 2 practices, 1 large and 1 small, to improve completion of a 2-dose HPV vaccine schedule by age 13 years. During the study, the HPV vaccine completion rate at 13 years increased from 37.4% to 77.2%, exceeding Massachusetts rates at 13 to 17 years. Nationally, HPV vaccine completion at 13 years only increased from 22.1% to 48.4% from 2013 to 2020.11 During the first 18 months of the pandemic, HPV vaccine completion rates were maintained even though limitations on clinic well care negatively impacted initiation of the vaccine at age 9. Our intervention trained staff on MI techniques, and streamlined clinical systems.31,36,37,39 Empowering nurses to initiate vaccinations using standard protocols was a key component. We found that the smaller clinic with higher nurse-to-provider ratios achieved greater gains in HPV vaccine rates. Both clinics were comparable because they shared clinic leadership and providers. Uptake of the intervention may have been better at the smaller clinic with fewer staff and trainees. In qualitative feedback, providers were generally positive about the intervention, especially regarding the clinicwide components, but highlighted the need for more intensity of training.
Our multilevel study extended and combined the strengths of previous research in its inclusion of Black and Hispanic girls and boys, as well as children as young as age 9, and we examined a population rather than a convenience sample over an extended period of time into the COVID-19 pandemic. Most previous studies have not studied HPV series completion at age 13. This is important because risk for HPV infection begins in early adolescence. Multilevel studies have used provider communication training, data feedback, systems changes, family education with videos, or other education materials, nurse prompts, and decision aides.21,37,39 In the postintervention, there were no sex disparities in HPV vaccine completion, although, in the United States, males continue to be less likely to complete the HPV series.11
Our focus on a predominantly Black and Hispanic population is important given racial/ethnic disparities in both HPV infection and rates of HPV vaccination that were prominent at the beginning of our intervention.11,12 Although these disparities narrowed for all patients, we found that Hispanic children improved the most for initiating the vaccine at age 9 and completing the series at age 13, potentially reflecting lower vaccine confidence within other racial/ethnic groups. This aligns with findings in our design phase qualitative study that Black adolescents and their parents were concerned about this vaccine being untested and new.36 Hispanic families may be more accepting of vaccination because of high cervical cancer rates in Hispanic females.3 There may also be neighborhood characteristics that we did not measure influencing HPV vaccine acceptance. This racial/ethnic variability in vaccine acceptance needs further study.
A culture of improvement with accountability, standardization, integration of tasks into daily processes, and data feedback helped sustain our interventions’ improvements.42 The practices’ QI committee continuously reviewed and communicated HPV vaccine completion rates. Qualitative feedback supported focusing on nursing protocols, standardized communication, and vaccine initiation at age 9 in the postintervention period, when vaccine completion continues to rise. Early vaccine initiation explains the sustained HPV completion rates observed during the pandemic, when care shifted significantly to telemedicine.
Limitations of the study include testing the intervention in only 2 clinics in 1 metropolitan area of the United States, lack of a randomized controlled design, and inability to account for the impact of national HPV prevention strategies through media and other modalities on vaccine acceptance. However, our improvements far exceeded national trends. We anticipate that many components could be replicated in other similar practice settings with appropriate consideration for local adaptation. We may have missed identifying doses of HPV given outside of Massachusetts or not entered into the Massachusetts Immunization Information System. Race/ ethnicity data were missing for 4.7% of cases, and it is unknown whether missingness varied by race/ethnicity. There may have been differences in study clinics other than size and staffing ratios that we did not identify. We did not measure individual nurse or provider adherence to the intervention, and did not compare pediatricians versus residents. By using a multilevel intervention, we were unable to identify which aspects were most important. Our intervention was initially designed for a 3-dose HPV schedule, and the Centers for Disease Control and Prevention recommendations changed in October 2016 to a 2-dose schedule, which could be completed at 2 annual visits. To simplify the evaluation, we chose to report only on the 2-dose series. Qualitative feedback suggested that creating nurse protocols was highly effective. Although we also obtained postintervention provider feedback, we were not able to obtain family feedback. Finally, no data were obtained on cost of training and implementation of the intervention.
In conclusion, our study demonstrated increased uptake of the HPV vaccine using a QI approach that addressed individual-, provider-, and systems-level factors. The intervention was feasible, acceptable to providers, and successful at increasing HPV immunization, especially for Hispanic children. Individual intervention initiatives can be applied to improving uptake of other vaccines, including COVID-19. Future research should investigate racial/ethnic differences in vaccine acceptability. Testing the effectiveness of our promising intervention using a clustered randomized design in other communities could facilitate examining neighborhood factors that affect implementation and vaccine adherence. A cost–benefit analysis study may help understand intervention impact on cost and revenue.
Dr Cox participated in the design of the study and intervention, helped to interpret the data, and drafted and revised the manuscript; Dr Bogart conceptualized and designed the study and intervention, helped to interpret the data, and contributed to drafting and revising the manuscript; Dr Elliott helped to design the study and interpret the data, and helped to draft and revise the manuscript; Dr Starmer helped design the intervention, helped with data analysis, and helped to draft and revise the manuscript; Ms Meleedy-Rey analyzed data, created process control charts, and helped to draft and revise the manuscript; Ms Banerjee helped design the study, coordinated the collection of data, helped to interpret the data, and helped to draft and revise the manuscript; Dr Goggin helped to design the intervention, interpret the data, and to draft and revise the final manuscript; Dr Samuels helped to implement the intervention, to interpret the data, and to draft the final manuscript; Mr Hahn analyzed the data and helped to draft and revise the manuscript; Dr Epee-Bounya helped to design the intervention, to interpret the data, and to draft the final manuscript; Dr Allende-Richter helped to implement the intervention, and helped to draft and revise the final manuscript; Ms Fu helped to design and implement the intervention, to interpret the data, and to draft and revise the final manuscript; Dr Schuster conceptualized and designed the study and intervention, helped to interpret the data, and contributed to drafting and revising the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: Funded by the National Institute of Minority Health and Health Disparities (supplement to 2 R24 MD001648 MA Schuster principal investigator). Funded by the National Institutes of Health (NIH).
CONFLICT OF INTEREST DISCLAIMER: The authors have indicated they have no conflicts of interest relevant to this article to disclose.
Comments