BACKGROUND AND OBJECTIVES:

Human papillomavirus (HPV) vaccination rates lag behind vaccination rates for other adolescent vaccines; a bundled intervention may improve HPV vaccination rates. Our objective is to evaluate the impact of quality improvement (QI) training plus a bundled practice-based intervention (provider prompts plus communication skills training plus performance feedback) on improving HPV vaccinations in pediatric resident continuity clinics.

METHODS:

Staff and providers in 8 resident clinics participated in a 12-month QI study. The intervention included training to strengthen provider communication about the HPV vaccine. Clinics also implemented provider prompts, received monthly performance feedback, and participated in learning collaborative calls. The primary outcome measure was eligible visits with vaccination divided by vaccine-eligible visits (captured HPV vaccination opportunities). Practices performed chart audits that were fed into monthly performance feedback on captured HPV vaccination opportunities. We used conditional logistic regression (conditioning on practice) to assess captured vaccination opportunities, with the time period of the study (before and after the QI intervention) as the independent variable.

RESULTS:

Overall, captured opportunities for HPV vaccination increased by 16.4 percentage points, from 46.9% to 63.3%. Special cause was demonstrated by centerline shift, with 8 consecutive points above the preintervention mean. On adjusted analyses, patients were more likely to receive a vaccine during, versus before, the intervention (odds ratio: 1.87; 95% confidence interval: 1.54–2.28). Captured HPV vaccination rates improved at both well-child and other visits (by 11.7 and 13.0 percentage points, respectively).

CONCLUSIONS:

A bundled intervention of provider prompts and training in communication skills plus performance feedback increased captured opportunities for HPV vaccination.

Most cases of cervical cancer1 and >60% of oropharyngeal cancers are related to human papillomavirus (HPV) infection.2 Despite the vaccine having been recommended for adolescent girls since 20073 and for boys since 2011,4 HPV vaccination rates in the United States lag behind those for other adolescent vaccines. The Centers for Disease Control and Prevention, the President’s Cancer Panel from 2012 to 2013, and conductors of systematic reviews5,7 have recommended reducing missed opportunities (MOs) to administer HPV vaccines to increase HPV vaccination coverage.8,9 

Several strategies might reduce MOs. Provider prompts is 1 strategy, yet few researchers have evaluated prompts for the HPV vaccine. One randomized clinical trial revealed that prompts combined with practice-based performance feedback reduced MOs and improved HPV vaccination rates.10,11 However, a second randomized clinical trial12 revealed that prompts alone reduced MOs slightly in 1 of 2 practice networks only and failed to raise HPV vaccination rates substantially. It is implied in these results that prompts alone may not be sufficient to raise vaccination rates. A second strategy relates to the finding that parents often report that the HPV vaccine was not recommended by their adolescent’s physician.9 Thus, an additional potential strategy is to strengthen provider recommendations.8 A third possible intervention is performance feedback for providers. Performance feedback can change provider behavior for many health services,13,14 including vaccination,11,15 because physicians do not accurately evaluate their own performance.16 Its effectiveness depends on how the feedback is provided; feedback is more effective when provided more than once, in multiple formats, and when it includes a goal.14 Therefore, in this quality improvement (QI) study, we combine provider prompts, training in giving a strong recommendation for the HPV vaccine, and performance feedback, with a goal of reducing MOs for HPV vaccination.

We conducted a QI study over a 12-month period in 8 practices from a national network of pediatric resident continuity clinics that are based at academic medical centers. Whereas much of the literature concerns MOs, our outcome was captured opportunities (eligible visits with HPV vaccination divided by HPV vaccine–eligible visits), which are the inverse of MOs (to display improvement as a positive change). Our specific aim in the project was to increase captured opportunities for HPV vaccination by 10%. A secondary goal was to improve documentation (by 10%) of reasons why the HPV vaccine was not given when due, to better identify issues leading to under-vaccination. We also performed a qualitative analysis of monthly practice diaries in which Plan-Do-Study-Act (PDSA) cycles were documented to assess office changes that were associated with greater success in improving captured opportunities.

This 12-month QI study was conducted in a national, practice-based research network that is composed of pediatric resident continuity practices (Continuity Research Network [CORNET]).17 An e-mail newsletter used to encourage participation was distributed to all 107 CORNET practices, with a goal of enrolling 12 to 14 practices overall. Eight CORNET practices volunteered to participate in the prompt intervention (5 other practices participated in a standing order intervention, described elsewhere). Maintenance of certification credit was provided for faculty as an incentive to participate, in addition to $1000 per practice to compensate for time needed for chart reviews. At the time of the baseline period, none of the practices had ways to prompt providers when an HPV vaccination was due for a patient, and vaccination was not generally recommended outside of well visits.

Basic QI Training

Initial QI training involved four 15-minute videos18 used to introduce the following topics: the model for improvement, leading change, initiating a QI project, and QI measurement. All videos focused on practice improvement in primary care.

Strong Provider Recommendation

Participants watched a 30-minute webinar on “Making a Strong Recommendation About HPV Vaccine to Parents in Primary Care,” based on slides created by the Centers for Disease Control and Prevention.19 The webinar covered communication strategies including: (1) recommending the HPV vaccine in the same way and at the same visit as other recommended adolescent vaccines, (2) providing accurate and succinct answers to frequently asked questions about the vaccine, and (3) training the entire office team to feel comfortable talking about the HPV vaccine to provide consistent messages supporting HPV vaccination.

Provider Prompts

Office staff members implemented prompts from nurses to remind the providers to give the HPV vaccine to adolescents who were due at any type of visit. Because it was already common practice to review vaccinations at well visits, the prompt intervention targeted acute and chronic care visits. Nurses or medical assistants were expected to review the vaccination record before or at every visit for any adolescent and to prompt the provider to order the vaccine. The prompt mechanisms varied by practice (eg, Vaccine Information Statement [VIS] forms placed on the clinician’s desk in the examination room; a laminated, erasable sign that was used to indicate which vaccines were due; a highlighted patient schedule for the day; printed immunization registry data). Some practices also used electronic health record (EHR) decision support in addition to the nurse prompt, but this was not the sole cue at any site. In addition, practices used visual cues (ie, door signs and posters) to raise awareness of the project for staff, clinicians, and parents.

Performance Feedback

Practices performed monthly chart reviews (10 charts per month)20 for adolescents aged 11 to 17 years who were due for an HPV vaccine at the visit. Practices received monthly feedback reports with run charts describing the overall (across 8 sites) and site-specific progress for HPV vaccine captured opportunities; they were asked to share these reports with providers and staff. Each practice also filled out a diary, describing specific changes the practice worked on during each month (using PDSA cycles).

Monthly Learning Collaborative Telephone Calls

We conducted monthly calls via webinar with intervention practices to teach improvement science, drive PDSA cycles, share practice run charts, and discuss barriers and successes with strong provider recommendations as well as prompt implementation. Learning collaborative (LC) faculty provided coaching to address obstacles and encourage those meeting their objectives.

Improved Documentation

Practices also worked on improving documentation if a vaccine was refused; they edited their EHR note templates to encourage documentation. When possible, they added the following choices to document when a patient did not get an HPV vaccine when due: refused, referred to later date, or unable to obtain consent.

Practice Teams

In addition to nurses, medical assistants, and pediatric residents, each practice team included at least 1 QI lead who was a continuity clinic attending physician. At some sites, receptionists were included for specific parts of the intervention (ie, scheduling follow-up appointments). The practice teams met monthly to review data and plan PDSA cycles. They communicated with other providers in the practice via e-mail and educational sessions for staff and residents.

Academic Pediatric Association Core Team

The project was led by individuals with expertise in immunization delivery (C.A., C.M.R., S.G.H., S.J.S., P.G.S.), QI (K.J.M., W.S.), and CORNET directors and staff (P.M.D., J.R.S., N.D.). Two immunization delivery experts (C.M.R., S.J.S.) led the monthly LC calls.

The target population was all adolescents aged 11 to 17 years who had a visit to a participating practice during the 12-month intervention period (baseline data were for those with visits in the 12 months before intervention). Visits were considered vaccine-eligible if the adolescent was eligible for the HPV vaccine at the visit on the basis of the 2013 Advisory Committee on Immunization Practices guidelines21; that is, HPV dose 1 if none previously, HPV dose 2 if >30 days from HPV dose 1, and HPV dose 3 if >24 weeks from HPV dose 1 and >12 weeks from HPV dose 2.

The primary outcome measure was captured HPV vaccination opportunities (the number of eligible visits with vaccination divided by the number of vaccine-eligible visits). Practices gathered retrospective data for the 12 months before the intervention (10 charts per month) and monthly during the intervention period by using randomly selected charts to report on patient characteristics, visit types, captured opportunities, and the reasons patients were not vaccinated if due (process measure). Reasons were coded as refused, deferred to later date, not mentioned and/or unclear (missing data), or other.

We assessed (1) captured opportunities for HPV vaccination by practice for all visits and by visit type and (2) reasons patients were not immunized if due. We compared the baseline period to intervention periods by using descriptive statistics and Pearson’s χ2 test. We also assessed the odds of HPV vaccination at the index visit using conditional logistic regression and conditioning on practice, with the time period of the study as the independent variable.

The primary outcome measure was plotted on a statistical process control chart (P-chart) by using Microsoft Excel QI Macros. Upper and lower control limits were set at 3 σ. Special cause (evidence of a change in the system) was noted when 8 consecutive data points were above the preintervention mean.

One researcher (C.M.R.) reviewed the monthly practice diaries to examine common themes and better understand the relationship between practice changes and improvement in captured opportunities.

The Research Subjects Review Board of the University of Rochester approved the project; the 8 CORNET sites obtained institution-specific institutional review board approval.

In Table 1, we show patient demographics in the 8 continuity clinics. Most (7 out of 8) practices were hospital-based, 5 were in urban settings, and all participated in the Vaccines for Children program; most patients were publicly insured.

TABLE 1

Practice Demographics

DemographicValue
Practice, n (%) (N = 8 practices) 
 Hospital-based clinic 7 (88%) 
 Private- and/or office-based 1 (12%) 
Primary patient population, n (%)  
 Suburban 3 (38%) 
 Urban 5 (63%) 
Use of an EHR, n (%)  
 Yes 6 (75%) 
 No 2 (25%) 
No. adolescent patients, mean (SD) 2224 (1748) 
Patient insurance type, mean (SD)  
 Public 68.9 (27.8) 
 Private 23.9 (28.7) 
 Military 1.6 (1.9) 
 None 5.5 (7.6) 
 Other 1.5 (3.5) 
DemographicValue
Practice, n (%) (N = 8 practices) 
 Hospital-based clinic 7 (88%) 
 Private- and/or office-based 1 (12%) 
Primary patient population, n (%)  
 Suburban 3 (38%) 
 Urban 5 (63%) 
Use of an EHR, n (%)  
 Yes 6 (75%) 
 No 2 (25%) 
No. adolescent patients, mean (SD) 2224 (1748) 
Patient insurance type, mean (SD)  
 Public 68.9 (27.8) 
 Private 23.9 (28.7) 
 Military 1.6 (1.9) 
 None 5.5 (7.6) 
 Other 1.5 (3.5) 

As shown in the P-chart of monthly chart reviews (Fig 1), captured opportunities for HPV vaccination increased by 16.4 percentage points (from 46.9% to 63.3%) overall. Special cause was demonstrated by centerline shift after the intervention.

FIGURE 1

P-chart of captured opportunities for HPV vaccine, monthly chart reviews. CL, control limit; LCL, lower control limit; UCL, upper control limit.

FIGURE 1

P-chart of captured opportunities for HPV vaccine, monthly chart reviews. CL, control limit; LCL, lower control limit; UCL, upper control limit.

Comparing the intervention period to the baseline period, we found that adolescents were more likely to receive a vaccine at an eligible visit during the intervention than before the intervention (odds ratio: 1.87; 95% confidence interval: 1.54–2.28). HPV captured vaccination rates improved at both well child (from 65% to 77%) and other visits (from 27% to 40%), with P < .001. Improvement in captured opportunities varied by practice, from 2.5 to 30 percentage points (Fig 2). Practices with the lowest initial rates made the least improvement; practices with the middle range at baseline made the most improvement. Documentation of refusals and deferring vaccination increased from 9.3% to 17.1% and from 5% to 12.8%, respectively, whereas no documentation for an MO declined from 80.2% to 65.5%, with P < .001 overall.

FIGURE 2

Captured opportunities for HPV vaccination by practice.

FIGURE 2

Captured opportunities for HPV vaccination by practice.

Monthly diaries were used to describe the changes in the prompt intervention, barriers encountered, parent responses, and lessons learned (Table 2). Barriers included nursing shortages, staff changes, and decreased response to the prompts at busy periods. A minority of parents continued to be resistant to vaccination, and safety was a concern that was raised by several parents. Lessons learned from successful practices included sending frequent communications about the project and meeting frequently about practice change.

TABLE 2

Qualitative Responses From Monthly Diaries

Improvement in Captured Opportunities (No. Practices)Changes in Prompt InterventionBarriers and How They Were AddressedParent ResistanceLessons Learned
25%–30% (3) • Laminated cards, VIS sheet on fluorescent paper, changing wall color sign, door flag • Nursing shortages; intervention improved as staffing improved • Some parents still prefer to defer to the well-care visit, want to discuss with a spouse, or wait until their child is older • Sending frequent reminders every month helped improve outcomes 
• Reminders to nurses to review and document • Priority to update new staff and review data • Patient sometimes not accompanied by the primary caregiver; try to obtain phone consent or schedule a nurse visit with the primary caregiver as soon as possible • Improvement occurred when nurse manager helped coordinate the project 
• Posted HPV process flowchart • When busy, constant reminders needed to get staff back on track   
6%–11% (3) • Laminated reminder signs used to promote the vaccine in the physician’s workroom, using door flags in addition to visits sheets • Staff changes; untrained substitutes assigned to the clinic; no vaccine reminders were provided • Continued resistance from a small minority of parents regarding questions of vaccine safety and necessity • Early, persistent reminders are helpful in long-term change and outcomes 
• Editing the note template to improve documentation of refusals and prompt education resources given • When new interns arrived, vaccines were often missed   
• Stamps on the printout of the immunization registry to identify that the patient was checked for HPV vaccine need (frequent reminders needed to use stamps)    
<5% (2) • Nurse attaches VIS sheet and puts “needs HPV” in intake note • When project champion was out, less prompting occurred • Parents concerned about young age and giving the vaccine to boys, think they can wait; sometimes want to talk to spouse • Generic prompt (review vaccines) had no impact 
• Incentivized nurses, reinforced importance • Staff notes vaccine due but doesn’t discuss unless parent asks • Concerns about vaccine safety • Repetition is important 
• Nurse puts prompt in intake of EHR • Prompting irregular; depends on staff and how busy it is  • QI Team needs to meet more often to initiate changes 
• LED light on computer for nurse to turn on • Changing behavior met with some resistance   
 • Some clinicians sympathized with the parent’s reluctance to vaccinate at a sick visit   
Improvement in Captured Opportunities (No. Practices)Changes in Prompt InterventionBarriers and How They Were AddressedParent ResistanceLessons Learned
25%–30% (3) • Laminated cards, VIS sheet on fluorescent paper, changing wall color sign, door flag • Nursing shortages; intervention improved as staffing improved • Some parents still prefer to defer to the well-care visit, want to discuss with a spouse, or wait until their child is older • Sending frequent reminders every month helped improve outcomes 
• Reminders to nurses to review and document • Priority to update new staff and review data • Patient sometimes not accompanied by the primary caregiver; try to obtain phone consent or schedule a nurse visit with the primary caregiver as soon as possible • Improvement occurred when nurse manager helped coordinate the project 
• Posted HPV process flowchart • When busy, constant reminders needed to get staff back on track   
6%–11% (3) • Laminated reminder signs used to promote the vaccine in the physician’s workroom, using door flags in addition to visits sheets • Staff changes; untrained substitutes assigned to the clinic; no vaccine reminders were provided • Continued resistance from a small minority of parents regarding questions of vaccine safety and necessity • Early, persistent reminders are helpful in long-term change and outcomes 
• Editing the note template to improve documentation of refusals and prompt education resources given • When new interns arrived, vaccines were often missed   
• Stamps on the printout of the immunization registry to identify that the patient was checked for HPV vaccine need (frequent reminders needed to use stamps)    
<5% (2) • Nurse attaches VIS sheet and puts “needs HPV” in intake note • When project champion was out, less prompting occurred • Parents concerned about young age and giving the vaccine to boys, think they can wait; sometimes want to talk to spouse • Generic prompt (review vaccines) had no impact 
• Incentivized nurses, reinforced importance • Staff notes vaccine due but doesn’t discuss unless parent asks • Concerns about vaccine safety • Repetition is important 
• Nurse puts prompt in intake of EHR • Prompting irregular; depends on staff and how busy it is  • QI Team needs to meet more often to initiate changes 
• LED light on computer for nurse to turn on • Changing behavior met with some resistance   
 • Some clinicians sympathized with the parent’s reluctance to vaccinate at a sick visit   

Regarding a strong provider recommendation, providers noted that staff recommendations were not as strong as expected at baseline but appeared to improve because of the intervention. Practice comments toward the end of the study were as follows:

The staff continues to get resistance from some parents. Overall seems less over the past 6 months. The 2 areas for this are: (1) my child is not sexually active yet, “I’ll wait.” and (2) The thought that the HPV is only for girls to prevent cervical cancer only; there is a hesitancy to give it to their sons. Some staff have stepped up their education and overall approach to the vaccine. I believe this change in staff has led to overall less resistance from the parents.

[We have] much less hesitancy or resistance than in the earlier part of the year. It now feels like the HPV is automatic, not just an add-on.

Practices with little improvement (<5%) lacked consensus for the QI activity from staff and providers. In addition, more complicated interventions (such as having the nurse place an LED light on the computer) were difficult to sustain. The simplest interventions, such as putting the VIS sheet on fluorescent paper, led to greater improvement (>25%). In addition, successful practices made it a priority to update new staff and to review monthly data with the QI team.

In this QI study in pediatric resident continuity clinics, we found that a bundled intervention consisting of a combination of provider prompts from nurses and EHRs, provider and staff training in giving a strong recommendation for the vaccine, and performance feedback to providers increased rates of captured opportunities for HPV vaccination by 16 percentage points. However, there was great variation in improvement by practice, ranging from 2.5 to 30 percentage points. In addition, absence of documentation for MOs declined from 80% to 66%.

The range of improvement for practices was broad and provides lessons for practice improvement. Practices with the greatest increases in captured opportunities used the following approaches: frequent communications for staff and residents, changing EHR note templates to improve documentation, using multimodal prompts (paper, VIS, EHR), and having a nurse champion who reviewed medical records ahead of time. By improving documentation, practices have a better understanding of causes of under-vaccination. A previous study that was focused solely on EHR prompts without other interventions was not effective.12 This may be because of “prompt fatigue,”22,23 indicating the necessity to use multimodal cues. However, prompts need to be simple to be sustainable. For example, a fluorescent VIS sheet worked well as a prompt, whereas stamping the immunization registry printout had an intermediate effect. We also believe that the addition of performance feedback to prompts contributed to the large effects. In addition, practice champions can improve adherence to guidelines, including hand hygiene,24 vaccine delivery,25 and infection prevention.26 Finally, the success of our intervention supports other studies in which researchers have found that multicomponent interventions are more effective27,28 than single-component ones.

Two practices with the lowest baseline rates of captured opportunities experienced the least improvement. In 1 case, the barriers were potentially identifiable a priori, whereas in the other practice they were not. In 1 practice, the project champions had trouble convincing all providers and nurses in the practice to support vaccinating at illness visits. Practice engagement is a critical component of successful QI initiatives.29 It is important to involve the entire office staff in the intervention30 and to engage leadership in practice change for it to be successful.31 A second practice experienced major staff turnover; hence, the intervention became a low priority. Such real-life factors can dramatically diminish practices’ abilities to improve the delivery of preventive services.32 Because primary care practices face a multitude of barriers to practice improvement,33 a potential target for future research is to identify factors that predict the likelihood of improvement from modifiable practice features.

At the other extreme, the highest performing practice at the baseline period also experienced a low rate of improvement in captured opportunities. At the high range of performance, there is minimal room for improvement, and alternative strategies may be needed to reach the remaining few MOs. This is consistent with other studies that have revealed that high-performing practices make fewer advances than midrange performers in QI interventions.34,35 

The highest rate of captured opportunities in any intervention practice was 84%, with an average of 63%. We believe rates of captured opportunities are unlikely to reach 100% for HPV vaccination in most practices because of several factors. First, parent delay and refusal are likely to contribute to ∼20% of HPV under-vaccination.36 Although a strong recommendation can impact vaccine refusals, as shown by a 12% increase in captured opportunities at well visits, other strategies are needed to reach the most vaccine-hesitant parents. Second, a few patients will be too sick to be vaccinated. Lastly, some days are likely to be too busy to allow time for nurses to assess vaccination status or to draw up and administer vaccines during acute visits. At busy times in the practice, such as during influenza season, the staff had particular difficulty reviewing vaccination records from multiple sources, and, consequently, some practices noted a drop in nurse prompts. During times of stress, practices may need to adjust the intervention.29 

One strength of this study was the multidisciplinary nature of the practice teams, which included attending physicians, residents, nurses, and staff. Collaborating with staff from different areas in a practice has been shown to enhance teamwork and communication and counters the “silo effect.”37 In particular, training residents in both QI and in providing a strong recommendation is important because residents tend to use the skills learned during residency during their later practice.38,39 A second strength was having a multisite network of practice-based teams that allowed providers to learn from one another.40 Such LC models are now increasingly used for a variety of health topics. A third strength is potential sustainability, which was enhanced by providers earning Maintenance of Certification Part IV credit for participation; this has been shown to motivate physicians to participate in QI for HPV vaccination.41 Finally, the 8 practices involved in the intervention care for a large number of adolescents, so this intervention has the potential to impact a sizeable population.

There are several characteristics of the setting and patient population that may limit generalizability. The intervention was performed in pediatric resident continuity clinics serving mostly low-income populations; findings may not be generalizable to other settings. Academic practices may be particularly motivated to implement newer guidelines and are expected to teach current recommendations. On the other hand, these large practices with rotating residents are complex environments in which to implement practice change. There may be selection bias in practices volunteering to participate. The practices received a modest incentive for performing the randomly selected chart reviews. Also, the research and LC faculty leads were supported by grant funding. To scale up this intervention, we would need to integrate such core funding into local, state, or national QI efforts.

In this QI study, we demonstrate that a bundled intervention consisting of multimodal provider prompts combined with training on the delivery of a strong provider recommendation for HPV vaccination and performance feedback to providers can substantially increase captured opportunities for HPV vaccination in pediatric resident continuity clinics. For best results, practices should have an engaged champion, communicate frequently with the practice team, ensure that all providers understand and support the initiative, and incorporate ongoing performance feedback. If scaled up, such a QI intervention might help raise HPV vaccination rates across the United States.

     
  • CORNET

    Continuity Research Network

  •  
  • EHR

    electronic health record

  •  
  • HPV

    human papillomavirus

  •  
  • LC

    learning collaborative

  •  
  • MO

    missed opportunity

  •  
  • P-chart

    process control chart

  •  
  • PDSA

    Plan-Do-Study-Act

  •  
  • QI

    quality improvement

  •  
  • VIS

    vaccine information statement

Dr Rand conceptualized and designed the study, contributed to analysis and interpretation of data, and drafted the initial manuscript; Dr Szilagyi conceptualized and designed the study, contributed to analysis and interpretation of data, and critically revised the manuscript; Drs Schaffer, Serwint, Darden, Humiston, Mann, and Stratbucker contributed to analysis and interpretation of data and revised the manuscript; Ms Dhepyasuwan coordinated and supervised data collection at all sites and critically reviewed the manuscript; Mr Blumkin conducted the initial analyses and reviewed and revised the manuscript; Ms Albertin contributed to the acquisition and coordination of data collection and revised the manuscript; and all authors approved the final manuscript as submitted.

FUNDING: Supported by grant 5U66IP000673 from the Centers for Disease Control and Prevention.

We appreciate the collaboration of the physicians and staff at the participating primary care practices. We thank Beth King for her assistance with this study.

1
Schiffman
M
,
Castle
PE
,
Jeronimo
J
,
Rodriguez
AC
,
Wacholder
S
.
Human papillomavirus and cervical cancer.
Lancet
.
2007
;
370
(
9590
):
890
907
[PubMed]
2
Gillison
ML
,
Chaturvedi
AK
,
Lowy
DR
.
HPV prophylactic vaccines and the potential prevention of noncervical cancers in both men and women.
Cancer
.
2008
;
113
(
suppl 10
):
3036
3046
[PubMed]
3
Markowitz
LE
,
Dunne
EF
,
Saraiya
M
,
Lawson
HW
,
Chesson
H
,
Unger
ER
;
Centers for Disease Control and Prevention (CDC)
;
Advisory Committee on Immunization Practices
.
Quadrivalent human papillomavirus vaccine: recommendations of the Advisory Committee on Immunization Practices (ACIP).
MMWR Recomm Rep
.
2007
;
56
(
RR-2
):
1
24
[PubMed]
4
Centers for Disease Control and Prevention (CDC)
.
Recommendations on the use of quadrivalent human papillomavirus vaccine in males—Advisory Committee on Immunization Practices (ACIP), 2011.
MMWR Morb Mortal Wkly Rep
.
2011
;
60
(
50
):
1705
1708
[PubMed]
5
Niccolai
LM
,
Hansen
CE
.
Practice- and community-based interventions to increase human papillomavirus vaccine coverage: a systematic review.
JAMA Pediatr
.
2015
;
169
(
7
):
686
692
[PubMed]
6
Walling
EB
,
Benzoni
N
,
Dornfeld
J
, et al
.
Interventions to improve HPV vaccine uptake: a systematic review.
Pediatrics
.
2016
;
138
(
1
):
e20153863
[PubMed]
7
Jacobson Vann
JC
,
Szilagyi
P
.
Patient reminder and patient recall systems to improve immunization rates.
Cochrane Database Syst Rev
.
2005
;(
3
):
CD003941
[PubMed]
8
The President’s Cancer Panel
. President’s cancer panel annual report 2012-2013. Accelerating HPV vaccine uptake: urgency for action to prevent cancer. 2012. Available at: http://deainfo.nci.nih.gov/advisory/pcp/annualreports/hpv/index.htm#sthash.ByHTZ4Wk.dpbs. Accessed November 7, 2016
9
Stokley
S
,
Jeyarajah
J
,
Yankey
D
, et al;
Immunization Services Division, National Center for Immunization and Respiratory Diseases, CDC
;
Centers for Disease Control and Prevention (CDC)
.
Human papillomavirus vaccination coverage among adolescents, 2007-2013, and postlicensure vaccine safety monitoring, 2006-2014–United States.
MMWR Morb Mortal Wkly Rep
.
2014
;
63
(
29
):
620
624
[PubMed]
10
Mayne
SL
,
duRivage
NE
,
Feemster
KA
,
Localio
AR
,
Grundmeier
RW
,
Fiks
AG
.
Effect of decision support on missed opportunities for human papillomavirus vaccination.
Am J Prev Med
.
2014
;
47
(
6
):
734
744
[PubMed]
11
Fiks
AG
,
Grundmeier
RW
,
Mayne
S
, et al
.
Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt.
Pediatrics
.
2013
;
131
(
6
):
1114
1124
[PubMed]
12
Szilagyi
PG
,
Serwint
JR
,
Humiston
SG
, et al
.
Effect of provider prompts on adolescent immunization rates: a randomized trial.
Acad Pediatr
.
2015
;
15
(
2
):
149
157
[PubMed]
13
Brouwers
MC
,
De Vito
C
,
Bahirathan
L
, et al
.
What implementation interventions increase cancer screening rates? A systematic review.
Implement Sci
.
2011
;
6
:
111
[PubMed]
14
Ivers
N
,
Jamtvedt
G
,
Flottorp
S
, et al
.
Audit and feedback: effects on professional practice and healthcare outcomes.
Cochrane Database Syst Rev
.
2012
;(
6
):
CD000259
[PubMed]
15
Gilkey
MB
,
Dayton
AM
,
Moss
JL
, et al
.
Increasing provision of adolescent vaccines in primary care: a randomized controlled trial.
Pediatrics
.
2014
;
134
(
2
). Available at: www.pediatrics.org/cgi/content/full/134/2/e346
[PubMed]
16
Davis
DA
,
Mazmanian
PE
,
Fordis
M
,
Van Harrison
R
,
Thorpe
KE
,
Perrier
L
.
Accuracy of physician self-assessment compared with observed measures of competence: a systematic review.
JAMA
.
2006
;
296
(
9
):
1094
1102
[PubMed]
17
Academic Pediatric Association
. Continuity Research Network (CORNET). Available at: https://www.academicpeds.org/research/research_CORNET.cfm. Accessed December 30, 2016
18
Academic Pediatric Association
. QI education modules. Available at: http://www.academicpeds.org/NPAI_QIModules/page_01.htm. Accessed January 26, 2018
19
Centers for Disease Control and Prevention
. You are the key to HPV cancer prevention–train the trainer. Available at: https://www.cdc.gov/vaccines/ed/hpv/index.html. Accessed January 26, 2018
20
Perla
RJ
,
Provost
LP
,
Murray
SK
.
Sampling considerations for health care improvement.
Qual Manag Health Care
.
2013
;
22
(
1
):
36
47
[PubMed]
21
ACIP Childhood/Adolescent Immunization Work Group
,
Akinsanya-Beysolow
I
,
Jenkins
R
,
Meissner
HC
;
Centers for Disease Control and Prevention
.
Advisory Committee on Immunization Practices (ACIP) recommended immunization schedule for persons aged 0 through 18 years–United States, 2013.
MMWR Suppl
.
2013
;
62
(
1
):
2
8
[PubMed]
22
Shojania
KG
,
Jennings
A
,
Mayhew
A
,
Ramsay
C
,
Eccles
M
,
Grimshaw
J
.
Effect of point-of-care computer reminders on physician behaviour: a systematic review.
CMAJ
.
2010
;
182
(
5
):
E216
E225
[PubMed]
23
Sittig
DF
,
Teich
JM
,
Osheroff
JA
,
Singh
H
.
Improving clinical quality indicators through electronic health records: it takes more than just a reminder.
Pediatrics
.
2009
;
124
(
1
):
375
377
[PubMed]
24
White
CM
,
Statile
AM
,
Conway
PH
, et al
.
Utilizing improvement science methods to improve physician compliance with proper hand hygiene.
Pediatrics
.
2012
;
129
(
4
). Available at: www.pediatrics.org/cgi/content/full/129/4/e1042
[PubMed]
25
Tierney
CD
,
Yusuf
H
,
McMahon
SR
, et al
.
Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics.
Pediatrics
.
2003
;
112
(
5
):
1076
1082
[PubMed]
26
Damschroder
LJ
,
Banaszak-Holl
J
,
Kowalski
CP
,
Forman
J
,
Saint
S
,
Krein
SL
.
The role of the champion in infection prevention: results from a multisite qualitative study.
Qual Saf Health Care
.
2009
;
18
(
6
):
434
440
[PubMed]
27
Zimmerman
RK
,
Brown
AE
,
Pavlik
VN
, et al
.
Using the 4 pillars practice transformation program to increase pneumococcal immunizations for older adults: a cluster-randomized trial.
J Am Geriatr Soc
.
2017
;
65
(
1
):
114
122
[PubMed]
28
Ornstein
S
,
Nietert
PJ
,
Jenkins
RG
, et al
.
Improving diabetes care through a multicomponent quality improvement model in a practice-based research network.
Am J Med Qual
.
2007
;
22
(
1
):
34
41
[PubMed]
29
Liddy
CE
,
Blazhko
V
,
Dingwall
M
,
Singh
J
,
Hogg
WE
.
Primary care quality improvement from a practice facilitator’s perspective.
BMC Fam Pract
.
2014
;
15
:
23
[PubMed]
30
Bleser
WK
,
Miller-Day
M
,
Naughton
D
,
Bricker
PL
,
Cronholm
PF
,
Gabbay
RA
.
Strategies for achieving whole-practice engagement and buy-in to the patient-centered medical home.
Ann Fam Med
.
2014
;
12
(
1
):
37
45
[PubMed]
31
Goldberg
DG
,
Mick
SS
,
Kuzel
AJ
,
Feng
LB
,
Love
LE
.
Why do some primary care practices engage in practice improvement efforts whereas others do not?
Health Serv Res
.
2013
;
48
(
2 pt 1
):
398
416
[PubMed]
32
Randolph
G
,
Fried
B
,
Loeding
L
,
Margolis
P
,
Lannon
C
.
Organizational characteristics and preventive service delivery in private practices: a peek inside the “black box” of private practices caring for children.
Pediatrics
.
2005
;
115
(
6
):
1704
1711
[PubMed]
33
Fieldston
ES
,
Hart
J
.
Quality improvement in primary care for children: interest and desire, but lack of action.
Acad Pediatr
.
2016
;
16
(
8
):
712
713
[PubMed]
34
Sebek
KM
,
Virkud
A
,
Singer
J
,
Pulgarin
CP
,
Schreibstein
L
,
Wang
JJ
.
Preliminary evaluation of a comprehensive provider feedback report.
J Med Pract Manage
.
2014
;
29
(
6
):
397
405
[PubMed]
35
Greene
L
,
Sapir
T
,
Moreo
K
,
Carter
JD
,
Patel
B
,
Higgins
PD
.
Impact of quality improvement educational interventions on documented adherence to quality measures for adults with Crohn’s disease.
Inflamm Bowel Dis
.
2015
;
21
(
9
):
2165
2171
[PubMed]
36
Dorell
C
,
Yankey
D
,
Jeyarajah
J
, et al
.
Delay and refusal of human papillomavirus vaccine for girls, national immunization survey-teen, 2010.
Clin Pediatr (Phila)
.
2014
;
53
(
3
):
261
269
[PubMed]
37
Epstein
NE
.
Multidisciplinary in-hospital teams improve patient outcomes: a review.
Surg Neurol Int
.
2014
;
5
(
suppl 7
):
S295
S303
[PubMed]
38
Cabana
MD
,
Rand
CS
,
Powe
NR
, et al
.
Why don’t physicians follow clinical practice guidelines? A framework for improvement.
JAMA
.
1999
;
282
(
15
):
1458
1465
[PubMed]
39
Guralnick
S
,
Ludwig
S
,
Englander
R
.
Domain of competence: systems-based practice.
Acad Pediatr
.
2014
;
14
(
suppl 2
):
S70
S79
[PubMed]
40
Lannon
CM
,
Peterson
LE
.
Pediatric collaborative networks for quality improvement and research.
Acad Pediatr
.
2013
;
13
(
suppl 6
):
S69
S74
[PubMed]
41
Fiks
AG
,
Luan
X
,
Mayne
SL
.
Improving HPV vaccination rates using maintenance-of-certification requirements.
Pediatrics
.
2016
;
137
(
3
):
e20150675
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: Dr Humiston has consulted for the Immunization Action Coalition; the other authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: During this study, Dr Humiston was a consultant to the Immunization Action Coalition; the other authors have indicated they have no financial relationships relevant to this article to disclose.