An Educational Intervention to Improve HPV Vaccination: A Cluster Randomized Trial

Given poor HPV vaccination rates in Indiana as well as limited evidence on consumer-centered technology applications that can be deployed in a typical clinical setting, we designed a cluster randomized trial to test the effect of a digital HPV vaccine educational intervention to be delivered during a clinic visit. With our study, we contribute evidence on whether and how information technologies can increase awareness and support parental decision-making about prevention behaviors such as vaccination. With this article, we summarize the results of the trial.

We employed a cluster randomized trial with a 2-arm design to examine the efficacy of an education intervention on HPV vaccine series first dose (initiation), second dose, and/or third dose (completion) among 11- to 17-year-old boys and girls from October 2015 to May 2016. Five urban health clinics were grouped into clusters on the basis of patient volume, race, and sex. Clusters were randomly assigned by coin flip. After randomization, the intervention cluster contained 2 clinics, and the control cluster contained 3 clinics. The study was approved by the Indiana University Institutional Review Board. The institutional review board waived the requirement for written informed consent.

Eskenazi Health is 1 of the 5 largest safety net health systems in the United States. The health system contains a 315-bed hospital and 9 community health centers located across the metropolitan area of Indianapolis, Indiana, the 11th largest city in the United States. All 5 pediatric clinics within the health system served as the setting for this study.

The Child Health Improvement through Computer Automation (CHICA) system is an operational CDS system used in the 5 pediatric clinics for >12 years.¹⁹ When a child was signed into 1 of the pediatric clinics, the electronic health record (EHR) system sent a Health Level 7 admit-discharge-transfer (registration) message to CHICA. In response, CHICA queried the patient’s medical record. At the same time, the system sent an electronic request to Children and Hoosier Immunization Registry Program (CHIRP), Indiana’s immunization information system (IIS).^20,21 In response, CHICA received a download of the child’s immunization history. The download included CHIRP’s “forecast” of the immunizations for which the patient was due. The electronic transfer of immunization information between CHICA and CHIRP is a form of health information exchange (HIE).²² 

Theo is an interactive, consumer-directed mobile health software developed by Noble.MD (recently acquired by WellTrackOne, Hilton Head Island, SC) that functions on a tablet platform. Theo can be used to screen for health risks at the point of care by using validated screening surveys, identify specific patient risks, and deliver a standardized educational video in real time. Theo is used to measure pre- and postintervention patient knowledge, attitudes, readiness for change, and risk mitigation. Theo can be used to create a digital record compliant with Health Insurance Portability and Accountability Act that can be integrated into an EHR.

The technical system integrating CHICA and Theo is summarized in Fig 1. Once CHICA determined an adolescent registered into the clinic to be eligible for the intervention by analyzing their combined medical record (1) and immunization registry (2) data, a medical assistant (MA) in the clinic was notified (3) and provided the patient’s study identifier (a randomly generated study identifier). The MA then provided the parent of the adolescent with a tablet and entered the study identifier into the Theo software (4). One of 2 programs then launched (5), depending on the adolescent’s current HPV vaccination status.

The first program, created for adolescents who had not yet received the first dose of the HPV vaccine, assessed whether the family had already decided in favor of the HPV vaccine or if the family wanted more information. If the system determined the family was in favor of receiving a vaccine at the visit, the program provided a simple reinforcement message. If the family indicated a desire for more information about the HPV vaccine, the program provided information specific to the cancer prevention benefits and safety profile of the vaccine. The second program, for adolescents who had already received the first or second vaccine in the series, emphasized the need to make the first vaccine count by receiving the full series. Both programs were available in English and Spanish.

The program scripts were created by several authors (G.D.Z., S.M.D., B.E.D., and B.L.) with expertise in adolescent health, HPV vaccination, epidemiology, and health communication. Although not assessed in this article, the scripts and questions used by participants were guided by the theory of planned behavior, which has been used in previous HPV vaccine uptake research.^23,24 Once finalized, the scripts were recorded in English and Spanish then integrated into the Theo platform. The scripts are available for review and use by others after gaining permission from the authors.

Participants were parents or guardians of adolescents aged 11 to 17 who were unvaccinated and partially vaccinated as of the date of visit during the study period. All parents or guardians of adolescents attending intervention clinics were potentially eligible. Parents were excluded if their children had received the full HPV vaccination series. Parents further needed to read and comprehend either English or Spanish.

Spurious records were removed before analysis (25 out of 1621 were deleted). These records were either missing key values, such as a determination of whether the adolescent received a dose of the HPV vaccine, or calculated values were out of range (eg, adolescent went from having the first dose of HPV vaccine before the visit to having not started the vaccine series 2 weeks after the encounter).

Demographic and insurance data were summarized by treatment group to examine the overall characteristics of the cohort. Continuous variables were summarized by using means and SDs. Comparisons were made between the 2 treatment groups using t tests. Categorical variables were summarized by using proportions and differences between proportions, and significance testing was conducted with the χ² test.

The primary outcome of interest was HPV vaccine uptake, defined as a change in vaccination status as a result of a clinic visit. Vaccination status could be 1 of 4 potential states: (1) “patient lacks documentation on starting the series,” (2) “patient had documented first dose,” (3) “patient had documented second dose,” or (4) “patient had documented third or final dose.” Documentation was determined by CHICA, which integrated records from both the adolescent’s EHR and CHIRP, the state immunization registry. The outcome variable was modeled as a binary change between 2 time periods in which a value of “1” represented a change in status (or state). A 2-week window was observed after a clinic visit to allow for any vaccines delivered in the clinic during an encounter to be recorded in the EHR or CHIRP.

To analyze the effect of the intervention on vaccine uptake, we employed an intention-to-treat (ITT) analysis using generalized estimating equations (GEEs) to accommodate the potential cluster effect of clinics when a significant cluster effect was identified. Specifically, GEEs account for the correlations in treatment outcomes from children seen at the same clinic. A GEE is frequently used in generalized linear models for correlated data and produces robust SE estimates.²⁵ We performed 2 ITT analyses to compare vaccine uptake in (1) the intervention clinics versus control clinics as well as (2) the group of adolescents who received the tablet versus those who did not receive the intervention. All significance testing was 2 tailed, and the significance level was set at 5%. All data analyses were completed with SAS version 9.3 (SAS Institute, Inc, Cary, NC).

A total of 1596 adolescents visited 1 of the clinics during the 7-month trial. One-third (N = 537) visited an intervention clinic with the remainder visiting a control clinic. The characteristics of the cohort are summarized in Table 1. Overall, adolescents were predominantly people of color, 11 to 12 years of age, and had Medicaid insurance.

Sex and race were similar between intervention and control clinics. Adolescents attending the 2 intervention clinics were slightly younger, on average, than those who attended the control clinics (mean age 12.2 vs 12.9 years; P < .001). The study arms also differed by insurance type with adolescents attending the intervention clinics more likely to have commercial or Medicaid insurance and less likely to use self-pay or another form of insurance than those attending the control clinics (P < .001).

Vaccine status during visits as well as 2 weeks after visits is summarized in Fig 2. Similar patterns are observed across intervention and control clinics. The proportion of adolescents with no dose of the HPV vaccine 2 weeks after a clinic visit (36.2%) is lower than at the time of the visit (56.8%). In parallel, there is a rise in the proportion of adolescents with a documented first, second, or third dose of the vaccine 2 weeks after a clinic visit. The exception to this pattern is in the intervention clinics, where the proportion of adolescents with a documented first dose actually fell slightly 2 weeks postvisit (29.1%–27.9%).

Results of the ITT analysis examining the effect of the intervention are summarized in Table 2. Of those presenting to a control clinic, 531 (50.1%) received a dose of the HPV vaccine within 2 weeks of a visit. Of those presenting to an intervention clinic, 348 (64.8%) received a dose of the HPV vaccine within 2 weeks of the visit. Comparing HPV vaccination uptake (eg, a change in HPV dose status) between intervention and control clinics, adolescents presenting to an intervention clinic had nearly double the odds of receiving a dose of the HPV vaccine (odds ratio [OR], 1.82; 95% confidence interval [CI], 1.47–2.25; P < .001).

A total of 141 (25% of those presenting to an intervention clinic) adolescents received a tablet from an MA during their visit. Nearly four-fifths (78.0%) of these adolescents received a dose of the HPV vaccine. Slightly more than half (52.8%) of adolescents who did not receive a tablet received a dose of the vaccine. Comparing HPV vaccination uptake between these groups, adolescents who received a tablet had a 3-times greater odds of receiving a dose of the HPV vaccine (OR, 3.07; 95% CI, 1.47–6.42; P = .003). For this analysis, the cluster effect was significant (P = .005); therefore, the GEE accounted for the clustering by clinic.

In a cluster randomized trial across 5 pediatric clinics within an urban health system, we examined the effect of a digital educational intervention aimed at increasing HPV vaccination uptake among adolescents. At intervention sites, a video on the risks and benefits of the HPV vaccine or reinforcement message was presented on a mobile tablet to parents or guardians of eligible adolescents in an examination room while they waited for clinicians during a routine, nonacute care visit.

The efficacy of the tablet-based educational intervention was significant, tripling the odds of HPV vaccine uptake among adolescents who received the tablets. Viewing tailored messages on the tablets with respect to vaccine initiation or series completion likely “activated” families to either request the vaccine or discuss the vaccine with providers during the visit. It is further likely that when families asked about the vaccine after use of the tablets providers delivered positive reinforcement messages that also contributed to vaccine uptake, a “feature” of interventions like the 1 used in this study. These results suggest that patient-centered education strategies delivered in a clinic setting via information technology platforms can positively impact the adoption of preventive health behaviors.

This study adds to our understanding of consumer-oriented interventions that seek to improve HPV prevention behaviors. Authors of previous studies^8,12 employed a heterogeneous set of communication media to reach adolescents or their parents, with most studies employing automated phone call reminders. Authors of just 2 studies^17,18 used interactive computer-based approaches. In DiClemente et al,¹⁸ adolescent girls 14 to 18 years in age viewed a 12-minute interactive media presentation. Researchers observed a significant difference only in completion of the second dose of the HPV vaccine; there was no change in initiation rate, and there was a nonsignificant increase in completion of the third dose. In Hopfer,¹⁷ female college students 18 to 26 years in age watched videos of vaccine decision narratives delivered by peers, medical experts, or a combination of peers and experts. Women who watched the videos delivered by a combination of peers and experts were almost twice as likely as women in the control group to vaccinate within 2 months of watching the video.¹⁷ Compared with previous studies, our study included younger adolescents (11–17 years of age) and observed significant increases in series initiation as well as second and third doses of the vaccine. The younger population in the intervention clinics may explain, in part, the impact on series initiation as compared with DiClemente et al.¹⁸ 

The impact of consumer-oriented educational interventions on public health could be significant if broadly used across the health system. Vaccination rates against HPV remain low in Indiana and many other states for early adolescents.⁷ Vaccine series completion rates also remain well below the Healthy People 2020 goals, especially for adolescent boys. An educational intervention accessible to adolescents as well as their parents or guardians could support more informed decision-making about the risks of HPV infection and benefits of vaccination. Moreover, an educational intervention might enable additive effects when combined with community- and provider-oriented interventions, which previous research suggests can also improve vaccination rates.^8,–11 

A key component of the intervention was the interoperable HIE network that enabled the CHICA system to gather and integrate data from the health system’s commercial EHR as well as the state IIS, known as CHIRP in Indiana. Automated query and integration of vaccine history information to the point of care provided a strong infrastructure^26,27 on which the intervention could be tested. Although HIE networks exist elsewhere, the technical architecture and approach used in this study would likely need to be adapted to succeed in states with less comprehensive HIE networks or IIS.

Although the results from this trial are encouraging, implementing the trial involved several challenges. A major challenge was distribution of the tablets to eligible patients. Only a quarter of eligible patients received a tablet after clinic check-in. This presented a logistical barrier. Before providing a tablet to a patient, the MA had to be alerted by CHICA to the patient’s eligibility and manually enter the patient’s trial study identifier into the tablet. These steps proved challenging for a busy pediatric clinic environment. If tablets were routinely distributed to all patients of a certain age or every patient regardless of age and patients could use the tablets to view a variety of content, then administration of the devices might be easier on clinic staff.

Another challenge was variable wait times, because some providers came into the room just before or during the video interaction. Similarly, some clinics take patients right away to the examination room, whereas other clinics routinely have waiting periods in a lobby. These present challenges as to when it would be an appropriate time to ask patients to interact with a tablet to view an educational intervention. One possibility for future research would be to examine previsit delivery of videos and interactive media by using the Internet or mobile phone to view an educational intervention 24 to 48 hours before the visit. In a recent review that examined mobile phone applications designed to support positive adolescent preventive health choices, Badawy and Kuhns²⁸ concluded that less than half of the studies (N = 19) observed a significant improvement in prevention decisions. Additional studies on mobile interventions would contribute to the growing evidence base on consumer-focused informatics applications.²⁹ 

This study has 3 limitations of note. First, the clustering of clinics occurred within a single, urban health system cautioning broad generalizations across all care settings. Additional testing across a variety of settings and multiple health systems would be necessary to demonstrate sustained effects before adoption into routine care. Second, the mere presence of the tablets in the clinic might have influenced provider behavior for all patients in that clinic (eg, unblinding) rather than just those patients who received the tablets. Finally, the outcome was modeled as a dichotomous variable with respect to a change in vaccine status. Because analysis was not performed separately for vaccine initiation versus series completion, the intervention may have had more effect on 1 of these groups of patients because the educational software showed different videos to each group of patients. Authors of future research should explore whether educational interventions are more effective for initiation or completion.

In a cluster randomized trial of an educational intervention to improve HPV vaccination rates delivered via a digital tablet in an examination room, efficacy was high. Although results are promising, the intervention occurred at 2 clinics in a single urban health system. An expanded, multisite trial of the intervention, perhaps in combination with an intervention targeting providers, would be necessary to demonstrate both how the intervention could scale across the health system and how the intervention might work in combination with other evidence-based methods for improving HPV vaccination rates.

CDS

clinical decision support

CHICA

Child Health Improvement through Computer Automation

CI

confidence interval

EHR

electronic health record

GEE

generalized estimating equation

HIE

health information exchange

HPV

human papillomavirus

IIS

immunization information system

ITT

intention-to-treat

MA

medical assistant

OR

odds ratio