BACKGROUND:

To increase vaccine acceptance, we created a Web-based the “Vaccines and Your Baby” intervention (VAYB) that provided new parents with vaccine information messages tailored to vaccine beliefs and values. We evaluated the effectiveness of the VAYB by comparing timely uptake of infant vaccines to an untailored version of the intervention (UT) or usual care intervention (UC) only.

METHODS:

Between April 2016 and June 2019, we conducted a randomized clinical trial. Pregnant women and new parents were randomly assigned to the VAYB, UT, or UC arms. In the VAYB and UT arms, participants were exposed to interventions at 4 time points from pregnancy until their child was 15 months of age. The primary outcome was up-to-date status for recommended vaccines from birth to 200 days of age. A modified intent-to-treat analysis was conducted. Data were analyzed with logistic regression to generate odds ratios (ORs) and 95% confidence intervals (CIs).

RESULTS:

We enrolled 824 participants (276 VAYB, 274 UT, 274 UC), 143 (17.4%) of whom were lost to follow-up. The up-to-date rates in the VAYB, UT, and UC arms were 91.44%, 92.86%, and 92.31%, respectively. Infants in the VAYB arm were not more likely to be up to date than infants in the UC arm (OR = 0.89; 95% CI, 0.45–1.76) or in the UT arm (OR = 0.82; 95% CI, 0.42–1.63). The odds of being up to date did not differ between UT and UC arms (OR = 1.08; 95% CI, 0.54–2.18).

CONCLUSIONS:

Delivering Web-based vaccine messages tailored to parents’ vaccine attitudes and values did not positively impact the timely uptake of infant vaccines.

What’s Known on This Subject:

Many parents have concerns about vaccination for their children, leading some to delay or decline administration. Tailored messaging provides individualized messages to patients and positively impacted a wide variety of health behaviors. However, results are limited for childhood vaccinations.

What This Study Adds:

This randomized clinical trial tested the effectiveness of a tailored messaging intervention to increase the timely uptake of childhood vaccinations. Results indicate it was not an effective approach to address vaccine acceptance among new parents in an integrated health system.

Universal vaccination has led to the eradication or control of numerous potentially fatal infectious diseases.1  Despite this success, ∼1 in 10 parents choose to refuse or delay vaccines for their children,24  a behavior responsible for a number of vaccine preventable disease outbreaks across the United States.5 

Although a large majority of parents choose to vaccinate their children, many parents report having concerns about childhood vaccines.6  Parents who are hesitant to vaccinate their children tend to question the safety and effectiveness of vaccines, despite the preponderance of evidence demonstrating that the benefits of vaccination greatly outweigh the risks. Vaccine-hesitant parents have expressed specific concerns that children receive too many vaccines in single well-child visits, as well as worries that vaccine ingredients, such as aluminum and formaldehyde, are toxic.7  Interventions designed to address such concerns and increase vaccine acceptance have been focused primarily on correcting a knowledge deficit, with limited success.79 

Knowledge about the benefits and risks of vaccines is just one of several factors that affects vaccination decisions. Emotion, trust, and values also play important roles.1012  Intervention strategies should therefore attempt to address the multiple determinants of vaccination decisions. One promising approach is message tailoring, which involves crafting messages to address each person’s unique attitudes, beliefs, experiences, knowledge, and values.13  Compared to nontailored information, tailored information is more likely to be read and remembered, thus rendering individuals more receptive to persuasion.14,15  Although message tailoring has been shown to improve a wide range of health behaviors, there are limited data on its impact on childhood vaccination behaviors.16,17  Moreover, the capacity to create and deliver tailored vaccine messages to parents in busy clinical settings may be limited, suggesting that methods to provide parents with the information in nonclinical settings are needed.

To improve vaccine acceptance, we designed and built the Web-based tailored messaging “Vaccines and Your Baby” intervention (VAYB).18  For this report, we evaluated the effectiveness of the VAYB with a randomized clinical trial (RCT) in which timely uptake of infant vaccines was compared across 3 study arms.

Between April 2016 and June 2019, we conducted a single-site RCT to evaluate the impact of the VAYB on timely childhood vaccination among infants of women recruited during the last trimester of pregnancy. The protocol for the study has been published previously but is summarized below.19 

Participants were randomly assigned to 1 of 3 study arms: the VAYB, an untailored version of the intervention (UT), or usual care (UC) only. The intervention was administered at 4 time points across the follow-up: the last trimester of pregnancy or child age <2 months, at child age 4 to 6 months, at child age 10 to 12 months, and at child age 13 to 15 months. Our primary outcome was up-to-date status for vaccines administered from birth to 200 days of age. We hypothesized that infants of participants in the VAYB arm would be more likely to be up to date than infants of participants in the UT or UC arms. This study was approved by the Institutional Review Boards of Kaiser Permanente Colorado (KPCO), University of Colorado, and the University of Michigan.

Participants were recruited from KPCO, a nonprofit integrated health care system that serves ∼5000 pregnant women and 140 000 children across the Denver and Boulder region per year.

Recruitment was conducted in 6-week waves between April 2016 and December 2017. Potential participants were pregnant women identified by using the KPCO electronic health record (EHR). Women had to be in the third trimester of pregnancy, ≥18 years of age, English speaking, and enrolled in the KPCO health plan at the time of recruitment. Pregnant women were excluded if they had a diagnosis of fetal demise, miscarriage, congenital anomaly in the pregnancy, or high-risk maternal condition. Eligible women received a combination of mail, e-mail, and phone calls that directed them to a study registration Web site where they could consent into the study. The recruited woman or their partner could enroll in the study. Study participants were able to enroll from the last trimester of pregnancy to when their child was <2 months of age.

After consent, participants were directed to the study Web site where they created a username and password. They were then administered a validated preintervention survey to assess demographics, vaccination values, intentions to vaccinate, and vaccine attitudes and concerns.19,20  Vaccine hesitancy was measured with the Parent Attitudes About Childhood Vaccines short (PACV-short), a validated 5-item screening instrument used to assess hesitancy on a scale of 0 to 10.21  Participants scoring ≥5 were classified as “vaccine hesitant,” and those scoring <5 were “nonhesitant.” To ensure balance across study arms, random assignment was conducted within the 2 strata of vaccine hesitancy.

We conducted stratified random assignment with a permuted block technique. Participants were assigned to the VAYB, UT, and UC study arms by using a random allocation ratio of 1:1:1. Random assignment was conducted by using the SAS/STAT procedure PROC PLAN (SAS Institute, Inc, Cary, NC). Study participants were not blinded to the assigned study arm. Study investigators and statisticians were blinded to the study arm assignment for analysis.

Conceptual Model

The intervention was based on a hybrid of the theory of planned behavior22  and the values-attitude-behavior model,23  combined with approaches from motivational interviewing and persuasive messaging.24,25  The values-attitude-behavior model complements the theory of planned behavior by stating that personal values affect attitudes. In contrast to attitudes and beliefs, values are thought to remain stable over time.26,27  The objective of the intervention was to influence vaccine attitudes and intentions through messages that address each participant’s vaccination attitudes and concerns, framed according to personal values (eg, acknowledging their “right to choose” or desire to “keep their child safe”). We crafted tailored messages that used empathetic language and emphasized autonomy to affirm individual values.28 

Tailored Intervention

The Web-based tailored intervention was developed by using an iterative, user-driven approach that included surveys, one-on-one interviews, and usability testing. The details of this process have been described previously.18  Informational content for the intervention was derived from peer-reviewed sources and online materials provided by the Centers for Disease Control and Prevention and the American Academy of Pediatrics.

The messages conveying the information were tailored to each participants’ intention to vaccinate, personal attitudes about vaccination, vaccination values, and the child’s nickname, sex, and age. These data were collected from the preintervention survey, which activated an embedded algorithm to deliver the tailored messaging (Supplemental Information). The details for how the parent vaccine values survey was developed and evaluated have been described elsewhere.20  After participants completed the preintervention survey, they were automatically directed to the VAYB Web site, which was personalized on the basis of their survey responses. Information on the Web site was arranged across 9 clickable tiles. The top 3 tiles were prominently labeled “Just for You” and contained the most highly tailored content that was based on the participants’ vaccination values and top 3 vaccination concerns. The remaining content was lightly tailored on the basis of the participants’ other, less pressing concerns identified by their survey responses. The lightly tailored content did not incorporate vaccination values.

The intervention and surveys were administered again when the child was age 4 to 6, 10 to 12, and 13 to 15 months. The Web site was retailored and refreshed at each time point on the basis of the updated survey responses. Tailoring on attitudes was updated at all 3 follow-up time points, whereas tailoring on values was updated only at the 10- to 12-month time point.

UT

An untailored version of the VAYB Web site was created to isolate the effect of the tailoring. This version (UT) had the same design and factual information as the VAYB Web site, but it was not personalized to the participants’ survey responses, and the content did not change across the time points.

UC

Participants in all 3 study arms were eligible to receive standard pediatric preventive care at KPCO. This consists of scheduled 20-minute well-child visits at 2, 4, 6, and 12 months of age, with an option for a 9-month visit.29  Recommended childhood immunizations are administered at these health supervision visits, and it is standard practice at KPCO to offer parents Vaccine Information Statements relevant to that visit.30 

Participants in the UT and UC arms were administered the same surveys at the 4 intervention time points as those received by participants in the VAYB arm.

The primary outcome was immunization status assessed over the first 200 days of age to cover a majority of the routinely administered infant vaccines and to minimize loss to follow-up. Immunization data were extracted from the KPCO EHR for the following 6 vaccines recommended by the Advisory Committee on Immunization Practices: hepatitis B, rotavirus, diphtheria-tetanus-acellular pertussis, Haemophilus influenzae type b, pneumococcal conjugate, and inactivated poliovirus. Immunization status was quantified by using days undervaccinated, a continuous metric used to measure the differences between the ages at which the vaccine doses were actually administered and ages at which the doses should have been administered according to the Advisory Committee on Immunization Practices recommendations.2  For these 6 vaccines, days undervaccinated can range from 0 to 648 days.

Days undervaccinated was analyzed as a dichotomous and continuous variable. For the dichotomized measure, infants with 0 cumulative days undervaccinated were considered up to date, and infants with ≥1 days undervaccinated were considered not up to date.

Two secondary outcomes were also assessed. First, we examined up-to-date status for the measles-mumps-rubella (MMR) and varicella vaccines among infants with at least 489 days of follow-up, which is when days undervaccinated for the first doses of these vaccines begin to accrue. Second, we repeated the primary analysis excluding the hepatitis B vaccine from the assessment of up-to-date status. This latter analysis was conducted because the hepatitis B birth dose tends to be administered in the inpatient rather than outpatient setting.

The trial was powered to detect a difference in proportion of up-to-date status between 6.8% and 9.4% between the 3 study arms. This required a sample size of 477 participants for a difference of 9.4% (odds ratio [OR] = 3) and 1002 participants for a difference of 6.8% (OR = 2), assuming a baseline vaccine hesitancy of 15%, a 1:1:1 allocation ratio, 80% statistical power, and 2-sided α = .05.

We conducted a modified intent-to-treat analysis that adhered to study arm assignment but excluded infants if their vaccination data could not be completely ascertained over the 200-day follow-up. Infants were excluded if they died, were not enrolled in KPCO, enrolled in KPCO after 60 days of age, disenrolled from KPCO before the end of follow-up, or were not receiving primary care at KPCO.

For the primary analysis, up-to-date status and days undervaccinated were assessed from birth to 200 days of age. Up-to-date status was analyzed with logistic regression to estimate ORs and 95% confidence intervals (CIs). For days undervaccinated, we used a nonparametric analysis and rank transformation approach because the distribution of days undervaccinated was highly skewed to the right. Days undervaccinated were first ranked in increasing order of magnitude for all infants independent of study arm assignment, and then the mean ranks were compared across the study arms by using a 1-way analysis of variance. As a preplanned subgroup analysis, the data were stratified by baseline vaccine hesitancy status. As a post hoc subgroup analysis, up-to-date status was assessed across the study arms among participants with baseline PACV-short scores between 4 and 7. Participants with these scores may represent vaccine “fence-sitters,” defined as parents who are willing to vaccinate but also have concerns about vaccines.31,32 

For the secondary analyses, logistic regression was used to assess up-to-date status for the MMR and varicella vaccines among infants with at least 489 days of follow-up and to assess up-to-date status excluding the hepatitis B vaccine. Analyses were conducted with SAS software 9.4.

In the KPCO cohort, there were 4929 pregnant women who were potentially eligible for the study (Fig 1). After exclusions, there were 824 participants enrolled in the study, 795 (96.5%) of whom were recruited during pregnancy, and 29 (3.5%) were recruited when their child was <2 months of age (Table 1). According to the 1:1:1 randomization ratio, 276 were randomly assigned to the VAYB arm, 274 were randomly assigned to the UT arm, and 274 were randomly assigned to the UC arm (Fig 1). Baseline characteristics were evenly distributed across study arms (Table 1). For the entire study population, the mean age at enrollment was 32 years, 85% were white, 12% were Hispanic, and 85.3% were college-educated. At baseline, 14.3% of the participants were categorized as being vaccine hesitant, with a mean hesitancy score of 2.25 (SD = 2.21).

FIGURE 1

Consolidated Standards of Reporting Trials (CONSORT) study flow diagram.

FIGURE 1

Consolidated Standards of Reporting Trials (CONSORT) study flow diagram.

TABLE 1

Baseline Characteristics of Enrolled Members, by Study Arm (N = 824)

CharacteristicaTotal (n = 824)VAYB (n = 276)UT (n = 274)UC (n = 274)
Enrolled during pregnancy, n (%)b 795 (96.48) 268 (97.10) 267 (97.45) 260 (94.89) 
Female sex, n (%)b 821 (99.64) 274 (99.28) 273 (99.64) 274 (100) 
Parent’s age at enrollment, y, mean (SD) 31.99 (4.37) 31.96 (4.49) 32.20 (4.22) 31.81 (4.41) 
Race, n (%)b     
 White 703 (85.32) 233 (84.42) 233 (85.04) 237 (86.50) 
 Black 8 (0.97) 3 (1.09) 4 (1.46) 1 (0.36) 
 Asian American or Pacific Islander 32 (3.88) 14 (5.07) 10 (3.65) 8 (2.92) 
 Multiracial or other 65 (7.89) 21 (7.61) 23 (8.39) 21 (7.66) 
 Nonresponse 16 (1.94) 5 (1.81) 4 (1.46) 7 (2.55) 
Ethnicity, n (%)b     
 Hispanic 100 (12.14) 39 (14.13) 31 (11.31) 30 (10.95) 
 Non-Hispanic 714 (86.65) 232 (84.06) 241 (87.96) 241 (87.96) 
 Nonresponse 10 (1.21) 5 (1.81) 2 (0.73) 3 (1.09) 
No. children, n (%)b     
 Pregnant with first child 155 (18.81) 52 (18.84) 54 (19.71) 49 (17.88) 
 One child 503 (61.04) 167 (60.51) 164 (59.85) 172 (62.77) 
 Two children 111 (13.47) 41 (14.86) 36 (13.14) 34 (12.41) 
 Three or more children 55 (6.67) 16 (5.80) 20 (7.30) 19 (6.93) 
Household income, n (%)b     
 <$40 000 59 (7.16) 21 (7.61) 18 (6.57) 20 (7.30) 
 $40 000–$80 000 222 (26.94) 84 (30.43) 66 (24.09) 72 (26.28) 
 $81 000–$120 000 299 (36.29) 95 (34.42) 105 (38.32) 99 (36.13) 
 $121 000–$150 000 83 (10.07) 26 (9.42) 27 (9.85) 30 (10.95) 
 >$150 000 129 (15.66) 38 (13.77) 49 (17.88) 42 (15.33) 
 Nonresponse 32 (3.88) 12 (4.35) 9 (3.28) 11 (4.01) 
Employment, n (%)b     
 Employed full-time 567 (68.81) 188 (68.12) 187 (68.25) 192 (70.07) 
 Employed part-time 112 (13.59) 41 (14.86) 40 (14.60) 31 (11.31) 
 Unemployed 13 (1.58) 7 (2.54) 2 (0.73) 4 (1.46) 
 Stay-at-home parent 123 (14.93) 35 (12.68) 42 (15.33) 46 (16.79) 
 Student 8 (0.97) 5 (1.81) 2 (0.73) 1 (0.36) 
 Nonresponse 1 (0.12) 0 (0) 1 (0.36) 0 (0) 
Marital status, n (%)b     
 Married 716 (86.89) 236 (85.51) 239 (87.23) 241 (87.96) 
 Separated 3 (0.36) 2 (0.72) 1 (0.36) 0 (0) 
 Divorced 7 (0.85) 2 (0.72) 1 (0.36) 4 (1.46) 
 Not married 13 (1.58) 3 (1.09) 7 (2.55) 3 (1.09) 
 Single 14 (1.70) 3 (1.09) 6 (2.19) 5 (1.82) 
 Living with a partner 71 (8.62) 30 (10.87) 20 (7.30) 21 (7.66) 
Education, n (%)b     
 Grade school 6 (0.73) 1 (0.36) 2 (0.73) 3 (1.09) 
 High school 19 (2.31) 9 (3.26) 6 (2.19) 4 (1.46) 
 Some college 92 (11.17) 35 (12.68) 28 (10.22) 29 (10.58) 
 College 323 (39.20) 105 (38.04) 103 (37.59) 115 (41.97) 
 Graduate school 380 (46.12) 125 (45.29) 134 (48.91) 121 (44.16) 
 Nonresponse 4 (0.49) 1 (0.36) 1 (0.36) 2 (0.73) 
Hesitancy by PACV-short scale,c mean (SD) 2.25 (2.21) 2.21 (2.36) 2.25 (2.10) 2.29 (2.18) 
Hesitant by PACV-short scale, n (%)b,c, d 118 (14.32) 39 (14.13) 39 (14.23) 40 (14.60) 
CharacteristicaTotal (n = 824)VAYB (n = 276)UT (n = 274)UC (n = 274)
Enrolled during pregnancy, n (%)b 795 (96.48) 268 (97.10) 267 (97.45) 260 (94.89) 
Female sex, n (%)b 821 (99.64) 274 (99.28) 273 (99.64) 274 (100) 
Parent’s age at enrollment, y, mean (SD) 31.99 (4.37) 31.96 (4.49) 32.20 (4.22) 31.81 (4.41) 
Race, n (%)b     
 White 703 (85.32) 233 (84.42) 233 (85.04) 237 (86.50) 
 Black 8 (0.97) 3 (1.09) 4 (1.46) 1 (0.36) 
 Asian American or Pacific Islander 32 (3.88) 14 (5.07) 10 (3.65) 8 (2.92) 
 Multiracial or other 65 (7.89) 21 (7.61) 23 (8.39) 21 (7.66) 
 Nonresponse 16 (1.94) 5 (1.81) 4 (1.46) 7 (2.55) 
Ethnicity, n (%)b     
 Hispanic 100 (12.14) 39 (14.13) 31 (11.31) 30 (10.95) 
 Non-Hispanic 714 (86.65) 232 (84.06) 241 (87.96) 241 (87.96) 
 Nonresponse 10 (1.21) 5 (1.81) 2 (0.73) 3 (1.09) 
No. children, n (%)b     
 Pregnant with first child 155 (18.81) 52 (18.84) 54 (19.71) 49 (17.88) 
 One child 503 (61.04) 167 (60.51) 164 (59.85) 172 (62.77) 
 Two children 111 (13.47) 41 (14.86) 36 (13.14) 34 (12.41) 
 Three or more children 55 (6.67) 16 (5.80) 20 (7.30) 19 (6.93) 
Household income, n (%)b     
 <$40 000 59 (7.16) 21 (7.61) 18 (6.57) 20 (7.30) 
 $40 000–$80 000 222 (26.94) 84 (30.43) 66 (24.09) 72 (26.28) 
 $81 000–$120 000 299 (36.29) 95 (34.42) 105 (38.32) 99 (36.13) 
 $121 000–$150 000 83 (10.07) 26 (9.42) 27 (9.85) 30 (10.95) 
 >$150 000 129 (15.66) 38 (13.77) 49 (17.88) 42 (15.33) 
 Nonresponse 32 (3.88) 12 (4.35) 9 (3.28) 11 (4.01) 
Employment, n (%)b     
 Employed full-time 567 (68.81) 188 (68.12) 187 (68.25) 192 (70.07) 
 Employed part-time 112 (13.59) 41 (14.86) 40 (14.60) 31 (11.31) 
 Unemployed 13 (1.58) 7 (2.54) 2 (0.73) 4 (1.46) 
 Stay-at-home parent 123 (14.93) 35 (12.68) 42 (15.33) 46 (16.79) 
 Student 8 (0.97) 5 (1.81) 2 (0.73) 1 (0.36) 
 Nonresponse 1 (0.12) 0 (0) 1 (0.36) 0 (0) 
Marital status, n (%)b     
 Married 716 (86.89) 236 (85.51) 239 (87.23) 241 (87.96) 
 Separated 3 (0.36) 2 (0.72) 1 (0.36) 0 (0) 
 Divorced 7 (0.85) 2 (0.72) 1 (0.36) 4 (1.46) 
 Not married 13 (1.58) 3 (1.09) 7 (2.55) 3 (1.09) 
 Single 14 (1.70) 3 (1.09) 6 (2.19) 5 (1.82) 
 Living with a partner 71 (8.62) 30 (10.87) 20 (7.30) 21 (7.66) 
Education, n (%)b     
 Grade school 6 (0.73) 1 (0.36) 2 (0.73) 3 (1.09) 
 High school 19 (2.31) 9 (3.26) 6 (2.19) 4 (1.46) 
 Some college 92 (11.17) 35 (12.68) 28 (10.22) 29 (10.58) 
 College 323 (39.20) 105 (38.04) 103 (37.59) 115 (41.97) 
 Graduate school 380 (46.12) 125 (45.29) 134 (48.91) 121 (44.16) 
 Nonresponse 4 (0.49) 1 (0.36) 1 (0.36) 2 (0.73) 
Hesitancy by PACV-short scale,c mean (SD) 2.25 (2.21) 2.21 (2.36) 2.25 (2.10) 2.29 (2.18) 
Hesitant by PACV-short scale, n (%)b,c, d 118 (14.32) 39 (14.13) 39 (14.23) 40 (14.60) 
a

No statistically significant differences in characteristics across study arms.

b

Percentages represent column percentages.

c

PACV-short scale is a validated 5-question screening tool assessing vaccine hesitancy on a scale from 0 to 10; higher scores indicate a higher degree of hesitancy.

d

Participants scoring ≥5 (on a scale from 0 to 10) on the PACV-short scale were classified as vaccine hesitant.

After random assignment, incomplete vaccination data led to 143 (17.4%) exclusions (Fig 1). These exclusions were considered loss to follow-up, which ranged from 13.1% to 19.6% across the study arms (P = .08). The mean vaccine hesitancy scores of the participants lost to follow-up did not differ significantly across the study arms (P = .92).

The proportions of infants who were up to date at age 200 days were 91.4%, 92.9%, and 92.3% for the VAYB, UT, and UC arms, respectively. The 95% CIs for the ORs comparing up-to-date status between study arms all included 1 and were not statistically significant (Table 2). Infants in the VAYB were not more likely to be up to date than infants in the UC arm (OR = 0.89; 95% CI, 0.45–1.76), nor were they more likely to be up to date than infants in the UT arm (OR = 0.82; 95% CI, 0.42–1.63). Similarly, the odds of being up to date did not differ between the UT and UC arms (OR = 1.08; 95% CI, 0.54–2.18). In the overall KPCO cohort of potentially eligible participants who were not enrolled in the study (n = 3366), the proportion of infants who were up to date during the study period was 85.6%.

TABLE 2

Proportion of Infants Up to Date for Vaccination Status and OR Estimates for Up-To-Date Vaccination Status Between Study Arms: All Infants in the Study (N = 681)

Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 222) 203 (91.44) VAYB versus UC 0.89 (0.45–1.76) .74 
UT (n = 238) 221 (92.86) UT versus UC 1.08 (0.54–2.18) .82 
UC (n = 221) 204 (92.31) VAYB versus UT 0.82 (0.42–1.63) .57 
Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 222) 203 (91.44) VAYB versus UC 0.89 (0.45–1.76) .74 
UT (n = 238) 221 (92.86) UT versus UC 1.08 (0.54–2.18) .82 
UC (n = 221) 204 (92.31) VAYB versus UT 0.82 (0.42–1.63) .57 

In the subanalysis stratified by vaccine hesitancy status, infants of vaccine-hesitant parents in the VAYB arm were less likely to be up to date than infants of vaccine-hesitant parents in the UT arm (OR = 0.28; 95% CI, 0.08–0.98) (Table 3). The odds of being up to date did not differ between the VAYB and UC arms (OR = 0.67; 95% CI, 0.23–1.96) or between the UT and UC arms (OR = 2.42; 95% CI, 0.65–9.01). Among infants of parents who were not vaccine-hesitant, the odds of being up-to-date did not differ between the study arms in any of the comparisons (Table 4). In the post hoc subgroup analysis of potential fence-sitters, the OR for up-to-date status comparing the VAYB to UC arm was not statistically significant (OR = 0.69; 95% CI, 0.16–3.01).

TABLE 3

Proportion of Infants Up to Date for Vaccination Status and OR Estimates for Up-To-Date Vaccination Status Between Study Arms: When Parents Were Vaccine Hesitant at Baseline (n = 98)

Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 33) 22 (66.67) VAYB versus UC 0.67 (0.23–1.96) .46 
UT (n = 33) 29 (87.88) UT versus UC 2.42 (0.65–9.01) .19 
UC (n = 32) 24 (75.0) VAYB versus UT 0.28 (0.08–0.98) .05 
Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 33) 22 (66.67) VAYB versus UC 0.67 (0.23–1.96) .46 
UT (n = 33) 29 (87.88) UT versus UC 2.42 (0.65–9.01) .19 
UC (n = 32) 24 (75.0) VAYB versus UT 0.28 (0.08–0.98) .05 
TABLE 4

Proportion of Infants Up to Date for Vaccination Status and OR Estimates for Up-To-Date Vaccination Status Between Study Arms: When Parents Were Not Vaccine Hesitant at Baseline (n = 583)

Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 189) 181 (95.77) VAYB versus UC 1.13 (0.43–3.0) .80 
UT (n = 205) 192 (93.66) UT versus UC 0.74 (0.31–1.77) .50 
UC (n = 189) 180 (95.24) VAYB versus UT 1.53 (0.62–3.78) .35 
Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 189) 181 (95.77) VAYB versus UC 1.13 (0.43–3.0) .80 
UT (n = 205) 192 (93.66) UT versus UC 0.74 (0.31–1.77) .50 
UC (n = 189) 180 (95.24) VAYB versus UT 1.53 (0.62–3.78) .35 

For the secondary analysis of MMR and varicella vaccination status at age 489 days, the ORs for up-to-date status comparing the VAYB to UC arm were not statistically significant (ORMMR = 1.18; 95% CI, 0.49–2.84; ORvaricella = 1.06; 95% CI, 0.41–2.74) (Tables 5 and 6). For the analysis excluding the hepatitis B vaccine, the OR for up-to-date status comparing the VAYB to UC arm was not statistically significant (OR = 0.94; 95% CI, 0.47–1.88).

TABLE 5

Proportion of Infants Up to Date on the First Dose of MMR Vaccine and OR Estimates for Up-To-Date MMR Vaccination Status Between Study Arms (n = 590)

Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 194) 184 (94.85) VAYB versus UC 1.18 (0.49–2.84) .72 
UT (n = 213) 204 (95.77) UT versus UC 1.45 (0.59–3.58) .42 
UC (n = 183) 172 (93.99) VAYB versus UT 0.81 (0.32–2.04) .66 
Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 194) 184 (94.85) VAYB versus UC 1.18 (0.49–2.84) .72 
UT (n = 213) 204 (95.77) UT versus UC 1.45 (0.59–3.58) .42 
UC (n = 183) 172 (93.99) VAYB versus UT 0.81 (0.32–2.04) .66 
TABLE 6

Proportion of Infants Up to Date on the First Dose of Varicella Vaccine and OR Estimates for Up-To-Date Varicella Vaccination Status Between Study Arms:(n = 590)

Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 194) 185 (95.36) VAYB versus UC 1.06 (0.41–2.74) .90 
UT (n = 213) 202 (94.84) UT versus UC 0.95 (0.39–2.35) .91 
UC (n = 183) 174 (95.08) VAYB versus UT 1.12 (0.45–2.76) .81 
Study ArmInfants Up to Date, n (%)Study Arm ComparisonsOR for Up-to-Date Vaccination Status (95% CI)P
VAYB (n = 194) 185 (95.36) VAYB versus UC 1.06 (0.41–2.74) .90 
UT (n = 213) 202 (94.84) UT versus UC 0.95 (0.39–2.35) .91 
UC (n = 183) 174 (95.08) VAYB versus UT 1.12 (0.45–2.76) .81 

The mean rank scores for days undervaccinated were 344.3, 338.7, and 340.2 for the VAYB, UT, and UC arms, respectively. These mean rank scores did not differ significantly for any of the study arm comparisons. P values for the VAYB versus UC, UT versus UC, and VAYB versus UT arm comparisons were 0.64, 0.87, and 0.52, respectively (Table 7). In the subgroup analysis stratified by vaccine hesitancy status, the mean rank scores did not differ for 5 of the 6 study arm comparisons (Tables 8 and 9). The only significant difference was between infants of vaccine-hesitant parents in the VAYB and the UT arms is infants of vaccine-hesitant parents in the VAYB arm had significantly more days undervaccinated than those of infants of hesitant parents in the UT arm (mean rank difference = 11.06; P = .03) (Table 8).

TABLE 7

Days Undervaccinated, Mean Ranks for Days Undervaccinated, and Difference in the Mean Ranks Between Study Arms: All Infants in the Study (N = 681)

Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
5th50th95th
VAYB (n = 222) 230 344.27 VAYB versus UC 4.10 .64 
UT (n = 238) 110 338.72 UT versus UC −1.44 .87 
UC (n = 221) 46 340.17 VAYB versus UT 5.54 .52 
Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
5th50th95th
VAYB (n = 222) 230 344.27 VAYB versus UC 4.10 .64 
UT (n = 238) 110 338.72 UT versus UC −1.44 .87 
UC (n = 221) 46 340.17 VAYB versus UT 5.54 .52 
a

Obtained by ranking all observations in increasing order of magnitude of days undervaccinated and calculating the mean.

b

Obtained by using 1-way analysis of variance on the ranks for days undervaccinated.

TABLE 8

Days Undervaccinated, Mean Ranks for Days Undervaccinated, and Difference in the Mean Ranks Between Study Arms: Infants of Parents Who Were Vaccine Hesitant at Baseline (n = 98)

Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
5th50th95th
VAYB (n = 33) 642 54.82 VAYB versus UC 4.88 .35 
UT (n = 33) 367 43.76 UT versus UC −6.18 .23 
UC (n = 32) 642 49.94 VAYB versus UT 11.06 .03 
Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
5th50th95th
VAYB (n = 33) 642 54.82 VAYB versus UC 4.88 .35 
UT (n = 33) 367 43.76 UT versus UC −6.18 .23 
UC (n = 32) 642 49.94 VAYB versus UT 11.06 .03 
a

Obtained by ranking all observations in increasing order of magnitude of days undervaccinated and calculating the mean.

b

Obtained by using 1-way analysis of variance on the ranks for days undervaccinated.

TABLE 9

Days Undervaccinated, Mean Ranks for Days Undervaccinated, and Difference in the Mean Ranks Between Study Arms: Infants of Parents Not Vaccine Hesitant at Baseline (n = 583)

Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
5th50th95th
VAYB (n = 189) 289.58 VAYB versus UC −0.98 .88 
UT (n = 205) 107 295.55 UT versus UC 4.99 .44 
UC (n = 189) 290.57 VAYB versus UT −5.97 .36 
Study ArmDays Undervaccinated PercentilesMean RanksaStudy Arm ComparisonsDifference in Mean RanksPb
5th50th95th
VAYB (n = 189) 289.58 VAYB versus UC −0.98 .88 
UT (n = 205) 107 295.55 UT versus UC 4.99 .44 
UC (n = 189) 290.57 VAYB versus UT −5.97 .36 
a

Obtained by ranking all observations in increasing order of magnitude of days undervaccinated and calculating the mean.

b

Obtained by using 1-way analysis of variance on the ranks for days undervaccinated.

In this RCT, provision of a Web-based tailored messaging intervention to expectant and new parents did not have a positive effect on the timely uptake of infant immunizations. Infants of parents in the VAYB arm were not more likely to be up to date for routinely administered vaccines or have fewer days undervaccinated over the first 200 days of life than infants of parents in the UT or UC study arms. These results do not support our hypothesis that the behaviors of parents can be influenced by carefully crafted messages tailored to their vaccination attitudes and values.

In several studies, authors have examined the impact of tailored messaging on vaccine attitudes, intentions, and behaviors related to human papillomavirus and influenza vaccination.3338  They tended to focus on Web-based formats or text messaging to deliver the tailored messages, and outcomes were assessed in adolescent girls, parents of adolescent girls, or adults. The results varied, with only 1 RCT revealing a positive effect on both initiation and completion of human papillomavirus vaccination series in adolescent girls.33  However, it is difficult to relate the VAYB results to these investigations because the study populations, intervention contents, and vaccination outcomes were considerably different. Perhaps most importantly, the other investigations were focused on 1 specific vaccine, whereas the VAYB attempted to address concerns across multiple vaccines.

This study had many strengths. The VAYB, survey instruments, and data collection tools were all developed by using iterative, theory- and user-driven approaches led by a multidisciplinary team.18  The effectiveness of the intervention was evaluated with a rigorous randomized design, and the study population was recruited during pregnancy, representing a key time period for infant vaccination decisions.39,40  In addition, rather than relying on recall, vaccination outcomes were extracted from the EHR, which captures vaccine administrations with a high degree of accuracy.41,42 

The subgroup analysis produced an unexpected result: among vaccine-hesitant participants, the UT arm was significantly more effective than the VAYB arm. Infants in the VAYB arm were significantly less likely to be up to date and had more days undervaccinated than infants in the UT arm. It is possible that our inferences based on the participants’ vaccine values and attitudes overreached and led to messages that were less relevant, thus undermining the credibility of the tailoring.43  In addition, we conducted 9 comparisons in the analysis without a correction for multiple comparisons, suggesting that this subgroup association could be significant by chance alone. Although we did not plan for a Bonferroni multiple comparison test a priori, such a correction would have rendered this association nonsignificant at the α < .05 level.

Although tailored messaging has been shown to be effective at eliciting health behavior change, the magnitude of effect sizes from randomized studies have been small.15  In integrated health systems such as KPCO, detecting modest effect sizes for infant vaccine acceptance is challenging because vaccination rates in these settings tend to be high.44  In our intervention trial, >90% of the infants across all 3 intervention arms were up to date, and 14% of the study participants were classified as vaccine hesitant at baseline. Although this rate of vaccine hesitancy aligns with other published estimates,4,45,46  it is possible that our intervention could have been effective if it had been implemented in other health care environments with lower infant vaccination rates and higher rates of vaccine hesitancy.

This study also had limitations. The trial was conducted in a single integrated health plan in Colorado, which may have affected the trial’s generalizability. Infants in the trial were more likely to be up to date than infants in the entire KPCO population during the study period (92.2% vs 85.6%), which may have also affected the generalizability of the trial. Overall, the loss to follow-up was 17.4%, largely attributed to parents who disenrolled from KPCO after their child was born. Although this may have also impacted the generalizability of the trial, the loss to follow-up did not differ significantly across the study arms. In addition, the rate of vaccine hesitancy among those lost to follow-up was similar across the arms, suggesting that these exclusions did not affect the internal validity of the results.

We designed the intervention to tailor messages on both vaccine values and attitudes. Although this is a potential strength of the intervention, it also possible that for some participants, the tailored content was on target for one construct but off target for the other construct. Such discrepant effects could explain the null results observed in the primary analysis. However, this could not be formally evaluated because we did not design the trial to evaluate the effects of the 2 types of tailoring separately.

This RCT suggests that Web-based tailored messaging does not positively impact parental immunization behaviors during infancy. Given that vaccine hesitancy remains a significant public health issue,47  researchers should continue to develop and test communication approaches to reduce parental vaccination concerns and improve the timely uptake of infant immunizations.

We acknowledge Courtney Kraus and Kathy Gleason who recruited participants and collected data for the trial.

Dr Glanz conceptualized and designed the study and drafted the initial manuscript; Dr Wagner conceptualized the study, designed the data collection protocols and instruments, and managed data acquisition; Dr Narwaney conceptualized the study, conducted the analyses, and interpreted the data; Ms Pyrzanowski conceptualized the study design; Mr Sevick contributed expertise to the analytic plan and conducted data interpretation; Drs Kwan and Resnicow conceptualized and designed the intervention and interpreted the data; Dr Dempsey conceptualized and designed the study and interpreted the data; and all authors reviewed and revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.

Deidentified participant data will not be made available.

This trial has been registered at www.clinicaltrials.gov (identifier: NCT02665013).

FUNDING: Funded by the National Institutes of Health, Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01 HD079457). Funded by the National Institutes of Health (NIH).

     
  • CI

    confidence interval

  •  
  • EHR

    electronic health record

  •  
  • KPCO

    Kaiser Permanente Colorado

  •  
  • MMR

    measles-mumps-rubella

  •  
  • OR

    odds ratio

  •  
  • PACV-short

    Parent Attitudes About Childhood Vaccines short

  •  
  • RCT

    randomized clinical trial

  •  
  • UC

    usual care intervention

  •  
  • UT

    untailored intervention

  •  
  • VAYB

    Vaccines and Your Baby intervention

1
Centers for Disease Control and Prevention
.
Ten great public health achievements--United States, 2001–2010
.
MMWR Morbidity and Mortality Weekly Report
.
2011
;
60
(
19
):
619
623
2
Glanz
JM
,
Newcomer
SR
,
Narwaney
KJ
, et al
.
A population-based cohort study of undervaccination in 8 managed care organizations across the United States
.
JAMA Pediatr
.
2013
;
167
(
3
):
274
281
3
Dempsey
AF
,
Schaffer
S
,
Singer
D
,
Butchart
A
,
Davis
M
,
Freed
GL
.
Alternative vaccination schedule preferences among parents of young children
.
Pediatrics
.
2011
;
128
(
5
):
848
856
4
Kempe
A
,
Saville
AW
,
Albertin
C
, et al
.
Parental hesitancy about routine childhood and influenza vaccinations: a national survey
.
Pediatrics
.
2020
;
146
(
1
):
e20193852
5
Phadke
VK
,
Bednarczyk
RA
,
Salmon
DA
,
Omer
SB
.
Association between vaccine refusal and vaccine-preventable diseases in the United States: a review of measles and pertussis
.
JAMA
.
2016
;
315
(
11
):
1149
1158
6
Kennedy
A
,
Lavail
K
,
Nowak
G
,
Basket
M
,
Landry
S
.
Confidence about vaccines in the United States: understanding parents’ perceptions
.
Health Aff (Millwood)
.
2011
;
30
(
6
):
1151
1159
7
Salmon
DA
,
Dudley
MZ
,
Glanz
JM
,
Omer
SB
.
Vaccine hesitancy: causes, consequences, and a call to action
.
Vaccine
.
2015
;
33
(
suppl 4
):
D66
D71
8
Sadaf
A
,
Richards
JL
,
Glanz
J
,
Salmon
DA
,
Omer
SB
.
A systematic review of interventions for reducing parental vaccine refusal and vaccine hesitancy
.
Vaccine
.
2013
;
31
(
40
):
4293
4304
9
Jarrett
C
,
Wilson
R
,
O’Leary
M
,
Eckersberger
E
,
Larson
HJ
;
SAGE Working Group on Vaccine Hesitancy
.
Strategies for addressing vaccine hesitancy - a systematic review
.
Vaccine
.
2015
;
33
(
34
):
4180
4190
10
MacDonald
NE
;
SAGE Working Group on Vaccine Hesitancy
.
Vaccine hesitancy: definition, scope and determinants
.
Vaccine
.
2015
;
33
(
34
):
4161
4164
11
Amin
AB
,
Bednarczyk
RA
,
Ray
CE
, et al
.
Association of moral values with vaccine hesitancy
.
Nat Hum Behav
.
2017
;
1
(
12
):
873
880
12
Witteman
HO
.
Addressing vaccine hesitancy with values
.
Pediatrics
.
2015
;
136
(
2
):
215
217
13
Hawkins
RP
,
Kreuter
M
,
Resnicow
K
,
Fishbein
M
,
Dijkstra
A
.
Understanding tailoring in communicating about health
.
Health Educ Res
.
2008
;
23
(
3
):
454
466
14
Skinner
CS
,
Strecher
VJ
,
Hospers
H
.
Physicians’ recommendations for mammography: do tailored messages make a difference?
Am J Public Health
.
1994
;
84
(
1
):
43
49
15
Campbell
MK
,
DeVellis
BM
,
Strecher
VJ
,
Ammerman
AS
,
DeVellis
RF
,
Sandler
RS
.
Improving dietary behavior: the effectiveness of tailored messages in primary care settings
.
Am J Public Health
.
1994
;
84
(
5
):
783
787
16
Noar
SM
,
Benac
CN
,
Harris
MS
.
Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions
.
Psychol Bull
.
2007
;
133
(
4
):
673
693
17
Lustria
MLA
,
Cortese
J
,
Noar
SM
,
Glueckauf
RL
.
Computer-tailored health interventions delivered over the Web: review and analysis of key components
.
Patient Educ Couns
.
2009
;
74
(
2
):
156
173
18
Dempsey
A
,
Kwan
BM
,
Wagner
NM
, et al
.
A values-tailored web-based intervention for new mothers to increase infant vaccine uptake: development and qualitative study
.
J Med Internet Res
.
2020
;
22
(
3
):
e15800
19
Dempsey
AF
,
Wagner
N
,
Narwaney
K
, et al
.
‘Reducing Delays In Vaccination’ (REDIVAC) trial: a protocol for a randomised controlled trial of a web-based, individually tailored, educational intervention to improve timeliness of infant vaccination
.
BMJ Open
.
2019
;
9
(
5
):
e027968
20
Cataldi
JR
,
Sevick
C
,
Pyrzanowski
J
, et al
.
Addressing personal parental values in decisions about childhood vaccination: measure development
.
Vaccine
.
2019
;
37
(
38
):
5688
5697
21
Oladejo
O
,
Allen
K
,
Amin
A
,
Frew
PM
,
Bednarczyk
RA
,
Omer
SB
.
Comparative analysis of the Parent Attitudes about Childhood Vaccines (PACV) short scale and the five categories of vaccine acceptance identified by Gust et al
.
Vaccine
.
2016
;
34
(
41
):
4964
4968
22
Ajzen
I
.
The theory of planned behavior
.
Organ Behav Hum Decis Process
.
1991
;
50
(
2
):
179
211
23
Homer
PM
,
Kahle
LR
.
A structural equation test of the value-attitude-behavior hierarchy
.
J Pers Soc Psychol
.
1988
;
54
(
4
):
638
24
Rollnick
S
,
Miller
WR
.
What is motivational interviewing?
Behav Cogn Psychother
.
1995
;
23
(
4
):
325
334
25
Sweeney
AM
,
Moyer
A
.
Self-affirmation and responses to health messages: a meta-analysis on intentions and behavior
.
Health Psychol
.
2015
;
34
(
2
):
149
159
26
Konty
MA
,
Dunham
CC
.
Differences in value and attitude change over the life course
.
Sociol Spectr
.
1997
;
17
(
2
):
177
197
27
Schwartz
SH
.
Are there universal aspects in the structure and contents of human values?
J Soc Issues
.
1994
;
50
(
4
):
19
45
28
Millner
W
,
Rollnick
S
.
Motivational Interviewing: Preparing People to Change Addictive Behaviors
.
New York, NY
:
Guilford Press
;
1991
29
Hagan
JF
 Jr.
,
Shaw
JS
,
Duncan
PM
, eds..
Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents
, 3rd ed.
Elk Grove Village, IL
:
American Academy of Pediatrics
;
2008
30
Centers for Disease Control and Prevention
.
Vaccine information statements (VISs): current VISs. 2019. Available at: https://www.cdc.gov/vaccines/hcp/vis/current-vis.html. Accessed December 11, 2019
31
Leask
J
.
Target the fence-sitters
.
Nature
.
2011
;
473
(
7348
):
443
445
32
Gust
D
,
Brown
C
,
Sheedy
K
,
Hibbs
B
,
Weaver
D
,
Nowak
G
.
Immunization attitudes and beliefs among parents: beyond a dichotomous perspective
.
Am J Health Behav
.
2005
;
29
(
1
):
81
92
33
Dempsey
AF
,
Pyrznawoski
J
,
Lockhart
S
, et al
.
Effect of a health care professional communication training intervention on adolescent human papillomavirus vaccination: a cluster randomized clinical trial
.
JAMA Pediatr
.
2018
;
172
(
5
):
e180016
34
Jordan
ET
,
Bushar
JA
,
Kendrick
JS
,
Johnson
P
,
Wang
J
.
Encouraging influenza vaccination among Text4baby pregnant women and mothers
.
Am J Prev Med
.
2015
;
49
(
4
):
563
572
35
DiClemente
RJ
,
Murray
CC
,
Graham
T
,
Still
J
.
Overcoming barriers to HPV vaccination: a randomized clinical trial of a culturally-tailored, media intervention among African American girls
.
Hum Vaccin Immunother
.
2015
;
11
(
12
):
2883
2894
36
Gerend
MA
,
Shepherd
MA
,
Lustria
ML
.
Increasing human papillomavirus vaccine acceptability by tailoring messages to young adult women’s perceived barriers
.
Sex Transm Dis
.
2013
;
40
(
5
):
401
405
37
Pot
M
,
Paulussen
TG
,
Ruiter
RA
, et al
.
Effectiveness of a web-based tailored intervention with virtual assistants promoting the acceptability of HPV vaccination among mothers of invited girls: randomized controlled trial
.
J Med Internet Res
.
2017
;
19
(
9
):
e312
38
Bennett
AT
,
Patel
DA
,
Carlos
RC
, et al
.
Human papillomavirus vaccine uptake after a tailored, online educational intervention for female university students: a randomized controlled trial
.
J Womens Health (Larchmt)
.
2015
;
24
(
11
):
950
957
39
Glanz
JM
,
Wagner
NM
,
Narwaney
KJ
, et al
.
A mixed methods study of parental vaccine decision making and parent-provider trust
.
Acad Pediatr
.
2013
;
13
(
5
):
481
488
40
Lieu
TA
,
Zikmund-Fisher
BJ
,
Chou
C
,
Ray
GT
,
Wittenberg
E
.
Parents’ perspectives on how to improve the childhood vaccination process
.
Clin Pediatr (Phila)
.
2017
;
56
(
3
):
238
246
41
Mullooly
J
,
Drew
L
,
DeStefano
F
, et al.;
Vaccine Safety DataLink Team
.
Quality of HMO vaccination databases used to monitor childhood vaccine safety
.
Am J Epidemiol
.
1999
;
149
(
2
):
186
194
42
Daley
MF
,
Glanz
JM
,
Newcomer
SR
, et al
.
Assessing misclassification of vaccination status: implications for studies of the safety of the childhood immunization schedule
.
Vaccine
.
2017
;
35
(
15
):
1873
1878
43
Kreuter
MW
,
Oswald
DL
,
Bull
FC
,
Clark
EM
.
Are tailored health education materials always more effective than non-tailored materials?
Health Educ Res
.
2000
;
15
(
3
):
305
315
44
Baggs
J
,
Gee
J
,
Lewis
E
, et al
.
The Vaccine Safety Datalink: a model for monitoring immunization safety
.
Pediatrics
.
2011
;
127
(
suppl 1
):
S45
S53
45
Opel
DJ
,
Taylor
JA
,
Zhou
C
,
Catz
S
,
Myaing
M
,
Mangione-Smith
R
.
The relationship between Parent Attitudes About Childhood Vaccines survey scores and future child immunization status: a validation study
.
JAMA Pediatr
.
2013
;
167
(
11
):
1065
1071
46
Henrikson
NB
,
Anderson
ML
,
Opel
DJ
,
Dunn
J
,
Marcuse
EK
,
Grossman
DC
.
Longitudinal trends in vaccine hesitancy in a cohort of mothers surveyed in Washington State, 2013–2015
.
Public Health Rep
.
2017
;
132
(
4
):
451
454
47
Patel
M
,
Lee
AD
,
Clemmons
NS
, et al
.
National update on measles cases and outbreaks - United States, January 1–October 1, 2019
.
MMWR Morb Mortal Wkly Rep
.
2019
;
68
(
40
):
893
896

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: Dr Dempsey serves on the advisory boards for Merck, Sanofi, and Pfizer and has worked as a consultant for Pfizer. None of these companies played a role in this research, and Dr Dempsey does not receive research funding from these groups; the other authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data