Addressing Myths and Vaccine Hesitancy: A Randomized Trial

This was a prospective online experiment testing a communication intervention aimed at reducing agreement with vaccination myths in parents of young children. Participants were parents of children aged 0 to 5 years. Eligible participants were 18 years or older, residing in Australia, and competent at reading and responding in English. Participants gave written informed consent. The Macquarie University Human Research Ethics Committee granted ethics approval (ref. 5201954658790).

Research company Quality Online Research recruited participants from its accredited online panel, the representativeness of which is obtained by using quota controls according to Australian Bureau of Statistics Census data. The company recruited participants between September 16 and October 30, 2019, inviting them via e-mail or survey technology and offering between A$1.00 and A$3.00 as incentive for participation. The company stopped recruitment when predetermined targets were achieved.

At baseline, participants responded to myth agreement, intention to vaccinate, and vaccine confidence items (see Materials for definitions). Participants were randomly assigned to receive 1 of 3 debunking interventions or a control text. Immediately after the intervention, participants responded to myth agreement and intention to vaccinate items again. Participants were also asked to provide demographic information. For quality control, participants were asked to summarize the intervention text in a free-response text box. After 1 week, participants were invited to complete a follow-up survey responding to myth agreement and intention to vaccinate items. They had up to 3 weeks to respond. At the close of study, participants were given a debriefing statement with credible information correcting vaccination myths used in the intervention.

The study aimed to recruit 452 participants to ensure a sample size of 376 participants (with an expected 17% loss to follow-up), calculated to allow detection of an effect size of d = 0.5 when comparing change in myth agreement (primary outcome) between groups (see Supplemental Information for sample size calculations). This study was powered at 80% to be confirmatory for the primary outcome. Participants with incomplete surveys or poor-quality free responses (off-topic, unclear, unanswered) or who responded too quickly (determined a priori by the research company as per their quality control measures) were excluded by the research company.

Of the 788 parents of children aged 0 to 5 years who consented and were randomly assigned, 454 (58%) completed the follow-up survey and were included in the analysis (Fig 1). Of the 454 participants included, 63% were female (284 of 454), 56% (255 of 454) were aged between 30 and 39 years, 60% (272 of 454) had a household income of $80 000+ per year, and 61% (275 of 454) had university qualifications.

Mean response time between baseline and follow-up survey was 16 (SD = 5.55) days. Of the 788 randomly assigned participants, 14% (107 of 788) were excluded because of their poor-quality responses, while 29% (227 of 788) failed to respond to the invitation to complete a follow-up survey; this attrition was higher than the expected 17%. There was no significant difference in exclusion across intervention groups (χ²[N = 788, df =3] = 1.70, P = .64).

Attrition analysis compared sex and measures of vaccine confidence, myth agreement, and intention to vaccinate at baseline. Participants who did not complete the follow-up survey were more likely to be female (χ² = 9.91, P = .007) and have a lower vaccine confidence score at baseline (P = .046, Cohen’s d = 0.15). There was no evidence of a difference in myth agreement or intention to vaccinate at baseline between participants who did and did not complete the follow-up survey. There was no significant difference in attrition across intervention groups (χ²[N = 681, df =3] = 2.85, P = .42).

Participants were asked to read a short piece of text (∼350 words) debunking 3 vaccination myths. The 3 myths were “It’s better for children to develop immunity from diseases”; “It’s safer to vaccinate babies and young children when they are older”; and “Vaccines overwhelm a baby's immune system.” The text was modified from a resource addressing common vaccine misperceptions developed to support health care providers’ consultations with parents.²⁶ 

Each intervention (myth, question, or statement) used a different debunking strategy to counter the myths. The myth intervention repeated the vaccination myths (“Myth: Vaccines overwhelm a baby’s immune system”) in the headings before providing corrective text. The question intervention posed questions (“Can vaccines overwhelm a baby’s immune system?”) in the headings before providing corrective text. The statement intervention made factual statements (“A baby’s immune system to able to respond to a vaccine and fight germs at the same time”) in the headings before providing corrective text. The corrective text was the same for each intervention (Fig 2); only the headings differed between interventions. Participants in the control group were given unrelated text with a similar length and structure about parenting strategies. Survey software required participants to view this page for a minimum of 30 seconds (see Supplemental Information for full intervention texts).

Myth agreement was assessed with 3 items, by using a 5-point scale (1 = strongly disagree, 5 = strongly agree). The responses to each of the 3 individual vaccination myths described above were averaged to create a myth agreement score, which revealed high internal consistency at baseline (α = .84), immediately after the intervention (α = .85), and 1+ weeks after the intervention (α = .84). Intention to vaccinate was assessed with a single item, by using a 0 to 100 scale (0 = definitely not, 100 = definitely). Myth agreement and intention to vaccinate items were consistent with survey questions used in studies with similar parent populations.^8,27  Vaccine confidence was measured by using the 4-item short form of the Vaccine Confidence Scale (benefits factor)^28,29  (5-point scale, 1 = strongly disagree, 5 = strongly agree; α = .85) and were averaged to create a vaccine confidence score.

The primary outcome measure was the change in myth agreement, calculated as the difference from baseline at each time point after the intervention. The primary analysis compared mean change in myth agreement between groups, at each time point after the intervention. For this analysis, independent samples t tests were used, adjusted for multiple comparisons between groups with Bonferroni correction (P < .0083; confidence intervals [CIs] of 99.17%). Cohen’s d was calculated to describe the magnitude of intervention effects.³⁰  Observational within-group changes in myth agreement from baseline were also analyzed by using repeated measures analysis of variance (ANOVA). The findings of the difference-in-difference analyses were confirmed with a repeated measures analysis of covariance (ANCOVA) (see Supplemental Information).

A secondary outcome measure was the change in intention to vaccinate, calculated as the difference from baseline at each time point after the intervention. Changes between groups were compared by using independent samples t tests. Observational within-group changes from baseline were analyzed by using repeated measures ANOVAs. All analyses were conducted by using SPSS (version 25; IBM SPSS Statistics, IBM Corporation).

A prespecified subgroup analysis included data from 217 moderate-low vaccine confidence participants only (48%; 217 of 454). Participants were categorized as moderate-low vaccine confidence if their vaccine confidence score measured at baseline (5 point scale, 1 = strongly disagree, 5 = strongly agree) was ≤4.38. High vaccine confidence participants (score >4.38) (52%; 237 of 454) were excluded. These categories are based on results of a previous study in parents of young children, in which a score of ≤4.38 (converted from a score by using an 11 point scale) was associated with delay of any vaccine.²⁹  Mean changes in myth agreement and intention to vaccinate were compared between groups, at each time point after the intervention. Independent samples t tests were used for this analysis, adjusting for multiple comparisons between groups with Bonferroni correction (P < .0083).

The study aims and hypotheses, methods, and data analysis plan were preregistered with the Open Science Framework (https://osf.io/jthn2). A minor variation to the preregistration was to analyze myth agreement as a composite score, with the aim of presenting simple and straightforward results in the article. An analysis of change in myth agreement for each individual myth is retained in the Supplemental Information.

At baseline, mean myth agreement scores were between neutral (3) and slightly disagree (2) (see Table 1). Within-group observational changes in myth agreement, both immediately after the intervention and 1+ weeks after the intervention, are shown in Table 2 and Fig 3.

The primary analysis compared the change in myth agreement from baseline, between groups, at each time point after the intervention. The results of this analysis are shown in Table 3. The null hypothesis for comparing change in myth agreement between posing questions and control immediately after the intervention was rejected: compared with the control group, the question group showed a significant decrease in agreement with vaccination myths of a medium effect size immediately after the intervention (difference between groups: −0.30 points, 99.17% CI: −0.58 to −0.02, P = .004, Cohen’s d = 0.39). We found no clear evidence of a difference between change in myth agreement between the control and other groups or between the groups themselves: there was no evidence of differences between any other groups at this time point or at 1+ weeks after the intervention. This includes the myth group, which did not increase myth agreement compared with the other groups or the control at any time point (see Table 3). There was no evidence of a difference between groups in intention to vaccinate at any time point. The results of the repeated measures ANCOVA aligned with those of the difference-in-difference analyses (see Supplemental Information).

Comparing changes in myth agreement between groups for each myth individually indicated differences between myths. For the “Vaccines overwhelm immune systems” myth, the question and myth groups showed a significant decrease in myth agreement of a medium size compared with the control (difference between question and control: −0.45 points, 99.17% CI: −0.81 to −0.09, P = .001, Cohen’s d = 0.45; difference between myth and control: −0.31 points, 99.17% CI: −0.59 to −0.03, P = .004, Cohen’s d = 0.38). There were no significant differences between groups for the “Disease-acquired immunity is better” myth or the “Delaying vaccines is safer” myth (see Supplemental Table 6 for full results of analysis per myth).

In the subgroup analysis of moderate-low vaccine confidence participants, there was no evidence of significant differences between groups in the change in myth agreement from baseline, at either time point. There was no difference in intention to vaccinate between groups at either time point (see Supplemental Table 5).

The results provide no evidence of a difference between debunking strategies that repeat myths alongside corrective text compared with strategies that do not repeat myths. This is inconsistent with findings from a recent study and reviews assessing the effects of debunking strategies, which suggest that repeating myths may inadvertently increase myth agreement.^16–19  The study revealed that repeating vaccination myths did not perform more poorly than the other debunking strategies. There was clear evidence of a reduction in myth agreement relative to the control group for the debunking strategy that posed questions. Given this followed appropriate statistical adjustment for multiple comparisons, it suggests that a false-positive is less likely to have been detected. No differences in parents’ intention to vaccinate between groups were observed with any strategy.

In an analysis of an important subgroup of moderate-low vaccine confidence parents, the study also revealed no evidence that repeating myths performed more poorly than other strategies. Research comparing groups with low and high vaccine confidence would be worthwhile.

These findings on repeating myths are consistent with conclusions from a recent meta-analysis, which suggests corrective attempts can reduce misinformation effects.³¹  Research comparing message formats for debunking influenza vaccination misinformation has also found that accurate knowledge increases after debunking, regardless of the message format, and that repeating misinformation does not inadvertently increase inaccurate knowledge.²²  Repeating misinformation with corrective text may in fact draw attention to the correction, thereby enhancing, rather than impairing, correct recall.^23,24  In addition to the specific debunking strategies, parents’ agreement with myths in this study may have been influenced by other characteristics of the debunking text, such as succinctness, ease of comprehension, and provision of detailed corrective information.^17,24 

Analysis by individual myth suggests that some misperceptions about vaccines, such as the superiority of “natural” disease-acquired immunity and time-based safety concerns, are more persistent than other myths. Future research may reveal why this is the case. There may be various factors at play, such as the association of an individual’s beliefs about “unnatural” (vaccine-acquired) immunity with moral values related to bodily purity versus degradation.³²  Equally, the novelty of a myth to an individual may render corrections ineffective.³³ 

This study has implications for how health professionals, global health authorities, and other advocates of vaccination debunk vaccine misinformation in written text. In contrast with existing debunking guidelines, this study’s findings suggest that repeating vaccination myths is not necessarily harmful. Any repetition of myths, however, should be done judiciously and be accompanied by corrective text, because familiarity may still lead to myths being misremembered as facts. Furthermore, it has been proposed that repeating new or obscure misinformation may unintentionally help disseminate it to broader audiences.^24,34  This is a particular consideration on social media platforms, where attempts to counter novel vaccination myths may increase their prominence by amplifying their reach, increasing audience exposure. Recent experiments repeating novel misinformation about a range of topics found no evidence of repetition leading to stronger misconceptions.³⁵  Experiments with vaccination misinformation specifically would be worthwhile conducting, as would further investigations of the relationship between novel vaccine misinformation and social media amplification.

This research was conducted in parents of children <5. This is a key strength of the study because it reveals generalizability to a population of key public health significance, especially given the high risk of exposure to vaccine misinformation parents experience online, and the limited understanding of how they respond to various debunking strategies in this context.

Measuring responses to just 3 myths may have limited the ability to draw conclusions about effective strategies to debunk other myths, such as new myths (like those about COVID-19 vaccines), prominent and persistent myths like the alleged link between vaccines and autism, or those about vaccines for specific diseases like measles, pertussis, or human papillomavirus. Not attributing the debunking text to a credible source may have lessened participants’ willingness to accept the debunking information. Although the sample was intended to be representative of the population, respondents analyzed were not, which may impact on the external validity of the findings. Parents’ vaccination intention, rather than uptake, was measured as an outcome. Although in keeping with similar studies, uptake would provide a more accurate measure of vaccination behavior. Furthermore, no significant findings for parents’ vaccination intentions were observed. Further investigation of how parents’ agreement with vaccination myths is associated with intentions and behavior is warranted, as is research into what types of debunking interventions may improve intentions. Finally, in this study, myth agreement was analyzed as a continuous variable. Informal analysis of the data as categories of agreeing and disagreeing parents (not included here) suggests exposing parents to vaccination myths without corrective text may increase myth agreement. This effect is worth investigating further in future research.

In this study, repeating myths as a debunking strategy did not appear to be inferior to strategies that do not. Posing myths as questions may be an effective debunking strategy when paired with corrective text. Further research should elucidate why some myths are more persistent than others and evaluate debunking strategies for novel vaccination myths and those that change behavior.

We acknowledge the contributions of Noel Brewer and Ullrich Ecker in critiquing and improving this article.

ANCOVA

analysis of covariance

CI

confidence interval

COVID-19

coronavirus disease of 2019