Financial Interventions to Increase Vaccine Coverage

Deidentified data were extracted from the Australian Immunization Register as at March 31, 2019, covering the period 2009–2017. To calculate vaccination coverage, we used the cohort method,³¹  which has been used for calculating vaccination coverage at the population level (national and state and/or territory) since the inception of the register, together with the nationally agreed definitions of fully vaccinated, which have been in place over the study period.^22–30  The proportion of children who met the definition for fully vaccinated at age 12 months (for vaccines due at 6 months), 24 months (for vaccines due at 6, 12, and 18 months), and at 60 months (for vaccines due at 48 months) were calculated on a quarterly basis from 2009 through to 2017^22–30  at the Statistical Area 3 (SA3) level. There are 325 SA3s in Australia. We also calculated the proportion of children with recorded conscientious objection within each SA3, for each year over the period 2009–2015. Data at the SA3 level were also aggregated (weighted by the relevant population of children in each SA3) to the state and national level.

We used time series analysis methods to identify and adjust for potential seasonal effects in vaccination rates. Autoregressive Integrated Moving Average (ARIMA) models are a widely used framework that allow testing for changes in trends in a time series in several ways, including testing for a jump in the trend at a particular time, an ongoing change in trend after a point in time or more complex intervention effects. We used data before the first date of policy implementation to model the time series structure of the data. We modeled the proportion of children who did not meet the definition of fully vaccinated at that age. First, we decomposed the time series into seasonal, irregular and trend components to help identify the most appropriate time series structure. This included identifying if differencing was required to produce a stationary time series, and what order of differencing was needed, as well as the optimal combination of autoregressive and moving average components to most effectively model the seasonal fluctuation in the time series. We then used the full run of data from 2009 to 2017 to fit an interrupted time series ARIMA model that tested for both a break in series at the time of policy implementation and an ongoing change in the direction of the trend in the series after this time.

We employed different methods for Australia’s most populous state, New South Wales. In 2014, New South Wales introduced the country’s first NJNPlay policy excluding unvaccinated children from preschool (although it initially still permitted Conscientious Objections). Because New South Wales’s policy was introduced before other states, and also before the Australian government’s NJNPay, we fitted a model to New South Wales data with 2 interruptions to the time series, reflecting both interventions. (New South Wales’s abolition of conscientious objections in 2018 occurred outside our study period.)

Although we had data at the SA3 level, it was not possible to test for any changes in the time series at this low level of aggregation because of the high degree of variability in the data at this level. Accordingly, we fitted interrupted time series models at the national level, at the state level, and nationally grouping SA3s into 3 groups based on their level of conscientious objection to vaccinations. In this final grouping, we sought to ascertain whether the mandatory policies were having a greater impact in communities where people had previously been formally refusing vaccines, as opposed to communities where undervaccination might be better explained by access issues. Because the proportion of children with recorded conscientious objection in each SA3 was relatively stable over the time period 2009–2014, but appeared to drop in the final year of 2015, we averaged the proportion of these children in each SA3 over the time period 2009–2014. We then aggregated all SA3s into 3 groups, which we labeled low conscientious objection if the proportion of children was <2%, medium if the proportion was 2% to 4%, and high if the proportion was >4%.³²  In total, 158 SA3s had a low level of conscientious objection, with 59.7% of children <12 months living in them; 132 SA3s were classified as medium, with 33.7% of children, and 33 SA3s were classified as high, with 6.6% of children <12 months.

In Figure 1, we show the proportion of children not fully vaccinated at 12, 24, and 60 months by quarter for Australia. Initial inspection of the data identified 2 clear anomalies. At 60 months, there was a clear reduction of 8% in the proportion not fully vaccinated in the first quarter of 2010, attributed to changes in the assessment algorithm flowing from changes to the immunization schedule. Consequently, we excluded the 2009 data from the analysis of the 60-month time series. We also observed a substantial 6% increase in the proportion of children not fully vaccinated at 24 months in the third quarter of 2014 (Fig 1). This was also attributed to changes in coverage assessment algorithm for this milestone. Because this occurred close to the time of the policy changes, and had such a large impact on the trend series (by 2017 the proportion fully vaccinated at 24 months was still lower than it had been in 2014), it was not possible to test for policy-related changes in this particular time series. As a result, we focused exclusively on the 12- and 60-month time series.

We determined the best fitting time series models as follows. Because the rate of fully vaccinated coverage had been increasing in the period of 2009–2015, 1 order of differencing was applied to produce a stationary time series. The optimal ARIMA model to describe the seasonal nature of the data varied between jurisdictions, but most commonly an ARIMA (1,1,1) model provided the best fit. We visually inspected residuals and used Q-tests to ensure there were no residual unexplained patterns over time and tested for white noise with the Ljung–Box statistic.

In Figure 2 is shown the model fitted to the 12-month data series at the Australian level. Before 2016, there was an average annual decrease in the proportion of 12-month-olds not fully vaccinated of 0.49 percentage points per annum (95% confidence interval: 0.27–0.71). At the time of the policy change, there was a 1.03 percentage point reduction in the proportion fully vaccinated, which was not statistically significant (P = .08). After the policy change, it was estimated that there was an additive increase in the annual rate of decrease in the proportion of 12-month-olds fully vaccinated of 0.55%, so that after the policy change the annual decrease in proportion fully vaccinated was 1.04%, although this change in slope was not statistically significant (P = .28).

In Figure 3, we show an ARIMA model fitted to New South Wales data with 2 interruptions to the time series. Both interruptions were fitted with a possible change in level or jump in the time series at the intervention start point, and a change in slope from that time point. Neither intervention was associated with a statistically significant jump (−0.098 when NJNPlay was introduced, P = .77; −0.57 when NJNPay was introduced, P = .20). However, there was a significant change in slope. Before the first intervention point, the slope of the time series was not significantly different from 0 (0.015% annual change, P = .35). After NJNPlay there was a change in slope of −1.33 percentage points per year (P < .001). After NJNPay, there was a further change in slope of 0.39 percentage points per year (SE = 0.375, P = .31). Although this change was not statistically significant compared with the period between the implementation of NJNPlay and NJNPay, the overall rate of change in the proportion not fully vaccinated at 12 months was significantly different from the rate before the introduction of NJNPlay at 0.91 percentage points per year (P = .032), meaning that in combination, both policies were associated with an increase in fully vaccinated rates.

In Table 2, we show the modeled regression coefficients when fitting separate ARIMA models for each State or Territory. In all states there was a statistically significant annual rate of decrease in the proportion of 12-month olds fully vaccinated. There was a reduction in the proportion not fully vaccinated in all states when NJNPay came in, although this was statistically significantly different from zero only in Queensland and Tasmania. In all states there was a larger annual rate of decrease after the policy intervention, but this was not statistically significant in any state, including Victoria, which introduced NJNPlay concurrently with the federal policy change (meaning we could not explore its effect independently). The relatively small number of data points currently available after policy implementation again reduced the statistical power of the test. The fitted models in each state are shown in Supplemental Fig 4.

The overall annual rate of change before the policy intervention was similar by level of conscientious objection across the 3 groups. The one-off changes in the time series at the time of intervention were not significant in any of the groups. There was an increase in the rate of change postintervention in all states, but again, this was not statistically significant in any of the 3 groups. Modeled coefficients are shown in Table 3, and fitted models are shown in Supplemental Fig 5.

Considering the proportion of all Australian children fully vaccinated at 60 months, there was an annual decrease in the proportion not fully vaccinated of 1.25% per annum, which was statistically significantly different from 0 (Supplemental Fig 6). However, there was no evidence of any break in series at the time of the policy intervention, or any statistically significant change in the slope of the time series afterward, in any state (Supplemental Table 4, Supplemental Fig 7). Similarly, there was no evidence of any change in the time series at or after the intervention date by level of conscientious objection (Supplemental Table 5).

To further explore the question of whether level of conscientious objection was associated with policy impacts, we modeled the time series by 3 levels of proportion of conscientious objections in the 3 largest states, New South Wales, Victoria, and Queensland. Modeled coefficients are shown in Supplemental Table 6, and time series are displayed in Supplemental Fig 8 (New South Wales), Supplemental Fig 9 (Victoria), and Supplemental Fig 10 (Queensland). Similar trends were seen across all 3 levels of conscientious objection in Victoria and Queensland. Although there was an increase in the rate of change in areas with a high level of conscientious objection in New South Wales after introduction of NJNPay, this was not statistically significant.

Vaccination coverage in Australia has been trending upwards since before recent vaccine mandates were introduced,³⁰  but until now, data have not been subjected to time series analysis, which can account for policy changes and consider their effect on overall trajectory. Our analysis reveals that from 2009 to 2014, fully vaccinated coverage at 12 and 60 months of age was already improving in all states. From 2014, when first New South Wales and then other states began to implement increasingly restrictive mandates, the positive trend continued with ongoing increases in the proportion of children fully vaccinated. We were not able to conclusively demonstrate any impact of the federal NJNPay and state NJNPlay policies on the trajectory of increase, in isolation from each other, at either national or state level. This suggests the policies may have largely reinforced existing strategies to reach Australian children and motivate their parents to vaccinate, which we elaborate below. We did, however, find a statistically significant impact on vaccination coverage at 12 months of age in New South Wales when assessing the policies in combination. New South Wales’s NJNPlay policy potentially had a broader impact than the federal policy, because it targeted unvaccinated parents at all income levels. Although the policy retained a nonmedical exemption process for refusers, this cohort comprises a smaller proportion of unvaccinated children compared with those who are unvaccinated for other reasons,³²  and hence motivated to change by the policy. By contrast, although the federal NJNPay policy exerted a stronger lever over vaccine refusers, the payment predominantly targets lower income families, potentially limiting its overall reach. Additionally, the longer time period available after the introduction of NJNPlay in New South Wales, compared with the federal and Victorian policies, both implemented in 2016, may have provided more statistical power.

The continued upward trajectory of vaccine coverage rates before and after the introduction of these restrictive mandatory policies might also mask their true impact. It is plausible that these policies prevented rates from stalling in the low 90s, and instead maintained their rate of increase into the mid 90s. It is notoriously difficult to obtain those last few percentage points of coverage.³³  In the Australian context, ∼2 percent of children had registered objections, and a further 1.3% were estimated to be incompletely vaccinated because of unregistered objection.³²  At least that number again remained poorly reached by government efforts. The new policy and heightened attention to vaccination in public discourse may have helped to push parents of children in all these categories toward vaccination.

With our study, we illuminate that mandatory policies may not have much impact on coverage rates in communities with high numbers of refusers. Before NJNPay, Australia still had vaccination requirements for preschool aged children to access benefits, but with nonmedical exemptions available after medical counseling. This resembled the “optimal” policy³⁴  evident in Washington (2011–2019) and California (2014–2016),^3,4  in which bureaucratic procedures made nonmedical exemptions hard to obtain. Such an approach nudges those who are complacent or experiencing access issues to vaccinate, but eschews compelling objectors. There are no published Australian data on the eventual vaccination status of children whose parents previously registered objections. Accordingly, analyzing changes by area-based level of objection is the best available proxy. Our results show little variation between regions with high, medium or low historical objection rates. Given the relatively low impact of NJNPay in refuser communities, it is likely that the policy’s greater impact lay in the expansion and heightened frequency of compliance assessment. This is consistent with a recent study which found that the proportion of Australian children aged 5 to <7 years who received catch-up vaccination in the first 2 years of NJNPay, compared to the 2 years prior, was 65% higher for the third dose of DTPa vaccine but 5% lower for the first dose of MMR vaccine,³⁵  suggesting limited impact of the policy on families who refuse all vaccines.

If we cannot discern NJNPay’s impact on refusers, this may be further confounded by the fact that NJNPay does not cover all Australian families equally. The Australian government withholds family assistance payments, which are only available to families earning under a particular income threshold, as well as child care subsidies, from families whose children are not fully vaccinated. This means that the policy leaves high-income earners who do not use child care completely free from vaccination policy levers. Some may still have been motivated to vaccinate by the heightened public discourse or to their desire to access child care or kindergarten in NJNPlay states. However, our state-based results are either confounded by cointroduction with the federal policy or not statistically significant.

Vaccination coverage has trended in the direction of more families vaccinating, indicating that Australia’s “permissive mandates” with exemptions¹⁷  before 2016 were achieving results, especially when combined with complementary strategies employed by governments to promote vaccination. We have already noted that scheduled vaccines in Australia are free and that the Federal Government incentivizes doctors to record them on the electronic register (and thus to deliver them). Governments have studied which children are unvaccinated and why to understand how to address coverage gaps.³⁶  They have also successfully mobilized resources to reach indigenous Australians,³⁷  a disadvantaged cohort.

This kind of nonmandatory investment in and promotion of vaccination in Australia was further enhanced following the introduction of NJNPay from 2016. For example, the Federal Government doubled the A$6 vaccination milestone payment to doctors when they brought overdue children up to date³⁸  and invested in a communications campaign to convince parents of the importance of vaccination.³⁹ 

Because these nonmandatory policy strategies appear to have delivered benefits in terms of coverage before and following the implementation of the restrictive “No Jab” mandates, other jurisdictions would be well-advised to consider what nonmandatory methods can achieve.⁴⁰ 

There are several limitations to this study. There were only a relatively small number of data points available post policy implementation, which presented some limits on the statistical power of the analyses. It is possible that there were changes in some jurisdictions smaller than detectable by our analyses. Re-examining the data when a longer time series is available may be informative.

Another limitation was the substantial changes to the Australian immunization schedule (and subsequent changes to the algorithms defining fully vaccinated at each of the 3 assessment milestone ages) complicating the analysis of policy impacts. This was particularly true for vaccination coverage at 24 months, which we had to exclude. Additionally, the algorithm for assessment of fully vaccinated at 60 months only includes vaccines due at 4 years of age (ie, it does not include any vaccines given at earlier milestones). The narrowness of the snapshot this provides of full coverage could have contributed to our lack of findings of policy impact at the 60-month age milestone. We were also unable to assess potential impacts on coverage due to government efforts to improve data reporting and quality.⁴¹ 

Another limitation emerged from the simultaneous introduction of state and federal “No Jab” policies in Victoria. There was no way to tease apart these multilevel policies. We also note that state policies differ in design. Local demographic and institutional factors, as well as additional vaccination promotion programs may have had an impact on coverage rates.

Recent vaccine mandates in Australia, as in other high income settings, have sought to change the behavior of parents, including those who would otherwise access nonmedical exemptions. We subjected highly visible policy changes at both state and federal levels to time series analysis to determine their impact on the number of children who are not fully vaccinated. Notwithstanding a range of confounders, including schedule and algorithm changes and simultaneous introductions of policies at different levels of government, we failed to provide, with our analyses, conclusive evidence of impact of the policies on vaccine coverage rates, with the percentage of not fully vaccinated children generally falling steadily both before and after the policies’ introduction. Because it remains difficult to ascertain the effects of any single policy contributing to Australia’s vaccine coverage trajectory, policymakers overseas should be wary of attributing the country’s coverage successes to its recent vaccine mandates.

The UWA team thank Vaille Dawson and Julie Lee for their research mentorship.

ARIMA

Autoregressive Integrated Moving Average

MMR

measles, mumps, and rubella

NJNPay

No Jab, No Pay

NJNPlay

No Jab, No Play

SA3

Statistical Area 3