Text Message Reminders for the Second Dose of Influenza Vaccine for Children: An RCT

The Flu2Text randomized controlled trial (clinicaltrials.gov: NCT03287830), funded by the National Institutes of Health [R01HD086045], took place over 2 influenza seasons (September 2017 to April 2018; September 2018 to April 2019). We recruited 50 pediatric primary care practices; 46 were from the American Academy of Pediatrics’ (AAP) national Pediatric Research in Office Settings primary care network, 3 from Children’s Hospital of Philadelphia (CHOP), and 1 from Columbia University (Northeast [28%], South [36%], Midwest [16%], West [20%]). Practice inclusion criteria were: (1) able to recruit minimum of 40 caregivers per season; (2) access to office medical records and/or city and state immunization information systems to confirm influenza vaccine dates; and (3) not already using or plan to use text message reminders for second dose influenza vaccine. In total, 203 practices were approached, with an emphasis on a mix of census regions, urbanicity and patient demographics.

Caregivers were eligible to participate if they spoke English or Spanish, had a cell phone with text messaging capabilities, and had a child who needed 2 doses of influenza vaccine that season.¹⁹  In both seasons, caregivers were introduced to the study at their child’s pediatric primary care office. During the 2017 to 2018 season, when an eligible child received their first influenza vaccine dose from date, the clinician shared a study number with caregivers who then texted to indicate their interest. Research staff called caregivers to complete consent and enrollment. For the 2018 to 2019 season, to counter loss of potential enrollees between indicating interest at the visit and the enrollment call, caregivers verbally consented and were enrolled by their child’s clinician at the child’s first influenza vaccine visit. Documentation of consent and key demographic characteristics were recorded by the clinician at enrollment and a link to a more comprehensive demographic survey was texted to caregivers. Participants were enrolled through mid-March and could only enroll in 1 season.

We used central randomization of caregiver-child dyads with a 1:1 allocation, stratified by practice site, child’s age group (6 to 23 months; 2 to 8 years), and caregiver’s preferred language for text messaging. The randomization lists, based on permuted blocks with varying block sizes, used the module “ralloc,” which supports allocation concealment and achieves balance of treatment assignment within strata throughout patient recruitment (StataCorp, v15 and 16, College Station, TX). The statisticians, clinicians and practice staff members remained blinded to arm assignment until analysis was complete.

Caregivers in the intervention arm received personalized, scheduled text messages including (1) reminders about an upcoming second dose and (2) educational information about the importance of a second dose. Messages were based on a successful previous study,²⁴  and modified based on feedback from pediatric and health literacy experts on the study team. The messages and study materials were professionally translated into Spanish. We pretested the English and Spanish messages with 30 caregivers from primary care practices at CHOP and Columbia. Messages were automatically sent centrally using a text messaging service on days 14 and 21 postvaccination to alert families about the next needed dose, day 25 just before the end of the minimal interval between doses, day 28 when the dose was due, and day 42, 2 weeks after the due date. The day 21 message was interactive, allowing caregivers to request further information about vaccine side effects, the necessity of the second dose, and/or the importance of timely second dose receipt (Supplemental Table 4). Messages were personalized with the child’s first name, gender, and practice name and included the due date for the child’s second influenza vaccine dose. Depending on how the practice primarily offered the second dose of influenza vaccine, practice specific walk-in hours or a phone number to call for an appointment was included.

In season 2 (2018 to 2019) only, caregivers randomized to usual care were sent a message upon enrollment with a link to the AAP’s Ages and Stages Web site as a 1-time attention control.²⁶  Usual care had variable reminder systems, including no second dose reminders or letter, phone, e-mail or patient portal messages, or a written card provided at the first dose.

The primary outcome was the receipt of the second influenza vaccine dose by the end of the influenza season (April 30^th), referred hereafter as “season end.” Secondary outcomes were timeliness of the second dose as assessed by receipt of dose by 42 days (represents a 2-week grace period after the 28 day minimal interval between first and second doses¹⁹ ), and time elapsed from first to the second dose. Practice staff blind to study arm assignment abstracted vaccination data from the medical record within each practice and from applicable vaccine registries.

Practice-level characteristics (typical scheduling approach [appointment scheduling or walk-in for second dose of influenza vaccine] and practice urbanicity [inner-city urban or noninner city urban, suburban, or rural]) were self-reported by practices before the trial. Key caregiver and child characteristics were collected at enrollment for both seasons: child’s age group, gender, insurance status, and caregiver’s preferred language for texting (English or Spanish). Public insurance and uninsured groups were combined for analyses since both were eligible for free vaccine in the Vaccine for Children program and because of the small size of the uninsured group (n = 15). Additional demographic characteristics were collected for those who completed the demographic enrollment survey. These included child ethnicity, child race, caregiver reported child health (excellent, very good, good, fair, and poor), caregiver’s relationship to the child (mother, father, grandparent, and other), and caregiver age. Because of nonresponse on the demographic enrollment survey in the 2018 to 2019 season, we reweighted each response by the inverse of the probability that the caregiver of a patient of a given age, gender, and language preference answered that survey question, so that survey responses reflect the entire sample characteristics.²⁷ 

Assuming 50 sites with 40 caregivers (20 per arm) per site, with a 2 sided test (α = .025 per tail), power was 0.8 to demonstrate an increase in vaccination of ≥6% points; power would increase if across-site variability was reduced. To assess the primary outcome of vaccine receipt (yes or no) by season end, we implemented Mantel Haenszel methods, stratifying by practice site and language to reflect randomization strata. For clarity, we reported effects on the absolute rather than relative scale. For that reason, we programmed in Stata v15 and validated against published data the algorithm of Bohning and Sarol for estimating stratified risk differences as of the end of influenza season.²⁸ 

For the secondary outcome of time from first to second vaccine dose, we implemented the Mantel Haenszel stratified methods for the outcome of vaccination by day 42 after the first dose and conducted a conventional survival analysis by means of the survival library in Stata v15 and 16. Kaplan-Meier analyses displayed the time from receipt of the first to second dose, censored at season end. We compared time-to-event using the Wilcoxon version of the log-rank test with start as the first dose date. As a sensitivity analysis, we repeated with the start date as the randomization day. We deliberately avoided estimation of hazard ratios owing to well-described limitations of the metric, and because of its lack of meaning in our study.²⁹  Therefore, we estimated differences between study arms in cumulative incidences (percentage of patients who received the second dose) as of day 42 and confidence intervals of those differences, based on these survival curves using the method of pseudo-observations of Andersen and colleagues in the program “stpsurv” for Stata.^30,31  As a sensitivity analysis, we replicated the day 42 analysis as of day 60.

Prespecified subgroup analyses (by child insurance, caregiver’s preferred language for texting, practice approach to providing second-doses, and practice urbanicity) proceeded along similar lines, with contrasts based on time-to-event curves, and differences in cumulative incidences (and confidence intervals) using pseudo-observations.³¹  Confirmatory confidence bounds were estimated using nonparametric bootstrap methods with 999 samples, with replacement, to calculate 95% confidence bounds using the percentile method.

A formal statistical analysis plan was drafted and agreed upon before the start of analysis. This study was approved by the Institutional Review Boards at Columbia University, CHOP, the AAP, and the University of South Carolina.

A total of 2086 eligible caregivers (2017 to 2018 season [n = 257]; 2018 to 2019 season [n = 1829]) were enrolled (Fig 1). Intervention and usual care participants had similar demographic composition (Table 1). Below we report results for the combined and individual seasons in intention to treat analyses.

Over the 2 seasons, text message reminders increased second dose of influenza vaccine receipt by season end (April 30^th). Overall, 83.8% of children in the intervention arm received the second dose by season end versus 80.9% in usual care (effect size: adjusted risk difference (ARD) 3.8%; 95% confidence interval [CI] [0.1 to 7.5]) (Table 2). The effect size by season end in the 2018 to 2019 season was similar to that observed in combined analyses across seasons (83.2% versus 80.1% (ARD) 3.8%; 95% CI [−0.4 to 8.1]). The effect size for the 2017 to 2018 season (n = 257), especially after adjustment was smaller (88.3% versus 86.8% (ARD) 0.3%; 95% CI [−4.9 to 5.6]) than observed in the 2018 to 2019 (n = 1829), and combined seasons.

The intervention significantly improved receipt of second dose by day 42, with 62.4% of children in the intervention arm receiving the dose vs. 55.7% in usual care (ARD 8.3%; 95% CI [3.6 to 13.0]) (Table 2). Effect sizes of similar or greater magnitude were seen for each of the 2 individual seasons (2018 to 2019: 60.7% versus 54.7% ARD 7.3%; 95% CI [2.3 to 12.3]; 2017 to 2018: 74.2% versus 62.8% ARD 17.7%; 95% CI [4.7 to 30.7]). The number of participants needed to text to have 1 extra child be vaccinated by day 42 was 12 (95% CI [8 to 28]). Kaplan-Meier analyses (Fig 2) also demonstrated that the intervention arm received more timely vaccination through the season. Individual season results were consistent (Supplemental Fig 3). In the sensitivity analysis, by day 60, 69.8% of children in the intervention arm received the second dose versus 63.1% in usual care (ARD 7.4%; 95% CI [2.9 to 11.9]).

We also compared intervention effectiveness by practice typical scheduling approach for the second dose of influenza vaccine (appointments [n = 40] versus walk-in hours [n = 10]) and practice urbanicity (inner-city urban [n = 11], noninner city urban [n = 9], suburban [n = 18], and rural [n = 12]). The intervention produced a bigger effect for practices which primarily had walk-in influenza vaccine hours versus practices which primarily scheduled second dose appointments (Fig 3, Table 3). In contrast, the intervention produced a similar positive effect for all 4 urbanicity groups both by season end and day 42.

The intervention led to more timely vaccination among commercially insured than publicly insured children (Fig 3, Table 3). In contrast, preferred text language was not an effect modifier; the intervention produced a similar positive effect for English (n = 1930) and for Spanish-speakers (n = 156), both by season end and day 42. Only 17 practice sites had Spanish speakers, with the majority of the 156 children receiving care in 2 of the practices.

In this multistate trial, text message reminders were effective in increasing delivery of the second dose of influenza vaccine. This potentially scalable and accessible intervention could be implemented as most adults in the United States own a cell phone and use texting often.^23,25  Perhaps most importantly, children of caregivers randomized to the intervention arm received second doses sooner. Timely receipt of the second dose is critical so that children are fully vaccinated before influenza begins circulating.

The effect sizes we observed are likely to be clinically meaningful. According to a recent modeling study, a mere 5% increase in nationwide influenza vaccination coverage could have prevented 228 000 illnesses and 4900 hospitalizations during seasons with typical influenza severity across all age groups.³²  For high-severity seasons such as the 2017 to 2018 influenza season, the impact could be even greater. In that season, vaccination was estimated to have prevented over 2.1 million illnesses, 1.4 million medical visits, 14 790 hospitalizations and 74 deaths among young children (6 months to 4 years).³³  The consistent impact of text message reminders on timeliness of second dose vaccination observed in our study might be especially important, because timely influenza vaccine coverage remains a problem in the United States.²⁰  The Centers for Disease Control and Prevention recommends people be fully vaccinated before influenza starts circulating,¹⁹  which can happen as early as October.³⁴  For young children who need 2 doses to be adequately protected, receiving both doses before the start of the influenza season can be logistically difficult.

A relatively newer concept of precision public health (and related precision primary care) suggest that different strategies may be needed to best serve distinct patient and practice populations.³⁵  Consistent with the tenets of precision care, we identified practice and family-level characteristics that modified intervention effects. Even more investigation may therefore be needed to understand how to best implement this type of intervention for maximal impact across a variety of populations. This result advances knowledge of vaccine reminders via text messaging, since prior studies focused primarily on lower income or single-area populations.^{16,20,36–39}  An encouraging finding was that our bilingual intervention worked similarly for both English and Spanish-speaking caregivers, although the number of Spanish-speaking caregivers was small and limited to certain practices.

We saw a larger intervention effect for practices that primarily relied on walk-in influenza vaccinations. Although walk-in vaccination offers flexibility and convenience, it puts recall responsibility on caregivers’ shoulders. For these caregivers, the text messages may function as an especially important “cue-to-action” to bring their child back for the second dose during the provided walk-in hours. Often arranged at the time of their last visit, scheduled appointments conversely tradeoff caregivers needing to remember to return with decreased flexibility.

Our study has several limitations. First, our participants agreed to participate in a text messaging study and may not be representative of the larger population. As this study was conducted pragmatically in a wide practice range, we do not know how many caregivers opted not to participate. However, this enrollment strategy reflects how texting might be implemented in practices where families would choose whether to receive reminders. Second, our usual care arm had much higher receipt of second doses than observed in previous studies, which could have blunted season end effects; this may be partly caused by many enrollees being 6 to 23 months-old who have higher 2 dose completion rates.^14–18  Practices with low baseline rates of return might realize even greater effects. Third, because of a technical error with the text message distribution, 253 of 913 usual care participants in season 2 received at least 1 intervention text message (105 received 1 extra, 148 received 2 or more). As all analyses were intention to treat, this could have blunted our effect size. In addition, we did find that usual care participants who received more messages were slightly more likely to be vaccinated at day 42 than those who did not, highlighting the possible benefits of text messaging. Fourth, because of survey nonresponse, we do not have the full set of demographic variables for all participants; therefore, we limited analyses to variables we decided a priori to capture at enrollment. Additionally, although this study involved a convenience sample of heterogeneous participants in many states, it was not designed to be nationally representative. Finally, we centrally implemented the reminders on behalf of practices; future studies may need to assess text message implementation by practices themselves or their chosen vendors. Despite these limitations, a key strength of this study is that it included over 2000 families, practices in multiple regions, and both English and Spanish speakers.

In this large-scale trial in practices across the country, we demonstrated the effectiveness of using personalized text message reminders to increase overall receipt and the timeliness of the second dose of the influenza vaccine. A next step would be to assess how text message reminders for the second dose of influenza vaccine could be broadly implemented in practice settings serving children.

We thank all Flu2Text practices (listed below), pediatricians, nurse practitioners, other physicians, staff, caregivers, and families who participated in our study, and the University of South Carolina Institute for Public Service and Policy Research, Survey Research Laboratory.

Flu2Text practices participating in this study, named here with their permission, included: ABC Pediatrics; Advanced Pediatrics, PC; All Pediatrics, PC; Altru Health System; Anaconda Pediatrics; Anchorage Pediatric Group; Ashley Clinic; Atlantic Coast Pediatrics; Bethesda Pediatrics; Bozeman Health Pediatrics; Building Blocks Pediatrics; Burlington Pediatrics; Cambridge Pediatrics; Child Health Partners, PC; Childhood Health Associates of Salem; Clinch Valley Physicians Associates – Pediatric Department; Dowd Medical Associates; Elmwood Pediatric Group, LLP; Fishing Bay Family Practice; Goshen – Columbus Pediatrics & Adolescent Care; Hirsch Pediatrics, LLC; Holyoke Pediatric Associates; Ivancic Pediatric Clinic, PA; Mesa Pediatrics; OHSU Doernbecher Pediatrics – Westside Clinic; One Hanson Place Pediatrics, PC; Pediatric & Adolescent Healthy Lifestyle Center; Pediatric Associates of Davidson County, PA; Pediatric Associates of Medford; Pediatrics by the Sea; Pennridge Pediatric Associates; Prattville Pediatrics; Priority Care Pediatrics; Purohit Pediatric Clinic; Quality Kids Kare, PC; Scarano & Taylor Pediatrics; Southeastern Pediatric Associates; Southwest Montana Clinic; Springfield Pediatrics; Sunset Park Family Health Center at NYU Langone; Swafford Pediatrics; The Child & Teen Wellness Center; UNM Pediatrics – 3ACC Faculty Clinic; Zaheer Pediatric Associates, SC; and Zimble & Reinstein Pediatrics.

AAP

American Academy of Pediatrics

ARD

adjusted risk difference