Video Abstract

Video Abstract

Close modal
BACKGROUND AND OBJECTIVES

Among children requiring 2 influenza doses in a given season, second dose receipt nearly halves the odds of influenza. Nationally, many children do not receive both needed doses. This study sought to compare the effectiveness of text message reminders with embedded interactive educational information versus usual care on receipt and timeliness of the second dose of influenza vaccine.

METHODS

This trial took place over the 2017 to 2018 and 2018 to 2019 influenza seasons among 50 pediatric primary care offices across 24 states primarily from the American Academy of Pediatrics’ Pediatric Research in Office Settings practice-based research network. Caregiver-child dyads of children 6 months to 8 years in need of a second influenza vaccination that season were individually randomized 1:1 into intervention versus usual care, stratified by age and language within each practice. Intervention caregivers received automated, personalized text messages, including educational information. Second dose receipt by April 30 (season end) and by day 42 (2 weeks after second dose due date) were assessed using Mantel Haenszel methods by practice and language. Analyses were intention to treat.

RESULTS

Among 2086 dyads enrolled, most children were 6 to 23 months and half publicly insured. Intervention children were more likely to receive a second dose by season end (83.8% versus 80.9%; adjusted risk difference (ARD) 3.8%; 95% confidence interval [0.1 to 7.5]) and day 42 (62.4% versus 55.7%; ARD 8.3% [3.6 to 13.0]).

CONCLUSIONS

In this large-scale trial of primary care pediatric practices across the United States, text message reminders were effective in promoting increased and timelier second dose influenza vaccine receipt.

What’s Known on This Subject

Many children in need of 2 influenza vaccinations in a season don’t receive needed doses at all, nor in a timely fashion. Pediatric text message vaccine reminder studies have not been conducted in diverse primary care practices across the United States.

What This Study Adds

In this randomized clinical trial that included 2086 caregiver-child dyads in 50 primary care offices across 24 states, text message reminders were effective in promoting increased and more timely delivery of the second dose of influenza vaccine.

Annual influenza virus infections in the United States lead to high morbidity, mortality, and societal financial strain.18  Young children, particularly those under 2, are at higher risk of hospitalization and serious complications.9  Children may also introduce the virus into their households.10  The total economic burden of influenza from direct medical expenses and productivity loss is estimated at $11.2 billion dollars annually, with children under 5 contributing nearly 1 billion dollars.11 

The influenza vaccine protects against infection. In the most recent season with published data (2019 to 2020), vaccination averted more than an estimated 1.4 million illness cases, 937 000 medical visits, 9600 hospitalizations, and 82 deaths among children under 5 years.12  Although rare, the majority of children who die of influenza are unvaccinated.13  Young children, especially those under 2, are particularly vulnerable. Children who are 6 months to 8 years getting vaccinated for the first time, and those who have only previously received 1 dose, need 2 doses of vaccine in a given season, at least 4 weeks apart. Receiving both doses nearly halves the odds of laboratory-confirmed influenza versus receiving only 1.14  However, only 40% to 60% of children who need 2 doses and receive the first get their second.1418  In addition, timeliness of vaccination matters, children should be fully vaccinated before influenza begins circulating in late fall or early winter.19,20  However, barriers remain, including gaps in caregiver knowledge of the influenza vaccine schedule21  and vaccine hesitancy that may persist even after first dose receipt.22 

Text messaging is a low-cost, scalable, and accessible way to provide vaccine reminders, potentially overcoming vaccination barriers. Most individuals in the United States own a cell phone with few differences across groups.23  Text message vaccine reminders can prompt families to return for a second dose and offer reliable vaccine education to address common barriers.21 

Previous text message reminder studies, including for influenza vaccine, have demonstrated success, yet have not included diverse practices across many states.24,25  Therefore, our study with a heterogeneous sample of pediatric practices across the United States is critical to understanding the effectiveness and optimal use of these reminders. Our objectives were to assess, through a randomized controlled trial, the effectiveness of personalized text message reminders in increasing both overall receipt and timeliness of the second dose of influenza vaccine. We also evaluated whether caregiver, child, and practice characteristics impacted message effectiveness.

The Flu2Text randomized controlled trial (clinicaltrials.gov: NCT03287830), funded by the National Institutes of Health [R01HD086045], took place over 2 influenza seasons (September 2017 to April 2018; September 2018 to April 2019). We recruited 50 pediatric primary care practices; 46 were from the American Academy of Pediatrics’ (AAP) national Pediatric Research in Office Settings primary care network, 3 from Children’s Hospital of Philadelphia (CHOP), and 1 from Columbia University (Northeast [28%], South [36%], Midwest [16%], West [20%]). Practice inclusion criteria were: (1) able to recruit minimum of 40 caregivers per season; (2) access to office medical records and/or city and state immunization information systems to confirm influenza vaccine dates; and (3) not already using or plan to use text message reminders for second dose influenza vaccine. In total, 203 practices were approached, with an emphasis on a mix of census regions, urbanicity and patient demographics.

Caregivers were eligible to participate if they spoke English or Spanish, had a cell phone with text messaging capabilities, and had a child who needed 2 doses of influenza vaccine that season.19  In both seasons, caregivers were introduced to the study at their child’s pediatric primary care office. During the 2017 to 2018 season, when an eligible child received their first influenza vaccine dose from date, the clinician shared a study number with caregivers who then texted to indicate their interest. Research staff called caregivers to complete consent and enrollment. For the 2018 to 2019 season, to counter loss of potential enrollees between indicating interest at the visit and the enrollment call, caregivers verbally consented and were enrolled by their child’s clinician at the child’s first influenza vaccine visit. Documentation of consent and key demographic characteristics were recorded by the clinician at enrollment and a link to a more comprehensive demographic survey was texted to caregivers. Participants were enrolled through mid-March and could only enroll in 1 season.

We used central randomization of caregiver-child dyads with a 1:1 allocation, stratified by practice site, child’s age group (6 to 23 months; 2 to 8 years), and caregiver’s preferred language for text messaging. The randomization lists, based on permuted blocks with varying block sizes, used the module “ralloc,” which supports allocation concealment and achieves balance of treatment assignment within strata throughout patient recruitment (StataCorp, v15 and 16, College Station, TX). The statisticians, clinicians and practice staff members remained blinded to arm assignment until analysis was complete.

Caregivers in the intervention arm received personalized, scheduled text messages including (1) reminders about an upcoming second dose and (2) educational information about the importance of a second dose. Messages were based on a successful previous study,24  and modified based on feedback from pediatric and health literacy experts on the study team. The messages and study materials were professionally translated into Spanish. We pretested the English and Spanish messages with 30 caregivers from primary care practices at CHOP and Columbia. Messages were automatically sent centrally using a text messaging service on days 14 and 21 postvaccination to alert families about the next needed dose, day 25 just before the end of the minimal interval between doses, day 28 when the dose was due, and day 42, 2 weeks after the due date. The day 21 message was interactive, allowing caregivers to request further information about vaccine side effects, the necessity of the second dose, and/or the importance of timely second dose receipt (Supplemental Table 4). Messages were personalized with the child’s first name, gender, and practice name and included the due date for the child’s second influenza vaccine dose. Depending on how the practice primarily offered the second dose of influenza vaccine, practice specific walk-in hours or a phone number to call for an appointment was included.

In season 2 (2018 to 2019) only, caregivers randomized to usual care were sent a message upon enrollment with a link to the AAP’s Ages and Stages Web site as a 1-time attention control.26  Usual care had variable reminder systems, including no second dose reminders or letter, phone, e-mail or patient portal messages, or a written card provided at the first dose.

Outcomes

The primary outcome was the receipt of the second influenza vaccine dose by the end of the influenza season (April 30th), referred hereafter as “season end.” Secondary outcomes were timeliness of the second dose as assessed by receipt of dose by 42 days (represents a 2-week grace period after the 28 day minimal interval between first and second doses19 ), and time elapsed from first to the second dose. Practice staff blind to study arm assignment abstracted vaccination data from the medical record within each practice and from applicable vaccine registries.

Participant Characteristics

Practice-level characteristics (typical scheduling approach [appointment scheduling or walk-in for second dose of influenza vaccine] and practice urbanicity [inner-city urban or noninner city urban, suburban, or rural]) were self-reported by practices before the trial. Key caregiver and child characteristics were collected at enrollment for both seasons: child’s age group, gender, insurance status, and caregiver’s preferred language for texting (English or Spanish). Public insurance and uninsured groups were combined for analyses since both were eligible for free vaccine in the Vaccine for Children program and because of the small size of the uninsured group (n = 15). Additional demographic characteristics were collected for those who completed the demographic enrollment survey. These included child ethnicity, child race, caregiver reported child health (excellent, very good, good, fair, and poor), caregiver’s relationship to the child (mother, father, grandparent, and other), and caregiver age. Because of nonresponse on the demographic enrollment survey in the 2018 to 2019 season, we reweighted each response by the inverse of the probability that the caregiver of a patient of a given age, gender, and language preference answered that survey question, so that survey responses reflect the entire sample characteristics.27 

Analyses

Assuming 50 sites with 40 caregivers (20 per arm) per site, with a 2 sided test (α = .025 per tail), power was 0.8 to demonstrate an increase in vaccination of ≥6% points; power would increase if across-site variability was reduced. To assess the primary outcome of vaccine receipt (yes or no) by season end, we implemented Mantel Haenszel methods, stratifying by practice site and language to reflect randomization strata. For clarity, we reported effects on the absolute rather than relative scale. For that reason, we programmed in Stata v15 and validated against published data the algorithm of Bohning and Sarol for estimating stratified risk differences as of the end of influenza season.28 

For the secondary outcome of time from first to second vaccine dose, we implemented the Mantel Haenszel stratified methods for the outcome of vaccination by day 42 after the first dose and conducted a conventional survival analysis by means of the survival library in Stata v15 and 16. Kaplan-Meier analyses displayed the time from receipt of the first to second dose, censored at season end. We compared time-to-event using the Wilcoxon version of the log-rank test with start as the first dose date. As a sensitivity analysis, we repeated with the start date as the randomization day. We deliberately avoided estimation of hazard ratios owing to well-described limitations of the metric, and because of its lack of meaning in our study.29  Therefore, we estimated differences between study arms in cumulative incidences (percentage of patients who received the second dose) as of day 42 and confidence intervals of those differences, based on these survival curves using the method of pseudo-observations of Andersen and colleagues in the program “stpsurv” for Stata.30,31  As a sensitivity analysis, we replicated the day 42 analysis as of day 60.

Prespecified subgroup analyses (by child insurance, caregiver’s preferred language for texting, practice approach to providing second-doses, and practice urbanicity) proceeded along similar lines, with contrasts based on time-to-event curves, and differences in cumulative incidences (and confidence intervals) using pseudo-observations.31  Confirmatory confidence bounds were estimated using nonparametric bootstrap methods with 999 samples, with replacement, to calculate 95% confidence bounds using the percentile method.

A formal statistical analysis plan was drafted and agreed upon before the start of analysis. This study was approved by the Institutional Review Boards at Columbia University, CHOP, the AAP, and the University of South Carolina.

A total of 2086 eligible caregivers (2017 to 2018 season [n = 257]; 2018 to 2019 season [n = 1829]) were enrolled (Fig 1). Intervention and usual care participants had similar demographic composition (Table 1). Below we report results for the combined and individual seasons in intention to treat analyses.

FIGURE 1

Consort diagrams 2017 to 2018 and 2018 to 2019 seasons.

FIGURE 1

Consort diagrams 2017 to 2018 and 2018 to 2019 seasons.

Close modal
TABLE 1

Caregiver and Child Demographic Characteristics and Survey Responses

Unweighted %a (No.)Weightedb %
Influenza season   
 2017–2018 12.3 (257) — 
 2018–2019 87.7 (1829) — 
Child ethnicity   
 Hispanic or Latino 17.7 (255) 19.2 
 Not Hispanic or Latino 82.3 (1184) 80.8 
Child racec   
 White 64.7 (906) 63.8 
 Black 19.6 (274) 19.4 
 Asian 8.8 (123) 8.5 
 Other 6.9 (97) 8.4 
Child age   
 6–23 months 92.6 (1931) — 
 2–8 years old 7.4 (155) — 
Child gender   
 Female 48.6 (1013) — 
 Male 51.4 (1073) — 
Child health   
 Excellent 68.6 (984) 68.3 
 Very good, good, fair, or poor 31.4 (450) 31.7 
Child insurance type   
 Commercial insuranced 54.5 (1137) 54.5 
 Public insurance or uninsurede 45.5 (948) 45.5 
Caregiver relation to child   
 Mother 88.8 (1277) 88.7 
 Father, grandparent, or other 11.2 (161) 11.4 
Caregiver age   
 <30 years 34.6 (484) 34.5 
 30–34 years 35.9 (502) 35.4 
 35–39 years 22.0 (308) 22.0 
 ≥40 years 7.6 (106) 8.0 
Caregiver education   
 High school or less 22.3 (320) 23.3 
 Vocational school or some college 17.9 (257) 18.1 
 Associates or bachelors 36.3 (522) 35.6 
 Masters or doctorate 23.5 (338) 23.1 
Preferred language for text messages   
 English 92.5 (1930) — 
 Spanish 7.5 (156) — 
Unweighted %a (No.)Weightedb %
Influenza season   
 2017–2018 12.3 (257) — 
 2018–2019 87.7 (1829) — 
Child ethnicity   
 Hispanic or Latino 17.7 (255) 19.2 
 Not Hispanic or Latino 82.3 (1184) 80.8 
Child racec   
 White 64.7 (906) 63.8 
 Black 19.6 (274) 19.4 
 Asian 8.8 (123) 8.5 
 Other 6.9 (97) 8.4 
Child age   
 6–23 months 92.6 (1931) — 
 2–8 years old 7.4 (155) — 
Child gender   
 Female 48.6 (1013) — 
 Male 51.4 (1073) — 
Child health   
 Excellent 68.6 (984) 68.3 
 Very good, good, fair, or poor 31.4 (450) 31.7 
Child insurance type   
 Commercial insuranced 54.5 (1137) 54.5 
 Public insurance or uninsurede 45.5 (948) 45.5 
Caregiver relation to child   
 Mother 88.8 (1277) 88.7 
 Father, grandparent, or other 11.2 (161) 11.4 
Caregiver age   
 <30 years 34.6 (484) 34.5 
 30–34 years 35.9 (502) 35.4 
 35–39 years 22.0 (308) 22.0 
 ≥40 years 7.6 (106) 8.0 
Caregiver education   
 High school or less 22.3 (320) 23.3 
 Vocational school or some college 17.9 (257) 18.1 
 Associates or bachelors 36.3 (522) 35.6 
 Masters or doctorate 23.5 (338) 23.1 
Preferred language for text messages   
 English 92.5 (1930) — 
 Spanish 7.5 (156) — 

The caregivers that completed some sections of the demographic survey were 1439 of 2086 (69% of all eligible enrolled caregivers; 100% and 64.6% from 2017 to 2018 and 2018 to 2019 season, respectively). —, not applicable.

a

All percentages were rounded up, therefore, the total percentages may not equal 100%.

b

Weighted percentages account for the probability that the patient of a given age, gender, and text language preference answered the survey question. The weighted percentages are provided for items that were not answered by all survey participants in 2018 to 2019 season.

c

These questions and responses are in accordance to the US Census. “American Indian or Alaska Native” and “Native Hawaiian or Other Pacific Islander” were collapsed into the “Other” category because of small sample size.

d

Twenty six participants had Tricare insurance and were categorized into Commercial.

e

Fifteen participants had no insurance and were categorized into Public Insurance or Uninsured.

Over the 2 seasons, text message reminders increased second dose of influenza vaccine receipt by season end (April 30th). Overall, 83.8% of children in the intervention arm received the second dose by season end versus 80.9% in usual care (effect size: adjusted risk difference (ARD) 3.8%; 95% confidence interval [CI] [0.1 to 7.5]) (Table 2). The effect size by season end in the 2018 to 2019 season was similar to that observed in combined analyses across seasons (83.2% versus 80.1% (ARD) 3.8%; 95% CI [−0.4 to 8.1]). The effect size for the 2017 to 2018 season (n = 257), especially after adjustment was smaller (88.3% versus 86.8% (ARD) 0.3%; 95% CI [−4.9 to 5.6]) than observed in the 2018 to 2019 (n = 1829), and combined seasons.

TABLE 2

Effect Size of the Text Message Intervention on Receipt of Second Dose Influenza Vaccine by April 30 (Season End) and by Day 42 (2 Weeks After Second Dose Due) Post Vaccination, By Overall Population for the Second Dose of Influenza Vaccine (Appointment Versus Walk-in) and Child Insurance Type (Commercial Versus Public and Uninsured)

(A) End PointArm (N)% Received second Dose (N)Adjusted (Stratified) Difference % (95% CI)
End of season Intervention (1044) 83.8 (875) 3.8 (0.1–7.5) 
Usual care (1042) 80.9 (843) 
Day 42 post first dose Intervention (1044) 62.4 (651) 8.3 (3.6–13.0) 
Usual care (1042) 55.7 (580) 
(A) End PointArm (N)% Received second Dose (N)Adjusted (Stratified) Difference % (95% CI)
End of season Intervention (1044) 83.8 (875) 3.8 (0.1–7.5) 
Usual care (1042) 80.9 (843) 
Day 42 post first dose Intervention (1044) 62.4 (651) 8.3 (3.6–13.0) 
Usual care (1042) 55.7 (580) 

All estimates stratified by practice site and by language in keeping with the stratification of randomization. Stratified differences estimated by the method of Bohning. Stratified differences will not equal differences of unstratified percentages.

The intervention significantly improved receipt of second dose by day 42, with 62.4% of children in the intervention arm receiving the dose vs. 55.7% in usual care (ARD 8.3%; 95% CI [3.6 to 13.0]) (Table 2). Effect sizes of similar or greater magnitude were seen for each of the 2 individual seasons (2018 to 2019: 60.7% versus 54.7% ARD 7.3%; 95% CI [2.3 to 12.3]; 2017 to 2018: 74.2% versus 62.8% ARD 17.7%; 95% CI [4.7 to 30.7]). The number of participants needed to text to have 1 extra child be vaccinated by day 42 was 12 (95% CI [8 to 28]). Kaplan-Meier analyses (Fig 2) also demonstrated that the intervention arm received more timely vaccination through the season. Individual season results were consistent (Supplemental Fig 3). In the sensitivity analysis, by day 60, 69.8% of children in the intervention arm received the second dose versus 63.1% in usual care (ARD 7.4%; 95% CI [2.9 to 11.9]).

FIGURE 2

Time to receipt of the second influenza vaccine dose among children who received an initial dose in 2017 to 2018 and 2018 to 2019 seasons combined: intervention versus usual care arm. N = 2086 children across 50 primary care pediatric practices.

FIGURE 2

Time to receipt of the second influenza vaccine dose among children who received an initial dose in 2017 to 2018 and 2018 to 2019 seasons combined: intervention versus usual care arm. N = 2086 children across 50 primary care pediatric practices.

Close modal

We also compared intervention effectiveness by practice typical scheduling approach for the second dose of influenza vaccine (appointments [n = 40] versus walk-in hours [n = 10]) and practice urbanicity (inner-city urban [n = 11], noninner city urban [n = 9], suburban [n = 18], and rural [n = 12]). The intervention produced a bigger effect for practices which primarily had walk-in influenza vaccine hours versus practices which primarily scheduled second dose appointments (Fig 3, Table 3). In contrast, the intervention produced a similar positive effect for all 4 urbanicity groups both by season end and day 42.

FIGURE 3

Time to receipt of the second influenza vaccine dose among children who received an initial dose. Intervention versus usual care arm, 2017 to 2018 and 2018 to 2019 seasons combined. N = 2086 children across 50 primary care pediatric practices: (A) by the typical scheduling approach for the second dose of influenza vaccine and (B) by child insurance type. (A) Kaplan-Meier estimates of time from receipt of the first influenza vaccine dose to receipt of the second dose, stratified by the typical scheduling approach of the pediatric clinic for the second doses. The typical scheduling approach (appointment versus walk-in) is a practice level variable (mutually exclusive for this analysis) that reflects practice’s most common or typical approach for scheduling the second influenza dose. N appointment clinics = 40; N walk-ins = 10. The x-axis represents the time in days since receipt of the first influenza vaccine dose. The y-axis represents the cumulative proportion of children vaccinated. Number at risk represents the number of eligible enrolled children that had not yet received the second influenza vaccine dose and are not censored. Follow-up of each child was censored at the end of influenza vaccine season, April 30th (day 250 on the x-axis) if the child had not yet received the second dose. Stratified Wilcoxon tests produced consistent results when performed with the start date as (1) the day of randomization and (2) the day of the first influenza vaccine dose. We report the p-value for Wilcoxon text with the start date as the day of the first influenza adding does for the purposes of clinical relevance. The smaller p-value on the right graph (Children From Practices With Appointments) results in part by the much larger sample size than on the left graph (Children From Practices With Walk In Visits). (B) Kaplan-Meier estimates of time from receipt of the first influenza vaccine dose to receipt of the second dose by child insurance type. The x-axis represents time in days since receipt of the first influenza vaccine dose until receipt of second influenza vaccine dose. The y-axis represents the cumulative proportion of children vaccinated. Number at risk represents the number of eligible enrolled children that had not yet received the second influenza vaccine dose and are not censored. Follow-up of each child was censored at the end of influenza vaccine season, Apritl 30th (day 250 on the x-axis) if the child had not yet received the second dose. Stratified Wilcoxon tests produced consistent results when performed with the start date as (1) the day of randomization and (2) the day of first influenza vaccine dose. We report the p-value for the Wilcoxon test with the start date as the day of the first influenza vaccine does for the purposes of clinical relevance.

FIGURE 3

Time to receipt of the second influenza vaccine dose among children who received an initial dose. Intervention versus usual care arm, 2017 to 2018 and 2018 to 2019 seasons combined. N = 2086 children across 50 primary care pediatric practices: (A) by the typical scheduling approach for the second dose of influenza vaccine and (B) by child insurance type. (A) Kaplan-Meier estimates of time from receipt of the first influenza vaccine dose to receipt of the second dose, stratified by the typical scheduling approach of the pediatric clinic for the second doses. The typical scheduling approach (appointment versus walk-in) is a practice level variable (mutually exclusive for this analysis) that reflects practice’s most common or typical approach for scheduling the second influenza dose. N appointment clinics = 40; N walk-ins = 10. The x-axis represents the time in days since receipt of the first influenza vaccine dose. The y-axis represents the cumulative proportion of children vaccinated. Number at risk represents the number of eligible enrolled children that had not yet received the second influenza vaccine dose and are not censored. Follow-up of each child was censored at the end of influenza vaccine season, April 30th (day 250 on the x-axis) if the child had not yet received the second dose. Stratified Wilcoxon tests produced consistent results when performed with the start date as (1) the day of randomization and (2) the day of the first influenza vaccine dose. We report the p-value for Wilcoxon text with the start date as the day of the first influenza adding does for the purposes of clinical relevance. The smaller p-value on the right graph (Children From Practices With Appointments) results in part by the much larger sample size than on the left graph (Children From Practices With Walk In Visits). (B) Kaplan-Meier estimates of time from receipt of the first influenza vaccine dose to receipt of the second dose by child insurance type. The x-axis represents time in days since receipt of the first influenza vaccine dose until receipt of second influenza vaccine dose. The y-axis represents the cumulative proportion of children vaccinated. Number at risk represents the number of eligible enrolled children that had not yet received the second influenza vaccine dose and are not censored. Follow-up of each child was censored at the end of influenza vaccine season, Apritl 30th (day 250 on the x-axis) if the child had not yet received the second dose. Stratified Wilcoxon tests produced consistent results when performed with the start date as (1) the day of randomization and (2) the day of first influenza vaccine dose. We report the p-value for the Wilcoxon test with the start date as the day of the first influenza vaccine does for the purposes of clinical relevance.

Close modal
TABLE 3

Effect Size of the Text Message Intervention on Receipt of Second Dose Influenza Vaccine by April 30 (Season End) and by Day 42 (2 Weeks After Second Dose Due) Post Vaccination, By Practices’ Typical Scheduling Approach for the Second Dose of Influenza Vaccine (Appointment Versus Walk-in) and Child Insurance Type (Commercial Versus Public and Uninsured)

(B) End PointPractices’ Scheduling Approach and Child Insurance % (N)ArmReceived Second Dose, %Difference % (95% Bootstrap CI)
End of season Appointment 75.5 (1574) Intervention 84.4 2.1 (−0.9 to 4.8) 
Usual care 82.3 
Walk-in 24.5 (512) Intervention 82.0 5.3 (0.5 to 9.4) 
Usual care 76.7 
Day 42 post first Dose Appointment 75.5 (1574) Intervention 64.5 5.1 (−0.2 to 10.0) 
Usual care 59.4 
Walk-in 24.5 (512) Intervention 55.7 11.3 (4.4 to 19.8) 
Usual care 44.4 
End of Season Commercial 54.5 (1137) Intervention 93.8 6.1 (3.2 to 9.0) 
Usual care 87.7 
Publicor uninsured 45.5 (948) Intervention 72.6 0.4 (−4.9 to 5.6) 
Usual care 72.1 
Day 42 post first dose Commercial 54.5 (1137) Intervention 75.0 8.8 (3.1 to 13.6) 
Usual care 66.2 
Public or uninsured 45.5 (948) Intervention 48.2 6.1 (−1.2 to 13.5) 
Usual care 42.1 
(B) End PointPractices’ Scheduling Approach and Child Insurance % (N)ArmReceived Second Dose, %Difference % (95% Bootstrap CI)
End of season Appointment 75.5 (1574) Intervention 84.4 2.1 (−0.9 to 4.8) 
Usual care 82.3 
Walk-in 24.5 (512) Intervention 82.0 5.3 (0.5 to 9.4) 
Usual care 76.7 
Day 42 post first Dose Appointment 75.5 (1574) Intervention 64.5 5.1 (−0.2 to 10.0) 
Usual care 59.4 
Walk-in 24.5 (512) Intervention 55.7 11.3 (4.4 to 19.8) 
Usual care 44.4 
End of Season Commercial 54.5 (1137) Intervention 93.8 6.1 (3.2 to 9.0) 
Usual care 87.7 
Publicor uninsured 45.5 (948) Intervention 72.6 0.4 (−4.9 to 5.6) 
Usual care 72.1 
Day 42 post first dose Commercial 54.5 (1137) Intervention 75.0 8.8 (3.1 to 13.6) 
Usual care 66.2 
Public or uninsured 45.5 (948) Intervention 48.2 6.1 (−1.2 to 13.5) 
Usual care 42.1 

Estimates for end of season, for which the follow up times differ across patients are based on a logistic regression, with confidence intervals estimated using 999 bootstrap samples (percentile method) in which resampling was done by practice site to reflect potential for clustering of patients within practice. Estimates of cumulative incidence at day 42 are based on survival curves using the method pseudo-values (Andersen et al), and confidence intervals are estimated using the same bootstrap sampling approach.

The intervention led to more timely vaccination among commercially insured than publicly insured children (Fig 3, Table 3). In contrast, preferred text language was not an effect modifier; the intervention produced a similar positive effect for English (n = 1930) and for Spanish-speakers (n = 156), both by season end and day 42. Only 17 practice sites had Spanish speakers, with the majority of the 156 children receiving care in 2 of the practices.

In this multistate trial, text message reminders were effective in increasing delivery of the second dose of influenza vaccine. This potentially scalable and accessible intervention could be implemented as most adults in the United States own a cell phone and use texting often.23,25  Perhaps most importantly, children of caregivers randomized to the intervention arm received second doses sooner. Timely receipt of the second dose is critical so that children are fully vaccinated before influenza begins circulating.

The effect sizes we observed are likely to be clinically meaningful. According to a recent modeling study, a mere 5% increase in nationwide influenza vaccination coverage could have prevented 228 000 illnesses and 4900 hospitalizations during seasons with typical influenza severity across all age groups.32  For high-severity seasons such as the 2017 to 2018 influenza season, the impact could be even greater. In that season, vaccination was estimated to have prevented over 2.1 million illnesses, 1.4 million medical visits, 14 790 hospitalizations and 74 deaths among young children (6 months to 4 years).33  The consistent impact of text message reminders on timeliness of second dose vaccination observed in our study might be especially important, because timely influenza vaccine coverage remains a problem in the United States.20  The Centers for Disease Control and Prevention recommends people be fully vaccinated before influenza starts circulating,19  which can happen as early as October.34  For young children who need 2 doses to be adequately protected, receiving both doses before the start of the influenza season can be logistically difficult.

A relatively newer concept of precision public health (and related precision primary care) suggest that different strategies may be needed to best serve distinct patient and practice populations.35  Consistent with the tenets of precision care, we identified practice and family-level characteristics that modified intervention effects. Even more investigation may therefore be needed to understand how to best implement this type of intervention for maximal impact across a variety of populations. This result advances knowledge of vaccine reminders via text messaging, since prior studies focused primarily on lower income or single-area populations.16,20,3639  An encouraging finding was that our bilingual intervention worked similarly for both English and Spanish-speaking caregivers, although the number of Spanish-speaking caregivers was small and limited to certain practices.

We saw a larger intervention effect for practices that primarily relied on walk-in influenza vaccinations. Although walk-in vaccination offers flexibility and convenience, it puts recall responsibility on caregivers’ shoulders. For these caregivers, the text messages may function as an especially important “cue-to-action” to bring their child back for the second dose during the provided walk-in hours. Often arranged at the time of their last visit, scheduled appointments conversely tradeoff caregivers needing to remember to return with decreased flexibility.

Our study has several limitations. First, our participants agreed to participate in a text messaging study and may not be representative of the larger population. As this study was conducted pragmatically in a wide practice range, we do not know how many caregivers opted not to participate. However, this enrollment strategy reflects how texting might be implemented in practices where families would choose whether to receive reminders. Second, our usual care arm had much higher receipt of second doses than observed in previous studies, which could have blunted season end effects; this may be partly caused by many enrollees being 6 to 23 months-old who have higher 2 dose completion rates.1418  Practices with low baseline rates of return might realize even greater effects. Third, because of a technical error with the text message distribution, 253 of 913 usual care participants in season 2 received at least 1 intervention text message (105 received 1 extra, 148 received 2 or more). As all analyses were intention to treat, this could have blunted our effect size. In addition, we did find that usual care participants who received more messages were slightly more likely to be vaccinated at day 42 than those who did not, highlighting the possible benefits of text messaging. Fourth, because of survey nonresponse, we do not have the full set of demographic variables for all participants; therefore, we limited analyses to variables we decided a priori to capture at enrollment. Additionally, although this study involved a convenience sample of heterogeneous participants in many states, it was not designed to be nationally representative. Finally, we centrally implemented the reminders on behalf of practices; future studies may need to assess text message implementation by practices themselves or their chosen vendors. Despite these limitations, a key strength of this study is that it included over 2000 families, practices in multiple regions, and both English and Spanish speakers.

In this large-scale trial in practices across the country, we demonstrated the effectiveness of using personalized text message reminders to increase overall receipt and the timeliness of the second dose of the influenza vaccine. A next step would be to assess how text message reminders for the second dose of influenza vaccine could be broadly implemented in practice settings serving children.

We thank all Flu2Text practices (listed below), pediatricians, nurse practitioners, other physicians, staff, caregivers, and families who participated in our study, and the University of South Carolina Institute for Public Service and Policy Research, Survey Research Laboratory.

Flu2Text practices participating in this study, named here with their permission, included: ABC Pediatrics; Advanced Pediatrics, PC; All Pediatrics, PC; Altru Health System; Anaconda Pediatrics; Anchorage Pediatric Group; Ashley Clinic; Atlantic Coast Pediatrics; Bethesda Pediatrics; Bozeman Health Pediatrics; Building Blocks Pediatrics; Burlington Pediatrics; Cambridge Pediatrics; Child Health Partners, PC; Childhood Health Associates of Salem; Clinch Valley Physicians Associates – Pediatric Department; Dowd Medical Associates; Elmwood Pediatric Group, LLP; Fishing Bay Family Practice; Goshen – Columbus Pediatrics & Adolescent Care; Hirsch Pediatrics, LLC; Holyoke Pediatric Associates; Ivancic Pediatric Clinic, PA; Mesa Pediatrics; OHSU Doernbecher Pediatrics – Westside Clinic; One Hanson Place Pediatrics, PC; Pediatric & Adolescent Healthy Lifestyle Center; Pediatric Associates of Davidson County, PA; Pediatric Associates of Medford; Pediatrics by the Sea; Pennridge Pediatric Associates; Prattville Pediatrics; Priority Care Pediatrics; Purohit Pediatric Clinic; Quality Kids Kare, PC; Scarano & Taylor Pediatrics; Southeastern Pediatric Associates; Southwest Montana Clinic; Springfield Pediatrics; Sunset Park Family Health Center at NYU Langone; Swafford Pediatrics; The Child & Teen Wellness Center; UNM Pediatrics – 3ACC Faculty Clinic; Zaheer Pediatric Associates, SC; and Zimble & Reinstein Pediatrics.

Dr Stockwell conceptualized and designed the study, designed the data collection instruments, and drafted the initial manuscript; Dr Fiks conceptualized and designed the study and designed the data collection instruments; Dr Shone contributed to design and conduction of the study and design of the data collection instruments; Ms Nekrasova aided in the conduction of the study, contributed to data collection, and contributed to the data; Ms Wynn aided in the conduction of the study and contributed to data collection; Ms Torres, Ms Griffith and Ms Harris aided in the conduction of the study and contributed to data collection; Dr Shults contributed to the design of the study and analytic plan; Dr Unger and Ms Ware contributed to the design of the study and the design of the data collection instruments; Drs Berrigan and Montague and Ms Kolff contributed to the design the study and the data collection instruments and aided in study conduction and data collection; Dr Localio contributed to design of the study, oversaw the statistical analysis plan, and analyzed the data and all authors reviewed and revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.

This trial is registered at ClinicalTrials.gov Identifier: NCT03287830.

Deidentified individual participant data (including data dictionaries) will be made available. The data will be made available after publication of the primary studies to researchers who provide a methodologically sound proposal for use in achieving the goals of the approved proposal and data use agreements. Proposals should be submitted to mss2112@cumc.columbia.edu and fiks@chop.edu.

FUNDING: This work was supported by the National Institutes of Health, National Institute of Child Health and Health Development grant number R01HD086045 (PI: Stockwell, Fiks). Additional infrastructure funding was provided by the American Academy of Pediatrics and the Health Resources and Services Administration of the United States Department of Health and Human Services under UA6MC15585 - National Research Network to Improve Children’s Health and U5DMC39344 - Pediatric Research Network Program. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by Health Resources and Services, United States Department of Health and Human Services, or the United States Government.

CONFLICT OF INTEREST DISCLOSURES: This research took place when Dr Montague was at the American Association of Pediatrics, but she has since moved to AADSM. The authors have indicated they have no conflicts of interest to disclose.

COMPANION PAPER: A companion to this article can be found at http://www.pediatrics.org/cgi/doi/10.1542/peds.2022-057363.

     
  • AAP

    American Academy of Pediatrics

  •  
  • ARD

    adjusted risk difference

1
Centers for Disease Control and Prevention (CDC)
.
Estimates of deaths associated with seasonal influenza --- United States, 1976-2007
.
MMWR Morb Mortal Wkly Rep
.
2010
;
59
(
33
):
1057
1062
2
Thompson
WW
,
Shay
DK
,
Weintraub
E
, et al
.
Influenza-associated hospitalizations in the United States
.
JAMA
.
2004
;
292
(
11
):
1333
1340
3
Molinari
NA
,
Ortega-Sanchez
IR
,
Messonnier
ML
, et al
.
The annual impact of seasonal influenza in the US: measuring disease burden and costs
.
Vaccine
.
2007
;
25
(
27
):
5086
5096
4
Centers for Disease Control and Prevention
.
Estimated influenza illnesses, medical visits, hospitalizations, and deaths in the United States — 2019–2020 influenza season
.
Available at: https://www.cdc.gov/flu/about/burden/2019-2020.html. Accessed on August 15, 2021
5
Neuzil
KM
,
Zhu
Y
,
Griffin
MR
, et al
.
Burden of interpandemic influenza in children younger than 5 years: a 25-year prospective study
.
J Infect Dis
.
2002
;
185
(
2
):
147
152
6
Neuzil
KM
,
Mellen
BG
,
Wright
PF
,
Mitchel
EF
Jr
,
Griffin
MR
.
The effect of influenza on hospitalizations, outpatient visits, and courses of antibiotics in children
.
N Engl J Med
.
2000
;
342
(
4
):
225
231
7
Jules
A
,
Grijalva
CG
,
Zhu
Y
, et al
.
Influenza-related hospitalization and ED visits in children less than 5 years: 2000-2011
.
Pediatrics
.
2015
;
135
(
1
):
e66
e74
8
Zhou
H
,
Thompson
WW
,
Viboud
CG
, et al
.
Hospitalizations associated with influenza and respiratory syncytial virus in the United States, 1993-2008
.
Clin Infect Dis
.
2012
;
54
(
10
):
1427
1436
9
Committee on Infectious Diseases
.
Recommendations for prevention and control of influenza in children, 2019-2020
.
Pediatrics
.
2019
;
144
(
4
):
e20192478
10
Tsang
TK
,
Lau
LLH
,
Cauchemez
S
,
Cowling
BJ
.
Household transmission of influenza virus
.
Trends Microbiol
.
2016
;
24
(
2
):
123
133
11
Putri
WCWS
,
Muscatello
DJ
,
Stockwell
MS
,
Newall
AT
.
Economic burden of seasonal influenza in the United States
.
Vaccine
.
2018
;
36
(
27
):
3960
3966
12
Centers for Disease Control and Prevention
.
Estimated influenza illnesses, medical visits, and hospitalizations averted by vaccination in the United States — 2019–2020 influenza season
.
Available at: https://www.cdc.gov/flu/about/burden-averted/2019-2020.htm. Accessed on November 22, 2021
13
Flannery
B
,
Reynolds
SB
,
Blanton
L
, et al
.
Influenza vaccine effectiveness against pediatric deaths: 2010-2014
.
Pediatrics
.
2017
;
139
(
5
):
e20164244
14
Chung
JR
,
Flannery
B
,
Gaglani
M
, et al
.
Patterns of influenza vaccination and vaccine effectiveness among young US children who receive outpatient care for acute respiratory tract illness
.
JAMA Pediatr
.
2020
;
174
(
7
):
705
713
15
Zhai
Y
,
Santibanez
TA
,
Kahn
KE
,
Srivastav
A
.
Parental-reported full influenza vaccination coverage of children in the U.S
.
Am J Prev Med
.
2017
;
52
(
4
):
e103
e113
16
Hofstetter
AM
,
Natarajan
K
,
Martinez
RA
,
Rabinowitz
D
,
Vawdrey
DK
,
Stockwell
MS
.
Influenza vaccination coverage and timeliness among children requiring two doses, 2004-2009
.
Prev Med
.
2013
;
56
(
3–4
):
165
170
17
Pabst
LJ
,
Chaves
SS
,
Weinbaum
C
.
Trends in compliance with two-dose influenza vaccine recommendations among children aged 6 months through 8 years
.
Vaccine
.
2013
;
31
(
31
):
3116
3120
18
Santibanez
TA
,
Grohskopf
LA
,
Zhai
Y
,
Kahn
KE
.
Complete influenza vaccination trends for children six to twenty-three months
.
Pediatrics
.
2016
;
137
(
3
):
e20153280
19
Grohskopf
LA
,
Alyanak
E
,
Broder
KR
, et al
.
Prevention and control of seasonal influenza with vaccines: recommendations of the advisory committee on immunization practices - United States, 2020-21 influenza season
.
MMWR Recomm Rep
.
2020
;
69
(
8
):
1
24
20
Hofstetter
AM
,
Natarajan
K
,
Rabinowitz
D
, et al
.
Timeliness of pediatric influenza vaccination compared with seasonal influenza activity in an urban community, 2004-2008
.
Am J Public Health
.
2013
;
103
(
7
):
e50
e58
21
Hofstetter
AM
,
Barrett
A
,
Stockwell
MS
.
Factors impacting influenza vaccination of urban low-income Latino children under nine years requiring two doses in the 2010-2011 season
.
J Community Health
.
2015
;
40
(
2
):
227
234
22
Nekrasova
E
,
Stockwell
MS
,
Localio
R
, et al
.
Vaccine hesitancy and influenza beliefs among parents of children requiring a second dose of influenza vaccine in a season: an American Academy of Pediatrics (AAP) Pediatric Research in Office Settings (PROS) study
.
Hum Vaccin Immunother
.
2020
;
16
(
5
):
1070
1077
23
Pew Research Center
.
Mobile fact sheet
.
Available at: https://www.pewresearch.org/internet/fact-sheet/mobile/ Accessed August 15, 2021
24
Stockwell
MS
,
Kharbanda
EO
,
Martinez
RA
,
Vargas
CY
,
Vawdrey
DK
,
Camargo
S
.
Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial
.
JAMA
.
2012
;
307
(
16
):
1702
1708
25
O’Leary
ST
,
Lee
M
,
Lockhart
S
, et al
.
Effectiveness and cost of bidirectional text messaging for adolescent vaccines and well care
.
Pediatrics
.
2015
;
136
(
5
):
e1220
e1227
26
American Academy of Pediatrics
.
Ages and stages
.
27
Michael Brick
J
,
Jill
M
.
Chapter 8 - nonresponse and weighting
. In:
Rao
CR
, eds.
Handbook of Statistics
,
Volume 29
,
Part A
.
Amsterdam, Netherlands
:
Elsevier
;
2009
:
163
185
28
Böhning
D
,
Sarol
J
Jr
.
Estimating risk difference in multicenter studies under baseline-risk heterogeneity
.
Biometrics
.
2000
;
56
(
1
):
304
308
29
Hernán
MA
.
The hazards of hazard ratios
.
Epidemiology
.
2010
;
21
(
1
):
13
15
30
Overgaard
M
,
Andersen
PK
,
Parner
ET
.
Regression analysis of censored data using pseudo-observations: an update
.
Stata J
.
2015
;
15
:
809
31
Andersen
PK
,
Perme
MP
.
Pseudo-observations in survival analysis
.
Stat Methods Med Res
.
2010
;
19
(
1
):
71
99
32
Hughes
MM
,
Reed
C
,
Flannery
B
, et al
.
Projected population benefit of increased effectiveness and coverage of influenza vaccination on influenza burden in the United States
.
Clin Infect Dis
.
2020
;
70
(
12
):
2496
2502
33
Rolfes
MA
,
Flannery
B
,
Chung
JR
, et al;
US Influenza Vaccine Effectiveness (Flu VE) Network, the Influenza Hospitalization Surveillance Network, and the Assessment Branch, Immunization Services Division, Centers for Disease Control and Prevention
.
Effects of influenza vaccination in the United States during the 2017-2018 influenza season
.
Clin Infect Dis
.
2019
;
69
(
11
):
1845
1853
34
Chow
EJ
,
Davis
CT
,
Abd Elal
AI
, et al
.
Update: influenza activity - United States and worldwide, May 20-October 13, 2018
.
MMWR Morb Mortal Wkly Rep
.
2018
;
67
(
42
):
1178
1185
35
Khoury
MJ
,
Iademarco
MF
,
Riley
WT
.
Precision public health for the era of precision medicine
.
Am J Prev Med
.
2016
;
50
(
3
):
398
401
36
Kharbanda
EO
,
Stockwell
MS
,
Fox
HW
,
Rickert
VI
.
Text4Health: a qualitative evaluation of parental readiness for text message immunization reminders
.
Am J Public Health
.
2009
;
99
(
12
):
2176
2178
37
Hofstetter
AM
,
Vargas
CY
,
Camargo
S
, et al
.
Impacting delayed pediatric influenza vaccination: a randomized controlled trial of text message reminders
.
Am J Prev Med
.
2015
;
48
(
4
):
392
401
38
Stockwell
MS
,
Hofstetter
AM
,
DuRivage
N
, et al
.
Text message reminders for second dose of influenza vaccine: a randomized controlled trial
.
Pediatrics
.
2015
;
135
(
1
):
e83
e91
39
Ahlers-Schmidt
CR
,
Chesser
AK
,
Nguyen
T
, et al
.
Feasibility of a randomized controlled trial to evaluate text reminders for immunization compliance in kids (TRICKs)
.
Vaccine
.
2012
;
30
(
36
):
5305
5309

Supplementary data