Video Abstract
Children with medical comorbidities are at greater risk for severe influenza and poorer clinical outcomes. Despite recommendations and funding, influenza vaccine coverage remains inadequate in these children.
We aimed to systematically review literature assessing interventions targeting influenza vaccine coverage in children with comorbidities and assess the impact on influenza vaccine coverage.
PubMed, Scopus, Embase, Cumulative Index to Nursing and Allied Health Literature, Cochrane Central Register of Controlled Trials, Allied and Complementary Medicine Database, and Web of Science databases were searched.
Interventions targeting influenza vaccine coverage in children with medical comorbidities.
Two reviewers independently screened articles, extracting studies’ methods, interventions, settings, populations, and results. Four reviewers independently assessed risk of bias.
From 961 screened articles, 35 met inclusion criteria. Published studies revealed that influenza vaccine coverage was significantly improved through vaccination reminders and education directed at either patients’ parents or providers, as well as by vaccination-related clinic process changes. Interventions improved influenza vaccine coverage by an average 60%, but no significant differences between intervention types were detected. Significant bias and study heterogeneity were also identified, limiting confidence in this effect estimate.
A high risk of bias and overall low quality of evidence limited our capacity to assess intervention types and methods.
Interventions were shown to consistently improve influenza vaccine coverage; however, no significant differences in coverage between different intervention types were observed. Future well-designed studies evaluating the effectiveness of different intervention are required to inform future optimal interventions.
Influenza remains a substantial cause of morbidity and mortality in children despite the widespread availability of seasonal vaccines.1 Children with medical comorbidities, including asthma, immunodeficiency, and epilepsy, are at increased risk of severe influenza infections and more adverse clinical outcomes.2 As a result, national immunization programs3–5 recommend and fund annual influenza vaccination for those with medical comorbidities.3,6 Despite these efforts, influenza vaccine coverage in children with comorbidities remains inadequate.7,8 The reasons for noncompliance with recommendations are complex and multifactorial,9 yet interventions targeting improvement in influenza coverage in children with medical comorbidities have been shown to be efficacious.10
A previous review of interventions targeting influenza vaccine coverage in children aged 6 months to 19 years with comorbidities, published in 2015, concluded that there was strong evidence for parental letter-based vaccination appointment reminders.11 Pooled estimates of the overall and relative impact of interventions were not assessed and the numerous biases of the included studies were not formally reported. The review was also limited by the search strategy restricted to “influenza AND vaccination OR immunization OR children AND asthma OR malignancy OR high risk AND reminder,” potentially missing nonreminder type interventions and other comorbidities.
To design and evaluate effective and sustainable interventions to improve influenza vaccine coverage in children with comorbidities requires an understanding of the overall and relative effectiveness of these interventions and any associated biases. To this end, we conducted a systematic review of published interventions targeting influenza vaccine coverage in children with medical comorbidities to evaluate the overall effectiveness and effectiveness of different intervention types.
Methods
The study was designed with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and checklist12 with registration through PROSPERO: The International Prospective Register of Systematic Reviews (CRD42019090623). We conducted a narrative review describing studies’ characteristics, results, and biases. Where possible, absolute results of influenza vaccine coverage were extracted to determine pooled effect estimates for each intervention type and study method.
Search Strategy and Data Sources
PubMed, Scopus, Embase, Cumulative Index to Nursing and Allied Health Literature, Cochrane Central Register of Controlled Trials, Allied and Complementary Medicine, and Web of Science databases were searched for articles published in English from January 1976 to March 2019. The search strategy was "Influenza vaccination or influenza immunization or influenza immunisation) and (reminder or interventions or best practices or strategies) and (children or pediatric or paediatric) and (high-risk or asthma or neurological or malignancy or cardiac or hematological or oncology or comorbidity or chronic diseases or medical conditions." This strategy was designed with free-text search terms encompassing influenza vaccination and comorbidities defined by the Australian,3 US,4 and UK5 national guidelines with appropriate terms for intervention types and methods. Hand searching of retrieved articles’ bibliographies was undertaken to identify any additionally relevant publications. Ongoing trials were searched through the National Institutes of Health’s clinicaltrials.gov Web site. Two authors (D.A.N. and R.B.) were responsible for screening of extracted articles and identification of potentially eligible studies.
Study Selection
Randomized control trials (RCTs), cohort, and quasi-experimental studies undertaken in community, primary care, and/or hospital settings were reviewed if they (1) included children (aged 6 months to 21 years old) who met medical comorbidity criteria as per national immunization guidelines3–5 and (2) evaluated at least 1 clinical, behavioral, or structural intervention designed to improve influenza vaccine coverage in children with 1 or more comorbidities. Exclusion criteria of studies included non–peer reviewed results, studies from which changes in influenza vaccination could not be extracted or separated from non-influenza vaccination, studies with influenza vaccination results of children with comorbidities which could not be separated from other populations, and other systematic reviews of eligible studies (Supplemental Information). Raw numbers were extracted, where possible. When required, the number of vaccinated patients were calculated from percentages included in study reports.
Data Extraction
A data form based on Cochrane’s “Data Collection Form: Intervention Review – RCTs and Non-RCTS”13 was used to ensure standardized extraction of articles. Data were extracted by a primary author (D.A.N.) and independently confirmed by supplementary authors (R.P., M.B., and K.A.A.). When discrepancies were identified, results were resolved by the senior author (C.C.B.). The extraction form included publishing journal, study methodology, intervention type, clinical setting, geographical location, participant comorbidity type(s), participant age group, study years, and vaccination status ascertainment method (ie, parental report, clinic vaccination records, and immunization registry records). Absolute vaccination numbers, rates, and relevant statistical testing results (ie, P value and risk ratios) were extracted from applicable studies. Studies were grouped by methodology: (1) RCT (clustered or individually randomized), (2) cohort studies (prospective and retrospective), or (3) a quasi-experimental study type including quality improvement (QI)14 or before and after studies.15 QI studies were classified as such if the study’s authors identified them as QI and were approved by a relevant ethics group or review board as a QI study. Before and after studies were defined by their observations of the study population’s influenza vaccine coverage taken before an intervention(s)’ introduction and after introduction without identifying as a QI study. Final observations of vaccine coverage in before and after studies were used as the postintervention vaccine coverage values for quantitative analysis. Before and after studies did not use negative control groups but may have randomly assigned individuals to different interventions.
Qualitative Data Analysis
Study characteristics and individual results as well as trends across studies were assessed. There were 5 intervention categories: (1) parental reminders (physical or electronic reminders directed to parents or guardians about their child(s)’ vaccination status and eligibility), (2) parental education (information for parents about influenza infection risks and vaccination benefits through counseling by a provider or written material), (3) provider reminders (physical or electronic reminders directed to clinicians managing children with medical comorbidities), (4) provider education (information for clinicians about influenza infection risks and vaccination benefits), and (5) clinic process changes. Clinic process changes were defined as interventions that impacted how patients received influenza vaccinations in a clinical environment, the flow of vaccination resources, vaccination status recording, and/or reporting within the study’s setting(s). Such interventions included introduction of electronic vaccination records, interclinic collaboration, and resource sharing. Studies in which researchers examined multiple interventions trialed in the same group(s) were identified as a multicomponent intervention, whereas those using the same intervention in single or multiple study groups were defined as single interventions.
Quantitative Data Analysis
Quantitative changes in influenza vaccination coverage either between allocated intervention and control groups (ie, RCTs) or across influenza seasons (ie, quasi-experimental and cohort studies) were extracted when applicable. Where applicable, absolute and relative changes in vaccine coverage were directly extracted from publication or back calculated when necessary. Subanalyses were performed to assess different interventions types, study designs, and single versus multicomponent interventions. Studies in which researchers evaluated the same intervention(s) in different populations or different interventions within same populations across different time points were treated as separate data sets. Changes in influenza vaccine coverage in an intervention group(s) compared with a control group or before and after introduction of an intervention were estimated by using risk ratios with 95% confidence intervals (CIs). Random effect models16 using the Mantel-Haenszel method (M-H)17 were used for pooling of vaccination coverage changes. Random effect models were used because of the high level of methodologic and clinical diversity between studies and the assumption of different effects across studies.18 Heterogeneity of quantitative results was evaluated by using the standardized τ2 and I2 statistics describing the proportion of dispersion across studies due to true heterogeneity.19
Risk of Bias Analysis
Individual studies’ risk of bias was assessed with Cochrane’s Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool.20 One primary reviewer (D.A.N.) examined risk of bias in all studies, and 3 secondary reviewers (R.P., K.A.A., and M.B.) independently reviewed an equal proportion of the studies. A senior reviewer (C.C.B.) acted to resolve any discordant assessments between the primary and the secondary reviewers. Studies were assigned to be low risk if all domains were identified as low risk, unclear risk if 1 or more domains were of unclear risk (with no high-risk scored domains), and high risk if at least 1 domain was identified as high risk.
Results
Study Search and Selection
Our initial search identified 961 primary articles, with 6 additional articles identified by searching references. No ongoing trials or intervention studies meeting our inclusion criteria were identified through the National Institutes of Health clinicaltrials.gov Web site. Once duplicates were removed, 611 unique articles remained. Title and abstract review excluded a further 442 and 108 articles, respectively. From the 61 articles remaining for full-text review, 35 articles met our inclusion criteria. Sufficient data were present in 26 of these articles for quantitative analysis. The reasons for exclusion after full article review are listed in Fig 1.
Of the 35 included articles, 33 examined seasonal influenza vaccination exclusively, 1 evaluated pandemic H1N1/09 influenza vaccination21 and 1 examined both seasonal and pandemic H1N1/09 vaccination.22 There were 5 RCTs23–27 and 4 cohort studies.10,22,28,29 The remainder, defined as quasi-experimental studies, included 20 before and after intervention studies21,28,30–47 and 6 QI studies.48–54 One article included both a quasi-experimental study and a cohort study.28
RCTs
All RCTs were singular type intervention of parental influenza vaccination reminders: researchers in 4 studies used postal letter reminders23,24,26,27 and those in 1 evaluated mobile phone text message reminders (Table 1).25 All studies were conducted within the United States between 1992 and 2017. Four were conducted at pediatric clinics, of which 3 only targeted asthmatic patients,23,26,27 and the fourth included patients with a range of comorbidity types.25 Dombkowski et al24 targeted children with different comorbidity types with reminders through a statewide immunization registry. Changes in vaccination coverage were recorded sufficiently for each RCT to be included within the quantitative analysis. All postal letter interventions revealed significant improvements in influenza vaccine coverage for those receiving reminders compared with their respective control groups. Statistically significant low to moderate absolute increases in influenza vaccine coverage of 6.5%, 18%, 23%, and 25% were present for their respective intervention groups of Dombkowski et al,24 Daley et al,23 Szilagyi et al,27 and Kemper et al26 compared with their control groups. Conversely, the intervention group in Hofstetter et al25 had a lower but not statistically significant absolute coverage decline of 7% compared with their control group.
Characteristics and Results of RCTs
| Authors and Publication Year | Setting, Country, and Study Years | Study Participants (Population) | Participant Ages | Study Methodology (Subtype) | Vaccination Status Ascertainment | Intervention Type | Absolute Effect Size (% of Vaccine Coverage) | Relative Effect Sizes (% of Vaccine Coverage) | Study Summary |
|---|---|---|---|---|---|---|---|---|---|
| Daley et al23 (2004) | Pediatric practices, United States (2002–2003) | Pediatric asthmatic patients: intervention group: n = 920; control group: n = 931 | Not specified | Individualized RCT | Immunization registry records and patient billing record | Parent reminder | Absolute influenza vaccine coverage increased by 17% | Intervention group: 42%; control group: 25% | Mailed letter influenza vaccination reminders to patients’ families significantly improved influenza vaccine uptake in asthmatic children |
| Dombkowski et al24 (2012) | Michigan immunization registry, United States (2008–2009) | Children with comorbidities (ACIP guidelinesa): n = 3618 | 2–5 y | Individualized RCT | Immunization registry records (ICD-9 codes) | Parent reminder | Absolute influenza vaccine coverage increased by 6.5% | Intervention group: 30.8%; control group: 24.3%; odds ratio of 1.39 (95% CI: 1.13–1.72) | Children with comorbidities were 1.39 times more likely to receive an influenza vaccine if they received a mailed vaccination letter reminder compared with those who did not |
| Hofstetter et al25 (2017) | Pediatric clinics, United States (2014–2015) | Pediatric patients with comorbidities (ACIP guidelinesa): education reminder: n = 154, plain reminder: n = 141 | 11–17 y | Individualized RCT | Electronic vaccination records | Parent reminder | Absolute difference in vaccine coverage of 7% between reminder groups | Educational text message group: 56.8%; plain text message group: 63.8% (P = .24) | Nonsignificantly lower influenza vaccination coverage was seen in children assigned an education-embedded vaccination reminder compared with a plain reminder |
| Kemper et al26 (1993) | Pediatric clinic, United States (1991) | Pediatric asthmatic patients: intervention group: n = 43; control group: n = 53 | Average of 4.5 y (no range given) | Individualized RCT | Clinic vaccination records | Parent reminder | Absolute vaccination coverage was 25% greater in the intervention group | Intervention group: 46%; control group: 21% | Pediatric asthmatic patients who received mailed letter influenza vaccination reminders were significantly more likely to receive the seasonal influenza vaccination |
| Szilagyi et al27 (1992) | Pediatric clinic, United States (1990–1991) | Pediatric asthmatic patients: intervention group: n = 63; control group: n = 53 | 1–18 y | Individualized RCT | Clinic vaccination records | Parent reminder | Absolute increase in influenza vaccine uptake of 23% | Intervention group: 30%; control group: 7% | Mailed influenza vaccination reminder letters to families considerably improved influenza vaccine uptake |
| Authors and Publication Year | Setting, Country, and Study Years | Study Participants (Population) | Participant Ages | Study Methodology (Subtype) | Vaccination Status Ascertainment | Intervention Type | Absolute Effect Size (% of Vaccine Coverage) | Relative Effect Sizes (% of Vaccine Coverage) | Study Summary |
|---|---|---|---|---|---|---|---|---|---|
| Daley et al23 (2004) | Pediatric practices, United States (2002–2003) | Pediatric asthmatic patients: intervention group: n = 920; control group: n = 931 | Not specified | Individualized RCT | Immunization registry records and patient billing record | Parent reminder | Absolute influenza vaccine coverage increased by 17% | Intervention group: 42%; control group: 25% | Mailed letter influenza vaccination reminders to patients’ families significantly improved influenza vaccine uptake in asthmatic children |
| Dombkowski et al24 (2012) | Michigan immunization registry, United States (2008–2009) | Children with comorbidities (ACIP guidelinesa): n = 3618 | 2–5 y | Individualized RCT | Immunization registry records (ICD-9 codes) | Parent reminder | Absolute influenza vaccine coverage increased by 6.5% | Intervention group: 30.8%; control group: 24.3%; odds ratio of 1.39 (95% CI: 1.13–1.72) | Children with comorbidities were 1.39 times more likely to receive an influenza vaccine if they received a mailed vaccination letter reminder compared with those who did not |
| Hofstetter et al25 (2017) | Pediatric clinics, United States (2014–2015) | Pediatric patients with comorbidities (ACIP guidelinesa): education reminder: n = 154, plain reminder: n = 141 | 11–17 y | Individualized RCT | Electronic vaccination records | Parent reminder | Absolute difference in vaccine coverage of 7% between reminder groups | Educational text message group: 56.8%; plain text message group: 63.8% (P = .24) | Nonsignificantly lower influenza vaccination coverage was seen in children assigned an education-embedded vaccination reminder compared with a plain reminder |
| Kemper et al26 (1993) | Pediatric clinic, United States (1991) | Pediatric asthmatic patients: intervention group: n = 43; control group: n = 53 | Average of 4.5 y (no range given) | Individualized RCT | Clinic vaccination records | Parent reminder | Absolute vaccination coverage was 25% greater in the intervention group | Intervention group: 46%; control group: 21% | Pediatric asthmatic patients who received mailed letter influenza vaccination reminders were significantly more likely to receive the seasonal influenza vaccination |
| Szilagyi et al27 (1992) | Pediatric clinic, United States (1990–1991) | Pediatric asthmatic patients: intervention group: n = 63; control group: n = 53 | 1–18 y | Individualized RCT | Clinic vaccination records | Parent reminder | Absolute increase in influenza vaccine uptake of 23% | Intervention group: 30%; control group: 7% | Mailed influenza vaccination reminder letters to families considerably improved influenza vaccine uptake |
ACIP, Advisory Committee on Immunization Practices; ICD-9, The International Classification of Diseases, Ninth Revision codes for disease diagnoses.
ACIP guidelines: guidelines on comorbidities that increase individuals’ risks from influenza infection and recommendations for seasonal influenza vaccination.
Cohort Studies
Four cohort studies met the review’s inclusion criteria, with 2 prospective10,22 and 2 retrospective evaluations (Table 2).28,29 All had sufficient results to be included within the quantitative analysis and pooled effect estimate. Moore and Parker29 evaluated the use of a postal letter for an intervention group versus verbal reminders in the control group. Fiks et al28 compared use of electronic provider reminders with a provider educational session in an intervention group versus a control group who received the educational session alone. Bay and Crawford10 tested electronic reminders delivered through an online patient portal system with educational material links, compared with a standard of care in a single cohort. Dombkowski et al22 examined postal letter reminders for a seasonal and pandemic influenza A/H1N109 vaccine compared with standard of care.
Characteristics and Results of Cohort Studies
| Authors and Publication Year | Setting, Country, and Study Years | Study Participants (Population) | Participant Ages | Study Methodology | Vaccination Status Ascertainment | Intervention Type | Absolute Effect Size (Vaccine Coverage %) | Relative Effect Sizes (% of Vaccine Coverage) | Study Summary |
|---|---|---|---|---|---|---|---|---|---|
| Bay and Crawford10 (2017) | Pediatric respiratory clinic, United States (2015) | Pediatric respiratory disease patients: n = 107 | Not specified | Retrospective cohort study | Parental reporting | Parent reminder | Absolute influenza vaccination increased by 20.8% | Intervention: 70.8%; no intervention: 50% | Families of pediatric respiratory clinic patients who reported receiving influenza vaccination reminders had significantly higher influenza vaccine uptake |
| Dombkowski et al22 (2014) | Michigan immunization registry, United States (2009–2010) | Children with comorbidities (ACIP guidelinesa): n = 142 383 | 0.5–18 y | Retrospective cohort study | Immunization registry records (ICD-9 codesb) | Parent reminder | Pandemic H1N1 vaccine: coverage increase of 2.2%; seasonal influenza vaccine: coverage increase of 2.1% | Pandemic H1N1 vaccine: intervention group: 6.5%, control group: 4.3%. Seasonal influenza vaccine: intervention group: 4.8%, control group: 2.7% | Children with comorbidities were more likely to receive both pandemic and seasonal influenza vaccines if they received a mailed vaccination letter compared with those who did not |
| Fiks et al28 (2009) | Pediatric outpatient clinics, United States (2006–2007) | Pediatric asthmatic patients: intervention group baseline year: n = 5329; intervention group study year: n = 6110; control group baseline year: n = 5338; control group study year: n = 5809 | 5–18 y | Prospective cohort studyc | Electronic vaccination records | Provider reminder | Absolute improve vaccination between coverage difference between intervention and control groups of 3.4% | Intervention group: before: 45.7%, after: 51.0%; control group: before: 46.0%, after: 47.9% | Electronic medical record vaccination reminder prompts for physicians had minimal impact on influenza vaccine uptake |
| Moore and Parker29 (2006) | Pediatric clinics, United States (2004–2005) | Pediatric asthmatic patients: letters reminders group: n = 114; verbal reminders group: n = 820 | 0.5–18 y | Prospective cohort study | Electronic vaccination records | Parent reminder | Absolute influenza vaccination coverage difference in intervention group = 14.1% | Letters reminders group: 57.9%, verbal reminders group: 43.8% | Influenza vaccination reminders for asthmatic pediatric patients by mailed letter showed superiority over verbal reminders by providers for influenza vaccine uptake |
| Authors and Publication Year | Setting, Country, and Study Years | Study Participants (Population) | Participant Ages | Study Methodology | Vaccination Status Ascertainment | Intervention Type | Absolute Effect Size (Vaccine Coverage %) | Relative Effect Sizes (% of Vaccine Coverage) | Study Summary |
|---|---|---|---|---|---|---|---|---|---|
| Bay and Crawford10 (2017) | Pediatric respiratory clinic, United States (2015) | Pediatric respiratory disease patients: n = 107 | Not specified | Retrospective cohort study | Parental reporting | Parent reminder | Absolute influenza vaccination increased by 20.8% | Intervention: 70.8%; no intervention: 50% | Families of pediatric respiratory clinic patients who reported receiving influenza vaccination reminders had significantly higher influenza vaccine uptake |
| Dombkowski et al22 (2014) | Michigan immunization registry, United States (2009–2010) | Children with comorbidities (ACIP guidelinesa): n = 142 383 | 0.5–18 y | Retrospective cohort study | Immunization registry records (ICD-9 codesb) | Parent reminder | Pandemic H1N1 vaccine: coverage increase of 2.2%; seasonal influenza vaccine: coverage increase of 2.1% | Pandemic H1N1 vaccine: intervention group: 6.5%, control group: 4.3%. Seasonal influenza vaccine: intervention group: 4.8%, control group: 2.7% | Children with comorbidities were more likely to receive both pandemic and seasonal influenza vaccines if they received a mailed vaccination letter compared with those who did not |
| Fiks et al28 (2009) | Pediatric outpatient clinics, United States (2006–2007) | Pediatric asthmatic patients: intervention group baseline year: n = 5329; intervention group study year: n = 6110; control group baseline year: n = 5338; control group study year: n = 5809 | 5–18 y | Prospective cohort studyc | Electronic vaccination records | Provider reminder | Absolute improve vaccination between coverage difference between intervention and control groups of 3.4% | Intervention group: before: 45.7%, after: 51.0%; control group: before: 46.0%, after: 47.9% | Electronic medical record vaccination reminder prompts for physicians had minimal impact on influenza vaccine uptake |
| Moore and Parker29 (2006) | Pediatric clinics, United States (2004–2005) | Pediatric asthmatic patients: letters reminders group: n = 114; verbal reminders group: n = 820 | 0.5–18 y | Prospective cohort study | Electronic vaccination records | Parent reminder | Absolute influenza vaccination coverage difference in intervention group = 14.1% | Letters reminders group: 57.9%, verbal reminders group: 43.8% | Influenza vaccination reminders for asthmatic pediatric patients by mailed letter showed superiority over verbal reminders by providers for influenza vaccine uptake |
ACIP, Advisory Committee on Immunization Practices; ICD-9, International Classification of Diseases, Ninth Revision.
ACIP guidelines: guidelines on comorbidities that increase individuals’ risks from influenza infection and recommendations for seasonal influenza vaccination.
ICD-9 codes: International Classification of Diseases, Ninth Revision.
Fiks et al28 conducted both a before and after evaluation and a prospective cohort study within their population for the 2006–2007 northern hemisphere influenza season.
All cohort studies were conducted within the United States and were published after 2005. Collectively, cohort studies revealed low to moderate improvements in influenza vaccination coverage. Coverage improvements were significant for all interventions except that of Fiks et al,28 who used a provider electronic reminder revealing only a nonsignificant increase of 3.4% in coverage for their intervention group versus their control group. The greatest improvement was shown by Bay and Crawford,10 with a 20.8% increase in coverage for children whose families reported receiving a reminder through reminder messages and education via an online patient portal.
Quasi-Experimental Studies
Quasi-experimental studies were the most varied, with researchers examining different interventions for both respiratory and nonrespiratory comorbidities (Supplemental Table 4). Multicomponent interventions were only evaluated by using quasi-experimental studies. QI studies were only conducted post-2006 and in the United States, with 60% targeting patients of pediatric oncology clinics with multicomponent interventions. Before and after studies took place across a longer time frame, in Europe, Australia, and North America, and examined both singular and multicomponent for a range of respiratory and nonrespiratory comorbidities. Appropriate quantitative results were present in 18 of the included quasi-experimental studies for inclusion in the pooled effect estimate. Quasi-experimental studies had the greatest range of influenza vaccination coverage impacts, with increases from 3.4%28 to 45.5%53 postintervention. Additionally, a minority of quasi-experimental studies had nonsignificant changes in vaccination coverage38,41,45,46 or reported results in which absolute coverage changes could not be ascertained.42,48,49,51
Pooled Estimates
From 5 RCTs, 4 cohort studies, and 18 quasi-experimental studies included within the quantitative analyses, 33 separate intervention results had sufficient data for pool effect estimates (Fig 2). Researchers in 2 studies examined 3 different telephone-based parental reminders from different types of clinicians (ie, pediatric oncologist and public health physicians) using before and after intervention designs in separate populations; these were considered as 6 distinct data sets for quantitative analysis.32,34 Dombkowski et al22 evaluated the impact of the same reminder system for both seasonal and pandemic influenza vaccinations programs in 2009. Fiks et al28 conducted both a before and after study and a prospective cohort study examining the introduction of an electronic health record alert. Rao et al52 evaluated separate provider reminder and parental education intervention groups to a control group within their QI study.
Pooled effect estimates and forest plots. a Dombkowski et al22 evaluated both seasonal and pandemic A/H1N109 vaccines for the 2009–2010 northern hemisphere influenza season. b Fiks et al28 conducted both a before and after evaluation and a prospective cohort study within their population for the 2006–2007 northern hemisphere influenza season. c Cecinati et al32 trialed 3 different variations of parental reminders in 3 distinct populations of pediatric oncology patients and therefore results were treated as 3 different before and after studies. d Esposito et al34 trialed 3 different variations of parental reminders in 3 distinct populations of pediatric asthmatic patients and therefore results were treated as 3 different before and after studies. e Rao et al52 evaluated both a provider reminder intervention and parental educational tool in distinct populations; as such, they were treated as 2 separate data sets. df, degrees of freedom.
Pooled effect estimates and forest plots. a Dombkowski et al22 evaluated both seasonal and pandemic A/H1N109 vaccines for the 2009–2010 northern hemisphere influenza season. b Fiks et al28 conducted both a before and after evaluation and a prospective cohort study within their population for the 2006–2007 northern hemisphere influenza season. c Cecinati et al32 trialed 3 different variations of parental reminders in 3 distinct populations of pediatric oncology patients and therefore results were treated as 3 different before and after studies. d Esposito et al34 trialed 3 different variations of parental reminders in 3 distinct populations of pediatric asthmatic patients and therefore results were treated as 3 different before and after studies. e Rao et al52 evaluated both a provider reminder intervention and parental educational tool in distinct populations; as such, they were treated as 2 separate data sets. df, degrees of freedom.
The effect estimate following pooling of interventions’ quantitative results revealed an average increase in influenza vaccine coverage of 60% (risk ratio: 1.60; 95% CI: 1.47–1.74) across 368 574 participants. Point estimates varied between intervention types, singular-component versus multicomponent intervention, and study methods; however, the relative risks of specific intervention types were not significantly different (Supplemental Table 5). Likewise, the effect estimates of different study methodologies were not significantly different (Supplemental Table 6).
Risk of Bias
Substantial heterogeneity was observed across studies, with the overall and subgroup analyses all having I2 scores >75% (Fig 2).55 Overall, a high level of bias was observed, with all but 1 study having a high risk of bias in at least 1 of GRADE’s domains (Table 3). RCTs had a lower average number of high-risk bias domains per study of 1.8 compared with quasi-cohort and experimental studies with average scores of 3.25 and 4.03, respectively.
GRADE Risk of Bias Domain Scores
| Study and Year | GRADE Risk of Bias Quality Assessment | |||||
|---|---|---|---|---|---|---|
| Random Sequence Generation | Allocation Concealment | Blinding of Participants and Personnel | Blinding of Outcome Assessment | Incomplete Outcome Data | Selective Reporting | |
| Bay and Crawford10 (2016) | − | − | − | − | − | − |
| Bjornson et al30 (2000) | − | − | − | − | − | − |
| Britto et al48 (2006) | − | − | − | − | − | − |
| Britto et al49 (2007) | − | − | − | − | − | − |
| Camurdan et al31 (2012) | − | − | − | − | − | − |
| Cecinati et al32 (2010) | + | + | ? | − | + | − |
| Crawford et al33 (2014) | − | − | − | − | + | + |
| Daley et al23 (2004) | + | + | − | − | + | + |
| Dombkowski et al24 (2012) | + | + | − | − | − | + |
| Dombkowski et al22 (2014) | − | − | − | − | + | + |
| Esposito et al34 (2009) | + | ? | − | − | − | ? |
| Fiks et al28 (2009) | ? | ? | − | + | ? | ? |
| Freedman et al50 (2014) | − | − | − | + | ? | ? |
| Gaglani et al35 (2001) | − | − | − | + | ? | ? |
| Gattis et al36 (2018) | − | − | − | + | ? | ? |
| Gregori et al21 (2012) | − | − | − | − | − | ? |
| Hofstetter et al25 (2017) | + | + | ? | + | − | ? |
| Huth et al37 (2015) | − | − | − | ? | − | ? |
| Kempe et al38 (2014) | + | ? | − | − | + | + |
| Kemper et al26 (1993) | + | + | − | + | − | − |
| Lin et al39 (2015) | − | − | − | − | + | + |
| Martin40 (2008) | − | − | − | − | + | + |
| McCreary41 (2013) | − | − | − | + | + | + |
| Merckx et al42 (2016) | − | − | − | − | + | − |
| Moore and Parker29 (2006) | − | − | − | + | + | + |
| Olshefski et al51 (2018) | − | − | ? | − | − | + |
| Patwardhan et al43 (2011) | − | − | − | ? | + | ? |
| Paul et al44 (2006) | − | − | − | + | + | + |
| Rao et al52 (2018) | − | − | − | − | + | + |
| Sobota et al53 (2015) | − | − | − | − | + | + |
| Szilagyi et al27 (1992) | + | + | + | ? | ? | + |
| Urkin et al45 (2016) | − | − | − | ? | − | − |
| Walter et al46 (1997) | − | − | ? | ? | ? | ? |
| Wong et al54 (2017) | − | − | − | − | + | ? |
| Zimmerman et al47 (2006) | − | − | − | + | + | − |
| Study and Year | GRADE Risk of Bias Quality Assessment | |||||
|---|---|---|---|---|---|---|
| Random Sequence Generation | Allocation Concealment | Blinding of Participants and Personnel | Blinding of Outcome Assessment | Incomplete Outcome Data | Selective Reporting | |
| Bay and Crawford10 (2016) | − | − | − | − | − | − |
| Bjornson et al30 (2000) | − | − | − | − | − | − |
| Britto et al48 (2006) | − | − | − | − | − | − |
| Britto et al49 (2007) | − | − | − | − | − | − |
| Camurdan et al31 (2012) | − | − | − | − | − | − |
| Cecinati et al32 (2010) | + | + | ? | − | + | − |
| Crawford et al33 (2014) | − | − | − | − | + | + |
| Daley et al23 (2004) | + | + | − | − | + | + |
| Dombkowski et al24 (2012) | + | + | − | − | − | + |
| Dombkowski et al22 (2014) | − | − | − | − | + | + |
| Esposito et al34 (2009) | + | ? | − | − | − | ? |
| Fiks et al28 (2009) | ? | ? | − | + | ? | ? |
| Freedman et al50 (2014) | − | − | − | + | ? | ? |
| Gaglani et al35 (2001) | − | − | − | + | ? | ? |
| Gattis et al36 (2018) | − | − | − | + | ? | ? |
| Gregori et al21 (2012) | − | − | − | − | − | ? |
| Hofstetter et al25 (2017) | + | + | ? | + | − | ? |
| Huth et al37 (2015) | − | − | − | ? | − | ? |
| Kempe et al38 (2014) | + | ? | − | − | + | + |
| Kemper et al26 (1993) | + | + | − | + | − | − |
| Lin et al39 (2015) | − | − | − | − | + | + |
| Martin40 (2008) | − | − | − | − | + | + |
| McCreary41 (2013) | − | − | − | + | + | + |
| Merckx et al42 (2016) | − | − | − | − | + | − |
| Moore and Parker29 (2006) | − | − | − | + | + | + |
| Olshefski et al51 (2018) | − | − | ? | − | − | + |
| Patwardhan et al43 (2011) | − | − | − | ? | + | ? |
| Paul et al44 (2006) | − | − | − | + | + | + |
| Rao et al52 (2018) | − | − | − | − | + | + |
| Sobota et al53 (2015) | − | − | − | − | + | + |
| Szilagyi et al27 (1992) | + | + | + | ? | ? | + |
| Urkin et al45 (2016) | − | − | − | ? | − | − |
| Walter et al46 (1997) | − | − | ? | ? | ? | ? |
| Wong et al54 (2017) | − | − | − | − | + | ? |
| Zimmerman et al47 (2006) | − | − | − | + | + | − |
The score of each risk of bias domain for the listed studies by their respective symbol, with low risk (plus sign), unclear risk (question mark), or high risk (dash).
Discussion
Interventions of vaccination reminders and influenza vaccination education targeting either parents or providers as well as clinic process changes were all shown to have effectiveness for improving influenza vaccine coverage in children with comorbidities. Numerous biases and high heterogeneity were observed across all study types because of study designs, vaccination status ascertainment methods, and outcomes reported. These biases limited our capacity to determine if significant differences existed between intervention types, and the high level of heterogeneity reduced confidence in the meta-analysis results.
RCTs revealed how simple vaccination reminders targeting patients’ parents or guardian could moderately improve influenza vaccine coverage. Cohort studies had similarly low to moderate improvements for influenza vaccination coverage with parental reminders. Conversely, researchers in quasi-experimental studies examined a variety of intervention types and moderate-to-high impact on increasing influenza vaccine coverage for a diversity of patients. Unsurprisingly, RCTs had a lower average number of high risk of bias domains compared with cohort and quasi-experimental studies. Future intervention designers will need to balance the low bias risk and rigorous design requirements of an RCT with the capacity and flexibility for intervention design and evaluation provided by cohort and quasi-experimental designs. Moreover, researchers in future intervention evaluations should implement standards of control groups because of the ethical concerns of not recommending influenza vaccination to vulnerable patients.
The high heterogeneity of studies made identifying discernible trends across studies methods and results through meta-analysis difficult. A narrative synthesis identified a shift toward multicomponent interventions, targeting of children with nonrespiratory comorbidities, and greater use of information technologies (ie, electronic medical records and e-mail or text message reminders), particularly with more recent publications (since 2010). This was likely due to greater use of multicomponent interventions across public health, clinical integration of information technologies, and recognition of the severe influenza infection impacts in children with nonrespiratory comorbidities.
Using quantitative analysis, we found an average improvement in coverage by 60% (RR: 1.60 [95% CI: 1.47–1.75]). The high heterogeneity of studies and strong risk of bias limited our confidence in this effect estimate and further subanalyses of quantitative results. Furthermore, we could not establish if any single intervention type or a multicomponent intervention showed greater efficacy. Moreover, differences in point estimates between different intervention types should be interpreted with caution given overlapping CIs and biases observed.
As reported in a previous systematic review,11 parental letter-based reminders improve influenza vaccine coverage. We additionally identified evidence revealing that provider reminders, educational interventions, and clinic process changes can also improve vaccine coverage. We identified a further 17 articles* not captured by the previous review11 and excluded 2 that were previously included (a conference abstract56 and a systematic review of already included articles57 ). In the previous review, authors claimed that meta-analysis was inappropriate because of high heterogeneity between studies. Although high heterogeneity was observed, we believe pooled analyses are important to identify potential effect sizes. These pooled estimates are required to inform development and design appropriately conducted clinical trials and evaluations assessing the relative benefit of different interventions.
This is the first attempt to quantify the overall impact on interventions designed to improve influenza vaccine coverage for children with comorbidities. Optimizing interventions to improve influenza vaccine use in high-risk pediatric populations is clinically relevant because of the increased severity of influenza infections in these children and potential negative impacts of unnecessary interventions in families with high-care needs. Superiority of multicomponent interventions compared with single interventions has been shown in similar contexts for improving vaccine coverage in high-risk adults58 and human papillomavirus vaccine use,59 but we were not able to demonstrate superiority in this context.
Further evaluation of interventions’ costs, feasibility, and acceptance by clinical staff, patients, and families are necessary for future optimal intervention design. Use of negative control groups within such studies, particularly populations at high risk of influenza, needs careful consideration. Assessing pre-established “standard of care models” concurrently with additional interventions enables researchers to assess impacts, maintain high methodology quality and ensure appropriate patient care.
Our results were primarily limited by the quality of studies available for analysis. The relatively high number of quasi-experimental studies published compared with RCTs and well-constructed cohort studies is reflective of their convenience, relatively low cost, and ease of implementation within diverse populations and settings. The ability to disaggregate and assess the relative impact of multicomponent interventions remains challenging given the diversity of interventions and study design. Use of the GRADE risk of bias evaluation tool was chosen a priori.20 We were limited in our capacity to compare the relative quality of less rigorous study designs, particularly the quasi-experimental studies.
Conclusions
Interventions targeting influenza vaccination in children with medical comorbidities through vaccination reminders, education targeting parents or providers, and clinical process changes improved coverage. No intervention type was clearly superior. Multicomponent interventions have been used for children with respiratory and nonrespiratory comorbidities, whereas single component interventions have been used predominately to target children with respiratory comorbidities. However, superiority of single or multicomponent interventions for improving influenza vaccination was not established. High level of methodologic bias, poor quality of evidence, and small study size limit conclusions that can be drawn from the literature. Future design and evaluation of interventions to improve influenza vaccine coverage should directly compare intervention types using rigorous study methodologies to optimize effectiveness and reduce the influenza disease burden in children with medical comorbidities.
Acknowledgments
This project constitutes part of Daniel Norman’s doctor of philosophy candidature and, as such, was not directly sponsored or funded. Mr Norman’s PhD is funded by the Commonwealth of Australian’ research training program PhD scholarship and the Wesfarmers Centre for Vaccines and Infectious Diseases PhD top-up scholarship. Prof Blyth is supported by a fellowship from the National Health and Medical Research Council of Australia. Dr Moore is supported by a Telethon Kids Institute Emerging Research Leader Fellowship. Prof Danchin is Melbourne University’s David Bickart Clinician Scientist fellowship recipient.
Mr Norman conceived and designed the study, led publication review, data extraction, quantitative analysis, qualitative analysis, and risk of bias assessments, and wrote the manuscript; Prof Blyth oversaw the design of the study, analyses, and manuscript writing; Dr Barnes contributed to the screening of articles, data extraction, and analysis; Drs Pavlos, Alene, and Bhuiyan contributed to the evaluation of risk of bias for included studies; Dr Moore, Prof Seale, and Prof Danchin assisted in designing of the review and drafting of the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
This trial has been registered with the National Institute for Health Research (https://www.crd.york.ac.uk/prospero/) (identifier PROSPERO CRD42019090623).
FUNDING: No external funding was secured for this study. Prof Blyth is supported by a fellowship from the National Health and Medical Research Council of Australia. Dr Moore is supported by a Telethon Kids Institute Emerging Research Leader Fellowship. The other authors received no additional funding.
Comments