Identifying Practices to Promote Inpatient Adolescent and Influenza Vaccine Delivery

Responses from a survey regarding inpatient vaccine delivery practices were combined with utilization data on inpatient adolescent and influenza vaccine administration from the Pediatric Health Information System (PHIS) (Children’s Hospital Association, Lenexa, KS) database to determine if specific hospital-level practices impact vaccine delivery. PHIS is a national pediatric clinical and resource utilization database representing 51 children’s hospitals. Participating institutions submit billing data to PHIS, which are deidentified and subject to a number of reliability and validity checks before inclusion in the database.

Our group previously delivered a survey via the Pediatric Research in Inpatient Settings (PRIS) Network,²⁵  a pediatric hospital medicine research network composed of 107 hospitals across the United States and Canada, to describe national inpatient immunization practices. The PRIS Network and PHIS database are independent organizations, and 13 PHIS hospitals are not members of the PRIS Network. Thus, to obtain a more complete data set, an identical survey also was sent to a designated PHIS contact or hospital medicine physician at each of these 13 institutions. PHIS data were matched by hospital with survey responses.

An electronic Qualtrics (Qualtrics, Provo, UT) survey regarding hospital-level inpatient vaccine delivery practices was distributed to PRIS site lead physicians via the PRIS e-mail LISTSERV in April 2019²⁵  and to the additional 13 PHIS hospitals in February 2020. To increase accuracy of responses, physician participants were asked to identify a nurse and/or pharmacy leader at their institution to also complete the survey. All participants received an initial survey invitation and up to 2 reminders. Responses from a single hospital (physician and nurse and/or pharmacist) were matched to verify agreement, and participants were contacted for clarification if differences occurred, yielding one unique response per institution. If a discrepant question item was unable to be resolved, any participant indicating the presence of a practice was considered valid.

Respondents were asked about the presence or absence of specific practices at their hospital to increase inpatient vaccine delivery (Table 1). For questions regarding hospital EMR integration with the state vaccine registry, participants were asked whether their hospital had this capability. For all other vaccine delivery practices, participants were asked to select whether their hospital employed each practice for multiple vaccine types, including tetanus, diphtheria, and acellular pertussis (Tdap); meningococcal; HPV; and influenza vaccines. Given previously described differences in provider perceptions and recommendation practices surrounding the HPV vaccine,^26–29  the adolescent vaccines were intentionally separated on the survey and not included as a single category.

The percentage of inpatient and observation discharges that included an adolescent or influenza vaccine at each participating hospital from January 1 to December 31, 2018, was obtained from the PHIS database. This time period was chosen to best reflect the practices in place before initial survey administration. Vaccines were identified by using billing data and grouped into 4 categories: HPV-containing, meningococcal-containing, tetanus- and/or diphtheria-containing, and influenza-containing vaccines.

Patients 11 to 18 years old were included for the adolescent vaccines (Tdap, meningococcal, and HPV); and patients 6 months to 18 years old were included for the influenza vaccine. We did not adjust for influenza vaccine seasonality because this should affect all hospitals similarly and therefore should not bias our results toward a specific delivery practice. However, because this likely underestimates vaccine delivery rates during applicable encounters, we calculated unadjusted influenza vaccine delivery rates for both total and seasonal (excluding April through August) cohorts. PHIS sites without a corresponding survey were excluded.

Demographic data for eligible encounters were collected from the PHIS database, including sex, race and ethnicity, primary source of payment, length of stay, number of encounters per patient during the study time frame, severity-level weights, and the presence of a complex chronic condition (CCC).³⁰  Publicly available sources were used to identify US Census geographic designations,³¹  hospital status as public or private,³²  and presence of pediatric resident physician trainees.³³ 

The primary outcome was adjusted hospital-level vaccine delivery rates for influenza, Tdap, meningococcal, and HPV vaccines, as well as for the adolescent vaccines as a group. We analyzed the adolescent vaccines both separately and combined in an attempt to account for (1) possible variation in practice between the 3 adolescent vaccines and (2) low expected inpatient adolescent vaccine delivery that may limit the ability to detect significant outcomes for individual vaccine types.

Unadjusted inpatient vaccine delivery rates for each vaccine type (number of vaccines delivered divided by patient discharges) were risk adjusted using generalized linear mixed-effects models. Variables included in the model were length of stay, the number of encounters per patient, severity weights using the Hospitalization Resource Intensity Scores for Kids,³⁴  presence of a malignancy CCC, and presence of a hematologic or immunologic CCC. These variables were chosen because the exposures (such as shorter length of stay or increased proportion of patients with oncologic or immunologic conditions) may decrease opportunities to provide inpatient vaccines. Additional demographic factors that may impact vaccination rates but do not alter the number of opportunities to vaccinate, such as sex and race,^15,35–37  were not included in the model. For example, although male adolescents have lower HPV vaccination rates than female adolescents,³⁵  both are equally eligible to receive HPV vaccines.

Categorical variables were summarized with frequencies and percentages, whereas continuous variables were summarized with medians and interquartile ranges. Hospital-level characteristics, risk adjustment variables, and unadjusted inpatient vaccine delivery rates were compared between respondent and nonrespondent PHIS hospitals by using χ² tests or Wilcoxon rank tests as appropriate. In subsequent analyses, we included only hospitals that responded to the survey.

Hospital-level adjusted vaccine delivery rates were generated from generalized linear mixed-effects models and compared with individual survey responses by using Wilcoxon rank-sum tests. We compared vaccine delivery rates between hospitals that indicated that they had each practice and those that did not using Wilcoxon rank-sum tests. Regression models were used to associate hospital-level adjusted vaccine delivery rates with the number of vaccine delivery practices for each vaccine type. To determine if a unique combination of practices was associated with adjusted vaccine delivery rates, we developed classification and regression trees (CART). CART is a modeling technique designed to elucidate high-order interactions associated with an outcome that is typically difficult to identify with traditional statistical models. We used the entropy method to grow the tree and allowed the tree to grow to its full potential.

Statistical significance was set at a P < .05, and SAS version 9.4 (SAS Institute, Inc, Cary, NC) was used for all analyses. This study was approved by the Children's Hospital Los Angeles Institutional Review Board.

Fifty-one hospitals participated in PHIS during the study period; 29 (57%) completed a survey. Twenty participants were from the original PRIS survey distribution (38 eligible, 53% response rate), and 9 were from non-PRIS sites (13 eligible, 69% response rate). Thirteen hospitals (45%) had survey responses from physicians only. Sixteen hospitals (55%) had responses from a physician and a nurse and/or pharmacist at their institution; discrepant responses were unable to be resolved in 20 of 1392 possible question items (1%) across all hospitals. From the PHIS database, 152 499 patient encounters were included for the adolescent age group and 423 046 patient encounters were included for the influenza age group.

Respondent hospitals varied geographically, and all sites had pediatric resident trainees. There were no differences between respondent and nonrespondent hospitals with respect to region, hospital type, presence of trainees, or unadjusted inpatient vaccine delivery rates. For both age cohorts, respondent hospitals had slightly longer lengths of stay, patient populations with higher severity and higher rates of hematologic or immunologic CCCs, malignancy CCCs, ICU stays, and any CCC, as well as differences in race and ethnicity and primary source of payment compared with nonrespondent hospitals (Table 2).

Unadjusted inpatient vaccine delivery rates varied between hospitals and vaccine types (Fig 1). The median unadjusted inpatient vaccine delivery rate for the influenza vaccine in respondent hospitals was 3.2% (range 0.4%–9.1%), compared with 0.5% for Tdap (range 0.2%–3.3%), 0.1% for meningococcal (range 0%–2.6%), and 0% for HPV (range 0%–2.3%) vaccines. When restricted to the influenza season, the median unadjusted inpatient influenza vaccine delivery rate was 6.9% (range 0.6%–14.7%). The median number of reported delivery practices also varied by vaccine type: 5 practices (range 0–10) for influenza, 1 (range 0–6) for Tdap, 0 (range 0–6) for meningococcal, and 0 (range 0–6) for HPV vaccines.

One outlier hospital with higher meningococcal and HPV vaccine delivery rates than other respondents was identified. With the outlier hospital removed, unadjusted inpatient vaccine delivery rates ranged from 0% to 0.5% for both vaccine types. Given low rates of meningococcal and HPV vaccine delivery, we were unable to perform risk adjustment for these 2 vaccine groups.

The total number of influenza vaccine delivery practices present at an institution was associated with increased influenza vaccine delivery (P = .02; Fig 2). Visual prompts (P = .02), nurse or pharmacist ability to order vaccines (P = .003), and existing quality improvement (QI) projects (P = .048) were independently associated with increased influenza vaccine delivery (Fig 3). When CART modeling was applied, only nurse or pharmacist ability to order vaccines showed the highest association with influenza vaccine delivery rates: 7.8% in sites with this practice, compared with 4.2% in sites without nonphysician vaccine ordering. No further splits were statistically significant in CART modeling.

The total number of practices was associated with increased unadjusted delivery of HPV vaccines (P = .002; Supplemental Fig 4), and nurse or pharmacist screening was independently associated with increased HPV vaccine delivery (P = .04). Neither the total number of practices nor the presence of any specific practice was associated with increased unadjusted meningococcal vaccine delivery rates. After removal of the outlier hospital, which reported 3 meningococcal and 4 HPV vaccine delivery practices, the associations remained between the number of practices present and inpatient vaccine delivery rates for HPV (P < .001) and meningococcal (P = .17) vaccines.

The total number of practices was not associated with increased delivery of the Tdap vaccine or the adolescent vaccines as a group. Nurse or pharmacist screening was independently associated with increased delivery of the adolescent vaccines as a group (P = .04); no practices were independently associated with increased vaccine delivery rates for the Tdap vaccine.

To our knowledge, this is the first study exploring the relationship between specific hospital practices and delivery rates for influenza and adolescent vaccines on a national level. We found that few vaccines were given in the inpatient setting, which was most pronounced for adolescent vaccines. We also found increased adjusted influenza vaccine delivery rates at hospitals with a higher number of influenza vaccine delivery practices, as well as at hospitals with nonphysician vaccine ordering, QI projects, and visual prompts.

Optimizing inpatient influenza vaccine delivery has significant public health implications. This may be especially beneficial for patients with difficulty accessing primary care or during times of disruption to health care delivery systems, such as the coronavirus disease 2019 pandemic, which reduced pediatric vaccine delivery.³⁸  Influenza vaccination has been shown to decrease the risk of death from influenza by 65%,^39,40  and reducing missed opportunities is an integral component of improving immunization rates. In one study of children hospitalized with influenza, the authors found that 16% of patients had a previous admission during that influenza season⁴¹ ; in another study, the authors found that 15% of missed influenza vaccination opportunities in pediatric patients hospitalized with influenza occurred in the inpatient setting.¹² 

Our finding of higher influenza vaccine delivery rates at hospitals with more vaccine administration practices may be due to several factors. These institutions may have better developed workflows to support immunization, as well as increased provider buy-in and staff who are familiar with discussion and delivery of vaccines. This may overcome previously identified barriers to hospital-based immunization programs, including lack of staff knowledge or training, inadequate systems supporting immunization (such as vaccine supply or parental handouts), or providers forgetting to identify patients who are underimmunized and order vaccines.^{9,10,23,42–44}  Although we did not find the same association for the Tdap vaccine, this may be due to the lower number of hospital practices in place for this vaccine, making the benefits of increased physician familiarity and up-to-date systems less appreciable.

Independent associations between increased influenza vaccine delivery rates and visual prompts, QI projects, and nonphysician vaccine ordering were also identified, with nurse or pharmacist ordering ability having the greatest impact on influenza vaccine delivery. Implementation of nonphysician protocols has previously been shown to increase inpatient influenza vaccination,²¹  and the authors of one QI study credited the sustainability of their success to a protocol allowing nurses to offer and administer influenza vaccines.²³  Our results are in line with this and suggest that nonphysician vaccine ordering may be the highest yield practice for hospitals looking to increase inpatient influenza vaccine administration. Interestingly, we did not find a significant impact of EMR and state vaccine registry integration on delivery of any vaccine type. Inpatient providers have previously cited difficulty in obtaining vaccination records as a major barrier to hospital-based immunization^3,45 ; however, in our study, we only assessed whether a practice was present and not the degree to which it was used, thus this finding deserves further investigation.

Finally, low rates of inpatient adolescent vaccine delivery were apparent. In our study, we used aggregated hospital-level billing data; thus, we are unable to determine the number of patients who refused vaccines or who were ineligible because of previous immunization. However, because hospitalized adolescents are the age group most likely to be undervaccinated,^1,3,4  having so few vaccines delivered likely represents missed opportunities. In contrast to the influenza vaccine, which is delivered annually, adolescents may present to the hospital years after their last vaccination, which may impact parental recall. Inpatient providers’ practices also may vary on the basis of vaccine type; for example, pediatric residents less frequently discuss or offer the HPV vaccine during hospitalization compared with other vaccines.²⁹  Finally, in the era of diagnosis-related group payments, the cost of adolescent vaccines (which can cost up to $228 for 1 dose of the HPV vaccine) may be a limiting factor.⁴⁶  With only half of adolescents, and one-fifth of uninsured adolescents, visiting a primary care physician in a one-year period,¹⁶  optimizing hospital-based vaccination appears particularly beneficial for this age group.

Several limitations to this study exist. First, we found multiple statistically significant differences between respondent and nonrespondent hospitals, such as higher incidences of malignancy and hematologic or immunologic CCCs, higher severity scores, incidence of any CCC, ICU stays, and longer length of stays. Although statistically significant, these differences (such as a length of stay of 2.51 vs 2.46 days) may not be clinically significant. Differences in severity level, incidence of malignancy and hematologic or immunologic CCCs, and length of stay may decrease immunization opportunities in respondent hospitals compared with nonrespondent hospitals; however, we adjusted for these variables in our model, and thus we would not expect them to bias results toward a particular vaccine delivery practice.

Second, all respondents represent children’s hospitals with pediatric trainees, which may limit generalizability, and 57% of eligible PHIS sites responded to the survey, which may introduce participant self-selection bias. After conducting our initial survey via the PRIS Network,²⁵  we aimed to create a more inclusive data set by sending our survey to the 13 PHIS hospitals that did not participate in the PRIS Network. We did find higher response rates in the latter recruitment period; thus, there may be unidentified factors that differentiate these two groups. Although we believe that inviting all eligible hospitals strengthened our study overall by garnering a more complete sample population, the time discrepancy between PHIS data collection and non-PRIS Network survey distribution limits our ability to account for any changes in hospital vaccine policy that might have occurred during this time.

Third, we were unable to create adjusted vaccine delivery rates for HPV and meningococcal vaccines given low delivery, which limits our ability to comment on associations with hospital practices for these vaccine types. In addition, because of the small number of hospital respondents, some statistical tests might have been underpowered. Finally, there might have been factors impacting hospital vaccine delivery that we were unable to account for in our study, such as local immunization rates or state policies.

Few vaccines are given in the hospital setting; however, the number and presence of specific practices, particularly nonphysician vaccine ordering, may impact inpatient influenza vaccine delivery. Further research is needed to identify strategies that enhance delivery of adolescent vaccines. In addition, qualitative studies of hospitals with robust vaccine delivery programs may be able to identify additional practices and strategies to overcome barriers that were not captured in our study. Given suboptimal adolescent and influenza immunization rates and the large number of pediatric inpatients who are undervaccinated, identifying strategies to better deliver vaccines to hospitalized children and adolescents has potentially significant public health benefits.

We thank Dr Sunitha Kaiser and the PRIS Executive Council for their contributions to designing this study.