Vaccination Status and Adherence to Quality Measures for Acute Respiratory Tract Illnesses

We conducted a prospective cohort study of children who presented with ARTI between July 2014 and June 2016 to 5 freestanding academic children’s hospitals in the United States participating in the Pediatric Research in Inpatient Settings Network. ARTI was defined as community-acquired pneumonia (CAP), croup, asthma, and bronchiolitis. These ARTIs were included in this study because vaccine-preventable diseases, such as pneumococcal disease, Haemophilus influenzae disease, pertussis, and influenza, were potentially relevant to the differential diagnosis, clinical presentation, and subsequent medical decision-making for each condition. Children were included in the study if they were between 2 weeks and 16 years old, if the family spoke English or Spanish, and if they had 1 of the 4 aforementioned ARTI conditions. All children included in the study who were diagnosed with CAP, asthma, and bronchiolitis were admitted to the hospital. Children with croup were either admitted to the hospital or discharged from the ED. We opted to include children with croup who were discharged from the ED given the lower rates of hospital admission for croup compared with the other 3 conditions. If a child had >1 hospitalization during the study period, only the first hospitalization was included in this analysis.

Subjects were excluded if they (1) had an underlying condition that would alter the routine childhood vaccination schedule (including requiring additional vaccines), such as immunodeficiencies, HIV, or asplenia, per the recommendations of the Advisory Committee on Immunization Practices¹³  or (2) had a chronic medical condition that would alter the standard of care for ARTI, including a history of cardiac disease requiring baseline medication, anatomic airway abnormalities, cystic fibrosis, neuromuscular disease, bronchopulmonary dysplasia, immunodeficiency, or chronic lung disease.¹⁴  The study procedures were reviewed and approved by the institutional review boards of the participating hospitals or the Western Institutional Review Board.

The primary independent variable was provider-documented up-to-date (UTD) vaccination status (yes or no) at the time of hospital presentation. This variable was abstracted from the electronic medical record (EMR) from either the ED or admission note. Although previous studies have revealed that provider-documented UTD status may not be reflective of true vaccination status,^6,15,16  we selected it as our independent variable because it represented the information the provider had at the point of care when making clinical decisions during the ED visit or hospitalization. All hospitals in the study had a specific vaccination question included in their EMR note templates for ED notes and admission history and physicals. Subjects were excluded if there was no provider documentation of vaccination status in the EMR. At the time of this study, none of the hospitals had vaccination data from population-based immunization information systems integrated into their EMRs. Thus, provider-documented status was presumed to be the only available information about vaccination status in the EMR. There was no uniform question specifically about influenza vaccines for all institutions. Whether influenza vaccination was included in being UTD depended on whether the provider documenting UTD status defined it as such.

We used the Pediatric Respiratory Illness Measurement System (PRIMES) to measure adherence to quality measures.¹⁴  PRIMES is a set of process-of-care quality indicators that can be used to assess the clinical management of children hospitalized with the following respiratory conditions: CAP, croup, asthma, and bronchiolitis.^14,17  Indicators were developed on the basis of a literature review and clinical practice guidelines and were validated by using a modified Delphi method.^14,18  We examined overall adherence to the condition-specific PRIMES indicators for each child (scale 0–100) and the overuse and underuse composite scores.¹⁹  Overuse composite scores included the quality indicators for processes of care that should not occur (eg, obtaining a CRP test in children admitted with uncomplicated CAP).²⁰  Underuse composite scores included quality indicators for processes of care that should occur (eg, giving a dose of dexamethasone to children presenting with croup). For all scores, higher scores indicated increased adherence to the indicators and higher quality care (Supplemental Table 4).

Each hospital in the study contributes data to the Pediatric Health Information System database. LOS, same-cause 30-day ED and/or inpatient reuse, and cost data were obtained from the Pediatric Health Information System database for each patient. All costs were inflation adjusted to 2016 dollars.

Covariates of interest included child age, sex, race and/or ethnicity, insurance, chronic disease status, admission to the ICU, and month of presentation. We categorized age into 4 groups (0–18 months, 19–35 months, 3–6 years, and >6 years) to correspond to the age categories typically used in population-level evaluations of vaccination coverage²¹  (although participants >6 years of age were aggregated because of the small sample size). Chronic disease status was determined by using the Pediatric Medical Complexity Algorithm (PMCA).²²  A chart review was performed for all subjects with complex chronic disease to ensure that they met study eligibility criteria. The month of presentation was included to account for seasonal trends and categorized into influenza season (October to March) and non–influenza season (April to September).²³ 

As an exploratory analysis, we examined the following additional covariates: (1) primarily speaking a language other than English at home (ie, limited English proficiency), (2) parent income level, (3) parent education level, and (4) parent-reported difficulty in accessing care. These variables were obtained from a parent-reported survey collected within 72 hours of admission. The difficulty in accessing care measure was from the 2009–2010 National Survey of Children with Special Health Care Needs.²⁴  Parents responded to the question, “In the last 6 months, did you have any difficulties or delays getting care for your child because there were waiting lists, backlogs, or other problems getting an appointment?” Responses were categorized as binary (yes or no). Further information on survey data collection has been previously published.¹⁹ 

Summary statistics were used to describe the cohort of subjects. We used χ² tests and bivariable logistic regression to assess the relationship between covariates and vaccination status, the relationship between covariates and the outcome variables, and the relationship between UTD status and our outcome variables. To test the independent association of vaccination status and the outcome variables, we used multivariable logistic regression models that included covariates (race and/or ethnicity, PMCA, and season of presentation) with a significant (P < .05) relationship with vaccination status and the outcome variables. Hospital site was included as a fixed effect in all multivariable models. We generated multivariable linear regression models to examine the relationship between vaccination status and (1) adherence to condition-specific PRIMES indicators, (2) overuse composite PRIMES scores, (3) underuse composite PRIMES scores, (4) LOS, and (5) cost. We examined results as the adjusted mean difference for continuous outcomes for children documented as not UTD compared with those documented as UTD. LOS and cost were truncated at the 99th percentile to prevent the skewed distribution from distorting SEs in the multivariable analyses. Observations above the 99th perentile were assigned the 99th percentile value; this occurred for 16 LOS observations and 24 cost observations. Models used to examine cost as the outcome were additionally adjusted for LOS. We generated multivariable logistic regression models to examine the relationship between vaccination status and 30-day same-cause readmission to the ED or inpatient setting.

Of 2380 children identified with ARTI, 46 (2%) were excluded for underlying conditions that would alter the routine childhood vaccination schedule and 32 (1%) had no documentation of vaccine status in their EMRs. Of the remaining 2302 participants included in the analysis, the mean age was 3.6 years (SD 3.7 years), 59% were boys, 40% were white, and 57% were publicly insured (Table 1). There were 568 (24%) diagnosed with CAP, 343 (15%) with croup, 653 (28%) with asthma, and 738 (32%) with bronchiolitis. Most (92%) were UTD on vaccinations by provider documentation. Vaccination rates by hospital ranged from 3% of subjects being documented as not UTD (hospitals 1 and 5) to 17% documents as not UTD (hospital 2) (Table 1). Race and/or ethnicity, PMCA, season of presentation, hospital, and diagnosis were all significantly associated with provider-documented UTD status (Table 1). We did not identify any significant associations among respondents to the parent-reported survey items (limited English proficiency, parent income, parent education level, and parent-reported difficulty in accessing care) with provider-documented vaccination status (Table 1).

Across all conditions, the adjusted mean difference in the overall PRIMES score by UTD status was not statistically significant (Table 2). However, there was a significant adjusted mean difference in overuse and underuse composite PRIMES scores by UTD status. Compared with children who were UTD, children not UTD had significantly lower mean overuse PRIMES composite scores and significantly higher mean underuse PRIMES composite scores.

In condition-specific adjusted models, there was no significant adjusted mean difference in the overall PRIMES score by UTD status. For children with CAP, however, children who were not UTD had significantly lower adjusted mean overuse composite PRIMES scores and significantly higher adjusted mean underuse composite PRIMES scores than children who were UTD (Table 2). For croup, children who were not UTD had significantly lower adjusted overuse composite PRIMES score compared with children who were UTD (Table 2). There were no differences in adherence to PRIMES quality indicators by vaccination status for children with asthma and bronchiolitis.

In adjusted models, we identified no significant differences in LOS in hours or costs for children who were documented as not UTD (versus UTD) (Table 3). There were no differences in readmissions for children who were not UTD compared with those who were documented as UTD.

In this large, multisite, prospective cohort of children with ARTI, we examined the association between adherence to evidence-based quality measures and provider-documented child vaccination status. We identified that children with ARTI who were documented at hospital presentation as not UTD had similar adherence to overall quality measures as children who were documented as UTD. However, when examining PRIMES scores across all 4 conditions, we identified differences in overuse and underuse composites by vaccination status, with lower adherence to overuse quality measures and higher adherence to underuse quality measures for children who were not UTD. Despite differential adherence to quality measures by vaccination status, we identified no significant differences in the LOS or cost for children who were undervaccinated and admitted with ARTI compared with children who were fully vaccinated.

Lower scores for overuse quality measures and higher scores for underuse quality measures may indicate a propensity for providers to “do more” for children who present to the hospital with ARTI and are documented as undervaccinated (ie, adhere less to quality indicators for processes of care that should not occur and adhere more to quality indicators for processes of care that should occur). This finding is consistent with previous work examining variation in testing and treatment by vaccination status for children presenting to the hospital with ARTI, in which children who were undervaccinated had a higher odds of receiving laboratory testing. Of note, differences in the overuse and underuse composite scores were only identified for CAP and croup. The overuse indicators for CAP and croup are largely related to diagnostic testing, including recommendations against getting routine blood cultures, CRP tests, and erythrocyte sedimentation rate (ESR) tests for CAP and chest radiographs for croup. We may have only seen differences in overuse and underuse indicators by vaccination status in CAP and croup (and not in asthma or bronchiolitis) because clinicians may face increased diagnostic uncertainty with these 2 conditions and therefore have to consider vaccine-preventable diseases in their differential diagnosis and subsequent medical decision-making. In further work in this area, researchers should consider examining variation in care by vaccination status for children with other potential vaccine-related illnesses and chief complaints.

It is also noteworthy that providers appeared to do more for children with ARTI who were undervaccinated not only when it was consistent with evidence-based recommendations (higher underuse scores) but also when it was not (lower overuse scores). In an era focused on deimplementation, safely doing less, and reducing low-value care,^25,26  it is important to consider the impact that low vaccination rates within a population can have on these efforts. Additionally, further work should be done to evaluate whether improving vaccination rates within the outpatient population can enhance efforts to safely do less in children who are hospitalized.

Our choice to use provider-documented vaccination status at the time of hospital presentation was meant to reflect the information available to the provider at the point of care. Provider-documented vaccination status is often inaccurate when compared with statewide vaccine registries, with most misclassifications in the direction of being documented as UTD when not truly UTD.⁶  It is unknown how documentation of vaccination status plays a role in clinical decision-making for children who present to the hospital with ARTI. With such a high percentage of children being documented as UTD in this study, there is a possibility that being documented as not UTD serves as a proxy for other important patient- and family-level factors with potential to influence patient care, such as health literacy. Understanding how patient- and family-level factors influence provider decision-making is key to providing high-quality care to children who present to the hospital with acute illnesses.

With the improvement in population-based immunization registries over the past decade, there has been a renewed focus on hospitalization as an opportunity to vaccinate.^4,15,27–29  Despite this, there are significant barriers to accurately identifying hospitalized children who are undervaccinated.^5,6,15,16  Hospital-based quality improvement efforts have revealed improvement in vaccination rates for high-risk children who are hospitalized.^4,30  Better integration of population-based immunization data into hospital EMRs may bolster efforts to provide catch-up vaccinations to children who are hospitalized. Consistent with the results of our study, however, improved recognition of true vaccination status may also result in hospital-based providers doing more for hospitalized children who are undervaccinated. Continued efforts should be focused on providing high-value care to children who are hospitalized and undervaccinated, including administering vaccinations when appropriate and clinically indicated.

In this study, we examined the association of documented vaccination status and adherence to quality indicators. Among variables that were measured, we were able to control for potential confounders of this association. However, we could not account for unmeasured variables, such as illness severity, that may confound the relationship between vaccination status and adherence to respiratory quality measures.¹⁹ 

Despite a large sample size, we had a low number of children who were not UTD by provider documentation, which limited the power of our study to detect differences. By categorizing UTD status as a binary yes or no variable, we were unable to evaluate the effect of missing specific vaccines related to ARTI. Lastly, all the hospitals in this study were academic children’s hospitals. They may not be representative of all care settings where children are hospitalized with ARTI, such as community-based settings.

We identified lower adherence to overuse quality indicators and higher adherence to underuse quality indicators among children who were not UTD, suggesting that clinicians do more for hospitalized children who are undervaccinated. The identified differences in adherence to quality measures by vaccination status were significant for children hospitalized with croup or CAP, whereas children with asthma and bronchiolitis had similar adherence regardless of vaccination status. Future efforts should be focused on improving vaccination rates for children who are hospitalized and examining how this relates to subsequent provision of high-value care in this high-risk population.