Vaccine Effectiveness Against Long COVID in Children

We conducted a retrospective cohort study based on EHR data from 41 health systems for children with any of (1) COVID-19, (2) a diagnosis of acute respiratory illness, fever, dyspnea, or cough after January 1, 2019, (3) SARS-CoV-2 testing, or (4) a SARS-CoV-2 vaccine. Data, extracted quarterly, were transformed to either the PCORnet (v6.0) or Observational Medical Outcomes Partnership (v5.3) data models and forwarded to the RECOVER-PCORnet Coordinating Center. For this study, we used the s8 version of the data, collected in April 2023, which comprises 25 389 893 patients.

Institutional Review Board oversight was provided by the Biomedical Research Alliance of New York, protocol # 21-08-508-380.

Patients were eligible if they had an in-person visit between vaccine eligibility and October 29, 2022 and were between 5 and <18 years at the date of vaccine eligibility. The date of SARS-CoV-2 vaccine eligibility was defined as January 1, 2021, for patients aged 12 to <18 years and October 29, 2021 for patients aged 5 to <12 years, corresponding to availability of vaccines. A small number of patients were excluded for documented COVID-19 before 28 days after their first vaccine dose to allow for full response to vaccine.

For vaccinated patients, cohort entry was the date of their first vaccine dose, whereas for unvaccinated patients, the date of an in-person visit after vaccine eligibility was selected randomly, resulting in matched distribution of entry dates between cohorts. In addition, we required one contact between 8 days and 2 years before cohort entry to assess baseline health characteristics; we did not require this to be an in-person visit to minimize bias toward higher utilizers. To determine long COVID outcomes, we also required at least 1 in-person visit between 28 and 179 days after cohort entry.

The main outcomes were diagnosed or probable long COVID. The case definition for diagnosed long COVID was two or more visits with diagnosis codes specific for long COVID; a single diagnosis code fell within the case definition for probable long COVID. In addition, to account for incomplete availability or use of long-COVID-specific diagnosis codes, especially for patients with early signs of long COVID, the case definition for probable long COVID included COVID-19 (SARS-CoV-2 polymerase chain reaction or antigen positive or COVID-19-specific diagnosis codes) plus at least two long-COVID-compatible diagnoses 28 to 179 days after infection, following the US Department of Health and Human Services definition.³⁵  These diagnoses were identified in prior work as post-acute associations with COVID-19^14,36  (Supplemental Table 5).

Some results are reported for preomicron and omicron eras. This applies only to adolescents, as vaccines for younger children were not available substantially before the omicron era. To maintain consistency across analyses, November 29, 2021, the first day a younger child could have had a qualifying postvaccine infection, was used as the dividing line between eras; at that point omicron was circulating in the United States and became the dominant strain a few weeks later. For each patient, the date of a SARS-CoV-2 infection after cohort entry, if one occurred, was used to place their results in either of the two eras; if the patient never developed COVID-19, the date of cohort entry was used.

The primary exposure was receipt of at least one dose of SARS-CoV-2 vaccine. Vaccine doses were ascertained using a vaccine code (CVX or RxNorm) designating an administered or patient-reported dose, a procedure code indicating SARS-CoV-2 vaccination, or a vaccine description containing the string “COVID” or “SARS.” Where available, data were retrieved from state and regional vaccine information systems via hospital operational procedures. Adequate vaccine data capture was defined as a site-reported vaccination rate at least 60% of the Centers for Disease Control and Prevention (CDC)-estimated rate²  for its coverage area based on addresses of patients seen during the pandemic. Where dose counts were used, all recorded doses within 14 days of a base dose were considered a single dose.

Although the likelihood that a recorded vaccine is incorrectly attributed to a patient is very low, the risk that dose count is incomplete in reported data are well-described,³⁷  and discontinuation before a second dose is uncommon. It is therefore more accurate in this study design to include patients with evidence of vaccination, and, because any compensatory bias introduced thereby will tend to reduce estimates, more responsible to report results that do not overestimate vaccine effectiveness. We analyzed all children with evidence of vaccination and no history of prior SARS-CoV-2 infection.

Age, sex, and ethnicity were taken from EHRs. Ethnicity was coded as Hispanic if EHR data so indicated, and otherwise as the race value in the EHR, based on the observation that race values are missing more often if a person indicates Hispanic ethnicity.³⁸  This combined variable was included in our analyses because prior work has shown an association between it and COVID-19 severity³  and both incidence of³⁹  and use of specific diagnosis codes for⁴⁰  long COVID, likely reflecting numerous medical and socioeconomic factors correlated with this administrative construct. For regression analyses, we recoded this variable using the 4 most populous categories plus a composite category for less frequent values. Comorbidity data used the body system taxonomy of the Pediatric Medical Complexity Algorithm (PMCA).⁴¹  Diagnoses received up to 3 years before cohort entry were considered. To classify patients as complex-chronic, chronic, or no chronic condition, the “more conservative” version of the algorithm was applied.

We matched vaccinated and unvaccinated patients on age group at cohort entry (5–11, 12–17) and time period (6-month cohort entry period). We limited long COVID observations to 12 months after cohort entry to account for waning of vaccine effectiveness (VE) over time.

To evaluate potential impact of attrition caused by matching, we performed a sensitivity analysis using inverse probability of treatment weighting that incorporated age group, time of cohort entry, sex, ethnicity, health system, number of baseline visits to the health system, PMCA-defined body system(s) affected by chronic illness, progressive disease, and malignancy. Further, we estimated VE after changing the window for outcomes to 6 months and 18 months post cohort entry and restricting the vaccinated cohort to those with at least 2 recorded doses.

For unadjusted analyses, follow-up time was computed from cohort entry until the end of observation. We calculated an incidence rate ratio (IRR) comparing vaccinated to unvaccinated groups and derived 95% confidence interval (CI)s using the estimation of Wald.⁴²  Corresponding VE was computed as 100 × (1 − IRR).

For adjusted analyses, we used conditional logistic regression to account for stratification. Covariates included sex, ethnicity, health system, baseline utilization (0, 1–5, 6–10, 11–24, 25–49, 50–99, or 100+ encounters in the 3 years before cohort entry), the number of PMCA body systems affected, and the presence of progressive or malignant condition. Since the prevalence of long COVID was low, IRR approximately equals OR, so the corresponding VE was computed as 100 × (1 − OR).

To understand the direct and indirect impact of vaccination on long COVID, we conducted causal mediation analyses, where the treatment was preinfection vaccination, the mediator was SARS-CoV-2 infection, and the outcome was long COVID. We fit two logistic regression models, the first of treatment on the mediator and the second of treatment and mediator on the outcome, adjusting for the same set of covariates. From these models, we estimated the effect and CIs of vaccination on long COVID independent of infection and of vaccination on long COVID mediated by infection.

Data analysis was performed using R¹⁷  version 4.0.2. Exact matching used the matchIt package⁴³  version 4.4.0, whereas inverse probability of treatment weighting (IPTW) were computed via twang⁴⁴  version 2.5. Conditional logistic regression used the clogit() algorithm implemented in the survival package⁴⁵  version 3.5. Significant effects were identified based on a threshold of p < .05.

Immunization data from 17 health systems was sufficiently robust for inclusion. Cohort formation is described in Fig 1: the 5 to 11-year-old group comprised 480 298 children, whereas the 12 to 17 year-old group comprised 557 638 children and adolescents. Overall, 67% received at least 1 SARS-CoV-2 vaccine, and 88% of vaccinated children received 2 or more doses. Compared with unvaccinated children, vaccinated patients were older, had lower baseline healthcare utilization, and fewer chronic conditions. Girls and children who identified as Asian or Hispanic were more common in the vaccinated group, whereas boys and those identifying as Black or White were less common (Table 1). Descriptive statistics for the all-ages matched cohorts are provided in Supplemental Table 3. The prevalence of probable long COVID was 0.3% in the cohort overall; for children with COVID-19 after cohort entry, prevalence of diagnosed long COVID was 0.8%, and adding probable cases raised the prevalence of long COVID to 4.5%.

For children with evidence of vaccination and no history of prior SARS-CoV-2 infection, VE was 35% (95% CI 25%–45%) for the combined age groups (Table 2), with greater effect in adolescents than younger children. For children known to be completely vaccinated (ie, 2+ recorded doses), VE was 45% (95% CI 35%–53%) against probable long COVID within 12 months (Fig 2), with consistent findings within age strata, supporting the validity of our approach. For comparison, VE estimates against SARS-CoV-2 infection are shown in Supplemental Fig 4. We observed decreased likelihood of long COVID among boys and children with non-Hispanic Black or Asian ethnicity (Supplemental Table 4). The presence of chronic conditions had minimal effect, though likelihood of long COVID rose with increasing baseline utilization.

The protective effect of vaccination appeared to wane over time. We observed higher VE against long COVID within 6 months of vaccination than 12 months, whereas extending the observation period to 18 months revealed a further diminution (Fig 3). Protection was also seen in children who started vaccination after recuperating from COVID-19 (Supplemental Fig 5).

One-to-one matching resulted in loss of 27% of unvaccinated and 64% of vaccinated patients because of the high overall vaccination rate. To assess the effect of this on our VE estimates, we used IPTW to adjust for multiple potential confounders without decreasing cohort size. Once again, a protective effect was seen, though with a smaller magnitude of 24% (95% CI 20%–28%) for vaccination on long COVID up to a year later (Supplemental Fig 6).

To assess where in the path to long COVID the vaccine might have impact, we performed mediation analysis, estimating effect of vaccine on long COVID via change in COVID-19 risk (indirect effect) separately from other effects on long COVID (direct effect). For 5 to 11-year-olds, who became vaccine eligible just before the omicron variant became prevalent, the long COVID-associated OR for indirect effect on probable long COVID was 0.67 (95% CI 0.64–0.70), whereas for direct effect the OR was 1.34 (95% CI 1.13–1.59). For adolescents, results were similar: in the preomicron era, the indirect effect OR was 0.46 (95% CI 0.45–0.48) and direct effect OR was 1.32 (95% CI 1.12–1.55); in the omicron era, the analogous ORs were 0.45 (95% CI 0.41–0.49) and 0.99 (95% CI 0.73–1.34). These results suggest that throughout the period of pediatric use, the effectiveness of vaccine against long COVID is closely tied to its effectiveness against the antecedent reported COVID-19 episode.

To our knowledge, this national-scale study, with a diverse cohort of over one million children, is the first to estimate the VE against long COVID in children. Our findings suggest that the SARS-CoV-2 vaccine is associated with reduced risk of long COVID in children and adolescents. The protective effect was greater among adolescents than younger children, reflecting activity against delta and omicron strains, whereas use for younger children is limited to the omicron era. Since diagnostic codes for long COVID (eg, ICD-10-CM U09.9) are rare in children, we have examined both specific diagnosis as an outcome and a broader definition based on the repeated presence from 1 to 6 months after infection of symptoms that occur more frequently after COVID-19 than in uninfected children.^14,36  The effect of vaccine is similar for both outcomes and is also seen in children vaccinated after an initial episode of COVID-19, suggesting that vaccination confers some benefit against future late effects. Mediation analysis suggests that the protective effect of vaccination may be primarily via reduction in the occurrence or severity of infection.

Our finding of a protective effect against long COVID in children are consistent with those observed in adults, whereas the VE estimates in our study are lower than studies assessing the short-term effect of vaccines in one week to one month after vaccination.³¹  Other reasons are varied definitions of long COVID between these studies and different methodological approaches. Our observed higher VE for adolescents is similar to studies examining acute infection.⁴⁶  Further analyses are required to explore whether other factors contribute to this difference, such as long COVID symptoms being more frequently reported or documented in adolescents, or attributed to other causes in younger children, such as common childhood infections.

Waning effects of vaccine against long COVID are to be expected,⁴⁷  similar to studies against acute infection.^34,48,49  The reasons include reduced neutralization efficiency against newer strains and waning of pre-existing antibodies over time.⁵⁰  It is also possible that long COVID starting several months after acute infection, or delayed diagnosis or recognition of long COVID, may contribute to this phenomenon. Further, a waning effect may reflect the impact of successive infections rather than late-onset long COVID. This is difficult to assess analytically, given fewer COVID-19 episodes are being documented in the EHR during the post omicron era. Evaluations of VE over time are needed to address whether these waning effects can be overcome through booster or annual vaccine doses.

VE studies using EHR data rely on accurate capture of clinical and vaccine data, with adjustment for important confounders to minimize potential bias.^51,52  Our focus on data quality, including comparison with the CDC’s county-level vaccination data, reduced our cohort size but yielded more reliable vaccination data. Our assessment of VE against acute SARS-CoV-2 was similar to VE estimates from other pediatric studies, further validating our findings and approach. Finally, we performed matching on age and time period to limit secular effects and conducted sensitivity analyses including inverse probability of treatment weighting, with similar findings.

Nonetheless, several challenges exist with the identification of long COVID in children using EHR data, which may confound assessment of our outcome. There has been low uptake of the long COVID (ICD-10-CM U09.9) diagnosis code in pediatrics; it is more often seen in older children and associated with respiratory, cardiovascular, or fatigue subtypes of long COVID.³⁶  For this reason, we supplemented our outcome definition with a symptom-based computable phenotype definition of long COVID we have developed, which includes features related to long COVID based on statistical association and chart review validation.^14,36  However, long COVID presentations are heterogeneous, and symptoms and conditions may overlap with other childhood illnesses. Conversely, in some cases long COVID may manifest as changes in pre-existing or common health conditions rather than unique new symptoms. Both of these carry the potential for misclassification of long COVID in our study. These limitations on classification accuracy are expected to affect both vaccinated and unvaccinated cohorts similarly, so they would not confound the comparison of the two.

There are other limitations that warrant discussion. First, our study relies on secondary use of EHR data and is subject to bias because of differential access to and utilization of healthcare during the pandemic, clinical practice variation across sites, and language or cultural barriers to effective communication in a health care setting. This may affect both opportunity to vaccinate and recording of diagnoses consistent with long COVID. We required both pre and postvaccine visits to establish a baseline of utilization, which may create bias toward sicker patients, but, importantly, we did not observe major imbalances in utilization rates between cohorts. Next, testing practices changed over time, with an increased reliance on home antigen testing in later phases of the pandemic, potentially leading to misclassification of symptom based long COVID. In addition, a large proportion of patients in our data with a long COVID diagnosis code do not have documentation in the EHR of the antecedent COVID-19 episode, which prevented use of severity as an adjustment to vaccine effect.

Our results provide substantial evidence in a large and diverse cohort of children receiving health care for protective effect of vaccination in children 5 years and older. This study adds to the growing body of knowledge about the mitigating effect of vaccines on COVID-19 while demonstrating the need for further research utilizing a range of designs to examine the protective effect of these vaccines against subsequent strains and to help guide vaccine policy.

We are grateful to the health systems, clinicians, and patients at member sites without whose trust research of this type would not be possible; the clinical data teams at each RECOVER site involved in data extraction and integration, as well as colleagues at the RECOVER Coordinating Center, particularly Susan Hague, for maintaining the RECOVER data infrastructure; and the National Community Engagement Group (NCEG), all patient, caregiver and community Representatives, and all the participants enrolled in the RECOVER Initiative.

CI

95% confidence intervals

EHR

electronic health record

ICD-10-CM

International Classification of Diseases, version 10, clinical modification

PASC

post-acute sequelae of coronavirus disease 2019

PMCA

Pediatric Medical Complexity Algorithm

VE

vaccine effectiveness