General Information
Conflict of Interest
Request Clinical Trials
Associated Trial(s): What type of data are you looking for?: Individual Participant-Level Data, which includes Full CSR and all supporting documentationRequest Clinical Trials
Data Request Status
Status: OngoingResearch Proposal
Project Title: Vaccine efficacy against negative health outcomes after infection
Scientific Abstract:
Background: Typical VE analyses infer total vaccine efficacy against post-infection outcomes like severe COVID-19, but researchers have been unable to partition the benefits of vaccination into vaccine efficacy against infection and vaccine efficacy against severe illness given infection because the latter is not generally identifiable.
Objective: We will apply statistical methodology developed in Trangucci, Chen, and Zelner 2022 to the individual-level patient data from the ENSEMBLE 2 trial in order to test whether Ad26.COV2.S reduces the risk of negative post-SARS-CoV-2 infection outcomes.
Study Design: The study design is the double-blind phase of the ENSEMBLE 2 RCT. The intervention is assignment to vaccination or placebo treatment arms, and the study measures outcomes in the 2 months following receipt of the first vaccination dose (or placebo dose).
Participants: The participants in the ENSEMBLE 2 study that are eligible for inclusion in our analysis are those who received at least one dose of the vaccine or of the placebo.
Primary and Secondary Outcome Measure(s): Binary SARS-CoV-2 infection status, binary moderate-to-severe COVID-19, and medically attended illness, secondary measures are time-to-event versions of these events.
Statistical analysis: We will use an extension of the principal stratification methodology proposed in Hudgens and Halloran, 2006, that we developed in Trangucci, Chen, and Zelner, 2022. Our extension allows for measurement errors in all primary outcomes, as well as nonmonotonicity in VE against infection; both extensions are novel.
Brief Project Background and Statement of Project Significance:
We developed the statistical methodology for VE against post-infection outcomes in order to fill a gap in the literature, namely sound causal inference methods that could handle the complexities of real-world clinical trial data in VE studies. Our method has solid theoretical grounding, and our manuscript investigates the finite-sample properties of our method via extensive simulation studies. We aim to demonstrate the usefulness of our method, as well as to answer the question as to whether vaccines mitigate severe outcomes in those who are infected. This is not a straightforward analysis; naive analyses that compare severe illness in infected individuals by vaccination status risks bias because those who are infected and vaccinated may have systematic differences vs. the infected and unvaccinated. The direction of this bias likely understates the efficacy of the vaccine, though it could also plausibly overstate the efficacy of the vaccine.
More generally, VE studies may include subgroup analyses which use post-randomization outcomes in order to subset the data; in the ENSEMBLE 2 analysis, for example, Hardt et al. 2022 investigate VE against moderate-to-severe COVID-19 in participants who have received two vaccine doses (or two placebo doses). This analysis necessarily conditions on the post-randomization event of remaining uninfected between doses 1 and 2. Subsetting the data in this way risks biasing the conclusions. The method of dealing with these comparison is called principal stratification, developed in Frangakis and Rubin, 2002, but these methods are insufficient for use in VE trials because they do not allow for measurement error in the intermediate outcome, and they assume that individual-level VE against infection cannot be negative. Neither of these is likely to hold in real-world VE trials. Our method relaxes both of these assumptions; more detail is included in the statistical analysis section.
Ideally, after demonstrating how our method can be used with real-world data, our method will be useful in future VE research and study design when causal effects stratified by post-randomization outcomes related to infection are of interest.
Specific Aims of the Project:
The aim of the research is to use the design of the ENSEMBLE 2 clinical trial in order to determine the efficacy of Ad26.COV2.S against severe illness given infection, which we measure as seroconversion.
We will apply the new statistical method introduced in Trangucci, Chen, and Zelner, 2022 to do so. The estimands of interest, defined below, can be properly defined only in the subgroup that would be infected regardless of vaccination status, also known as the Always-Infected group (Hudgens and Halloran, 2006).
We will test the following hypotheses:
- Ad26.COV2.S reduces the risk of COVID-19 requiring medical intervention at least 14 days after vaccination in Always-Infected patients
- Ad26.COV2.S reduces the risk of moderate-to-severe COVID-19 measured at least 14 days after vaccination in Always-Infected patients
Study Design: Methodological research
What is the purpose of the analysis being proposed? Please select all that apply.: New research question to examine treatment effectiveness on secondary endpoints and/or within subgroup populations Develop or refine statistical methods
Software Used: R
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: The analysis set will be the all participants who received at least one dose of the placebo or vaccine. Thus the sole exclusion criterion is not receiving the assigned treatment.
Primary and Secondary Outcome Measure(s) and how they will be categorized/defined for your study:
Primary outcome measures: Binary SARS-CoV-2 infection prior to unblinding visit ascertained via RT-PCR or immunoassay, Binary medical intervention at least 14 days after vaccination, Binary moderate-to-severe COVID-19 symptoms at least 14 days after vaccination, viral load of SARS-CoV-2.
Secondary outcomes: Time to SARS-CoV-2 infection prior to unblinding visit ascertained via RT-PCR or immunoassay, time to medical intervention at least 14 days after vaccination, time to moderate-to-severe COVID-19 symptoms at least 14 days after vaccination
Main Predictor/Independent Variable and how it will be categorized/defined for your study:
Main independent variables: Receipt of Ad26.COV2.S vaccine.
Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: Study site membership ID, country of residence, ELISA-measured serostatus at baseline (measured in EU/mL), age group (18-<40, 40-<=59, 60-<=69, 70-<=79, >=80 years), sex, baseline comorbidity category (None, One, Two, 3 or more).
Statistical Analysis Plan:
We will use a novel statistical method developed in Trangucci, Chen, and Zelner, 2022 to infer VE against infection, VE against moderate-to-severe COVID-19, and VE against COVID-19 requiring medical intervention. The method is a modification of principal stratification (Frangakis and Rubin, 2002, Hudgens and Halloran, 2006) using Bayesian statistical methods implemented in Stan (Carpenter et al. 2017). Principal stratification is a statistically principled way of defining a causal effect when there is an event that occurs post-randomization that impinges upon the ultimate event of interest. Examples include estimating the effect of new chemotherapy vs. standard of care on quality-adjusted life-years when some patients die, or estimating the effect of a vaccine on severe illness when some patients are not infected. The solution that principal stratification employs is to limit comparisons to patients that are alike in their intermediate outcomes. For example, one compares quality adjusted life years only in the group that would survive no matter the treatment assignment. In VE trials, this means comparing severe illness outcomes only in the group of patients who would be infected no matter their vaccination status. While the comparisons eliminate selection bias, the estimands are typically not identifiable because of the "fundamental problem of causal inference" (Holland, 1986), namely that, of course, for each patient we see only the outcome corresponding to the treatment assignment.
Despite this fundamental limitation of principal stratification, our method uses the design of vaccine efficacy trials to identify the causal estimand of interest, namely the VE for Ad26.COV2.S on negative post-infection outcomes in the Always-Infected patients. Our method takes advantage of two factors that are common to many VE trials: (a) trials are run across many different health centers, and (b) trials measure baseline covariates that are plausibly imperfect observations of the principal stratum, which is a latent variable governing the infection outcomes for each individual under treatment and control. In ENSEMBLE 2, the study measured type-N antibodies at baseline using ELISA to determine if participants had had prior exposure to COVID-19. Our method uses the variation in rates of infection across centers and variation in antibody levels to learn the distribution of principal strata by study site. Given these distributions, we can infer the VE against the post-infection outcome.
The novelty of our method lies in the fact that we allow for measurement error of both infection, and post-infection outcome with unknown sensitivity and specificities. We also show that with a mismeasured antibody level, the estimand of interest is still identifiable. Furthermore, we do not assume that VE against infection is nonnegative for all participants. Both of these extensions, as well as our identifiability results, improve upon existing methods in the literature for principal stratification in multisite randomized trials (Linbo, Richardson, and Zhou, 2017, Lo-Hua, Feller, and Miratrix, 2019, Luo, Li, and He, 2023). Much of the prior statistical work has occurred in the setting of truncation by death, which is mathematically analogous to vaccine efficacy for post-infection outcomes. The difference is that there is typically very little measurement error in observations of death, which plays the role of the intermediate outcome, compared to infection, which is the intermediate outcome in VE studies.
We will also extend this analysis to measure viral load of SARS-CoV-2, as well as an extension to interval-censored time-to-event outcomes.
Narrative Summary: We will use the Phase 3 clinical trial data to ascertain whether there is a protective benefit from the vaccine against negative health outcomes, like severe COVID-19 or medically attended illness among those who were infected with SARS-CoV-2. Previous analysis has focused on inferring total vaccine efficacy (VE) against post-infection outcomes like severe COVID-19, but researchers have been unable to partition the benefits of vaccination into VE against infection and VE against severe illness given infection. We have developed a novel statistical method that can partition the effects of vaccination into VE against infection, and VE against post-infection outcomes in always-infected patients
Project Timeline: The project will start in September 2024, with primary data analysis being completed by October 2024. Initial publication will be submitted by November 2024. Extension of the statistical methodology to the continuous primary outcome viral load will begin alongside the primary data analysis with a planned completion of January 2025 with a second publication submitted by June 2025. Further extensions to interval-censored outcomes will begin in March 2025, with third publication submission in August 2025.
Dissemination Plan:
The research product primarily targeted at statisticians will be the statistical methodology paper, Trangucci, Chen and Zelner, 2022, with a target journal of Journal of the American Statistical Association, Theory and Methods, or Journal of the Royal Statistical Society, Series B, or Biometrics. A second research product, the extension of the previous research methodology to continuous secondary outcomes will target an epidemiology audience with potential journals being the American Journal of Epidemiology and Epidemiology. The third research product will be targeted at statisticians in the health field and the target journal will be Statistical Methods in Medical Research.
Along the way, I will present the research findings at scientific meetings like the American Causal Inference Conference in May of 2025, the 2025 Western North American Region of the International Biometric Society meeting, and the 2025 Society for Epidemiologic Research meeting.
Bibliography:
Carpenter, Bob, et al. “Stan: A probabilistic programming language.” Journal of statistical software 76 (2017).
Frangakis, Constantine E., and Donald B. Rubin. “Principal stratification in causal inference.” Biometrics 58.1 (2002): 21-29.
Hardt, Karin, et al. “Efficacy, safety, and immunogenicity of a booster regimen of Ad26. COV2. S vaccine against COVID-19 (ENSEMBLE2): results of a randomised, double-blind, placebo-controlled, phase 3 trial.” The Lancet Infectious Diseases 22.12 (2022): 1703-1715.
Holland, Paul W. “Statistics and causal inference.” Journal of the American statistical Association 81.396 (1986): 945-960.
Hudgens, Michael G., and M. Elizabeth Halloran. “Causal vaccine effects on binary postinfection outcomes.” Journal of the American Statistical Association 101.473 (2006): 51-64.
Luo, Shanshan, Wei Li, and Yangbo He. “Causal inference with outcomes truncated by death in multiarm studies.” Biometrics 79.1 (2023): 502-513.
Trangucci, Rob, Yang Chen, and Jon Zelner. “Identified vaccine efficacy for binary post-infection outcomes under misclassification without monotonicity.” arXiv preprint arXiv:2211.16502 (2022).
Wang, Linbo, Thomas S. Richardson, and Xiao-Hua Zhou. “Causal analysis of ordinal treatments and binary outcomes under truncation by death.” Journal of the Royal Statistical Society Series B: Statistical Methodology 79.3 (2017): 719-735.
Yuan, Lo-Hua, Avi Feller, and Luke W. Miratrix. “Identifying and estimating principal causal effects in a multi-site trial of Early College High Schools.” The Annals of Applied Statistics 13.3 (2019): 1348-1369.