General Information
Conflict of Interest
Request Clinical Trials
Associated Trial(s):- NCT00216476 - CONSTATRE: Risperdal® Consta® Trial of Relapse Prevention and Effectiveness
- NCT00679627 - A Randomized, Double-Blind, Placebo-controlled Trial of Long-term (2-year) Treatment of Galantamine in Mild to Moderately-Severe Alzheimer's Disease
Request Clinical Trials
Data Request Status
Status: Unknown - RevokedResearch Proposal
Project Title: A novel statistical approach to detect the efficacy of drugs that have heterogeneous treatment effects
Scientific Abstract:
Background: The statistical tests currently used in clinical trials to prove drug efficacy are looking for a difference in mean between intervention groups. Therefore, these methods can be underpowered in a setting with heterogeneous treatment effects. We developed a novel statistical test that is better suited for these situations.
Objective: We aim to apply our test on several clinical trials to showcase its usefulness. The goal is to prove that we have better power at detecting efficacy compared to traditional approaches.
Study design: For each clinical trial studied, we will use our test to assess whether there is a significant difference between cases and controls in the main outcome measures. Our test is empowered to detect heterogeneous treatment effects.
Participants: All participants in the clinical trials requested.
Outcome measure: Power to detect efficacy (with small enough statistical p-value) as a function of the sample size in comparison with traditional approaches.
Statistical analysis: We will use our novel statistical test. (https://www.biorxiv.org/content/10.1101/2020.03.23.002972v2 publication under review)
Brief Project Background and Statement of Project Significance:
Full narrative summary (vivli), included here because of space constraint on the narrative summary section:
There is often a lot of variability among patients with the same diseases, this includes variability in disease stage, variability in the underlying disease causes and pathways, and variability in drug metabolism profiles. For all these reasons, a given drug might work in some patients and show no effect on many others. This is called a heterogeneous treatment effect.
Clinical trials are often conducted with the aim of proving the drug's efficacy. However, the statistical methods frequently used in these trials are looking for an average effect and are not specifically designed to detect a heterogeneous treatment effect scenario. Therefore these methods can sometimes be underpowered to detect drug efficacy.
We designed a novel statistical test for this scenario and showed that it has superior power compared to widely used statistical tests: https://www.biorxiv.org/content/10.1101/2020.03.23.002972v2
Better power means potentially the discovery of more beneficial drugs and/or the need for fewer participants and lower costs for future trials.
We aim to show the usefulness of our test in real data of clinical trials.
The novel statistical test can be useful for any setting with heterogeneous effects. In the future, it can be applied in other clinical trials as well as other
research fields such a biomarker discovery.
Brief description section on Vivli:
The trials we are interested in have a case-control design (such as drug-placebo) with a continuous outcome measure (such as a difference from baseline in a measure of disease severity). We will apply our statistical test to the data and assess the power to detect efficacy by sample size in comparison to traditional statistical approaches previously used. Our test looks at the distribution of the outcome variable in the cases and in the controls to see if there is a difference. More specifically, it detects an enrichment in cases (individuals who were given the drug rather than a placebo) among the individuals that had better outcomes. More details about our test can be found here: https://www.biorxiv.org/content/10.1101/2020.03.23.002972v2 In each trial considered, we will use all participants that we included in the original analysis: All cases and all controls. When there are more than two groups, we will do the same pairwise comparison as the original analysis (found in both clinicaltrials.gov and the associated publications).
Specific Aims of the Project:
We aim to show the performance of our statistical test on real clinical trials. We will use the data to demonstrate superior power to detect efficacy under heterogeneous treatment effects compared to traditional tests previously used in these trials such as ANCOVA, Wilcoxon, LMM, etc.
The power is the ability to detect the association between the intervention and the outcome with a significant p-value, given sample size.
The null hypothesis: There is no difference between the cases (individuals receiving the drug) and controls (placebo or standard care).
The alternative hypothesis: The distribution of the outcome is different between cases and controls, with a significant proportion of cases benefiting from the drug.
Study Design:
What is the purpose of the analysis being proposed? Please select all that apply.: Develop or refine statistical methods
Software Used: RStudio
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: No exclusion criteria
Primary and Secondary Outcome Measure(s) and how they will be categorized/defined for your study:
We will test the main outcomes of each study for association with the intervention using our test.
In each clinical trial, we will use the same main outcome as the original analysis (found in clinicaltrials.gov and the associated publications) provided the outcome is continuous. In studies having both discrete and continuous main outcomes, we will only test the continuous outcome.
The outcome of our analysis is an assessment of the power to detect the drug efficacy (using p-vaues) and a comparison with the traditional approaches that were used in the trial.
Main Predictor/Independent Variable and how it will be categorized/defined for your study:
In each clinical trial analyzed, the main predictor will be the Intervention (drug, placebo).
We are looking at a case-control setting. If there are more than two groups (for example multiple drug dosages), we will restrict the analysis to two groups at once, doing the same pairwise comparisons as the original analysis.
Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: We will correct for the exact same covariates (such as gender, age and others) as was done in the original analysis of each clinical trial (details were found in both clinicaltrials.gov and the associated clinical trial publication).
Statistical Analysis Plan:
We went through the list of trials on clinicaltrial.gov and curated trials that fit the following criteria: A large enough sample size, a continuous main outcome, and the potential for heterogeneous treatment effects (by looking at the summary statistics).
The reason for this selection is that we want trials where our test is applicable and where we can hope to have enough power to detect drug efficacy. We requested the clinical trials at the intersection of our list and the list of trials available on Vivli (NCT00428090, NCT00783692, NCT01255787, NCT00195663, NCT00321984, NCT00635219, NCT00672958, NCT02389816, NCT01313676, NCT00144339, NCT00299104, NCT00531817, NCT01262365, NCT01261793, NCT00216476, NCT00679627).
We will apply our novel statistical test on the main outcomes of each trial we are granted access to.
For example, for the NCT00679627 trial requested here on Yoda, we will apply our test on the main outcome variable which is the change from baseline in MMSE score (scores are on a scale from 0 to 30). Most secondary outcomes can also be considered continuous such as the change from baselines in the different subscales of MMSE and DAD scores, and we will use our test on them as a secondary analysis. For study NCT00216476 also requested on Yoda, given that the main outcome is a time-to-event variable, we will instead apply our test to the secondary outcomes that are continuous: change from baseline in PANSS, SF_12 and CGI scores.
Here is a reference to our test with more details: https://www.biorxiv.org/content/10.1101/2020.03.23.002972v2
In each study, we will correct for the same covariates and interactions (same model) as was done in the original analysis of the trial. We will handle missing data (imputation or exclusion) in the same way and using the same methods as the original analysis whenever possible.
We do not plan on doing an analysis of subgroups based on covariates.
We are following a case-control design. In scenarios with more than two groups (such as multiple drug dosages), we will consider the same groupings and we will do the same pairwise comparisons as the original analysis.
Our goal is to detect efficacy as a significant statistical difference in distribution between cases and controls. Our main result will be a p-value per outcome, corrected for multiple hypothesis testing.
We will run our analysis using Vivli's platform. All analyses will be performed by Aziz M. Mezlini.
Narrative Summary:
We developed a novel statistical test that is particularly empowered to detect drug efficacy in scenarios where the drug works well on a small proportion of individuals. We would like to apply our test to clinical trials to showcase its usefulness.
For the full narrative summary we used on Vivli, please refer to the section "Brief Project Background and Statement of Project Significance" on the research proposal. We were not able to add all the text here because of the constraint on the number of characters.
Project Timeline:
Once we are granted access to the data (anticipated in August 2020), we will run our analysis and get results within 3 months (anticipated November 2020).
We will prepare a publication and a report back to Yoda Project by February (2021).
We anticipate that the whole project will take less than 12 months from the data access is granted.
Dissemination Plan:
We intend to submit the manuscript to a wide audience journal with interest in Statistics, Computational biology, and clinical trials.
Depending on how good the results are, our first choice is currently Nature Methods but that is still open to change.
Bibliography:
https://www.biorxiv.org/content/10.1101/2020.03.23.002972v2