Conflict of Interest
Request Clinical TrialsAssociated Trial(s):
- NCT00638690 - A Phase 3, Randomized, Double-Blind, Placebo-Controlled Study of Abiraterone Acetate (CB7630) Plus Prednisone in Patients With Metastatic Castration-Resistant Prostate Cancer Who Have Failed Docetaxel-Based Chemotherapy
Request Clinical Trials
Data Request StatusStatus: Ongoing
Project Title: An evaluation of the performance of two population-adjusted indirect comparison methods in anchored and unanchored settings
Background: The role of population-adjusted indirect comparison methods in health technology assessments (HTA), both in anchored and unanchored settings, has been a controversial subject. However, these methods are becoming increasingly popular in HTA submissions where there is limited trial data to estimate a treatment effect of interest.
Objective: This research aims to quantify the degree of uncertainty and possible bias that may be associated with performing a population-adjusted indirect comparison in a setting where a reliable estimate of the actual treatment effect can be made. The comparison of abiraterone versus enzalutamide in metastatic castration resistant prostate cancer will be used as the motivating example in this research. This research will introduce a new variation of a marginal model as an alternative to fitting a matching adjusted indirect comparison (MAIC).
Study Design: Individual patient-level data (IPD) from the COU-AA-301 trial and summary level data from the AFFIRM trial, both conducted in patients with metastatic castration resistant prostate cancer with prior chemotherapy, will be used to create two scenarios; one if COU-AA-301 and AFFIRM trials were the only evidence available, and the other if COU-AA-301 had been a single-arm trial. The results will be compared to those from a network meta-analysis (NMA). Sensitivity analyses will be conducted with imputation of variables that were not reported in the same way or were not reported from the AFFIRM trial.
Participants: The intent-to-treat sample for IPD from the COU-AA-301 trial and trial-level data and re-constructed patient-level data from the AFFIRM trial will be used in this research. Data will include time-to-event data for overall survival and demographic and baseline patient characteristic data.
Primary Outcome Measure: Hazard ratios for overall survival will be estimated from population-adjusted indirect comparison methods where all variables in common from both studies are included and sensitivity analyses conducted for missing or absent data.
Statistical Analysis: Population-adjusted indirect comparison methods require all important variables to be included in these analyses. Random forest for survival data and Cox regression models will be used to determine important prognostic effects and investigate treatment-effect modifiers. The population-adjusted indirect comparison methods will include MAIC and multiple imputation marginalization. These will be conducted in anchored and unanchored settings and an NMA used to estimate a hazard ratio for abiraterone versus enzalutamide. These hazard ratios will be compared to those estimated from an NMA fitted to trial-level data from a network of evidence.
Brief Project Background and Statement of Project Significance:
In many health technology assessment (HTA) settings, there are no head-to-head trials comparing the interventions of interest and there may not be a common comparator in trials with the intervention/s of interest. Population adjustment methods depend on having individual patient-level data (IPD) in 1 or more trials. However, typically IPD are only available for 1 intervention, i.e., from the submitting company's own trial.
The NICE Decision Support Unit has published guidelines for population adjustment with limited access to IPD (Phillippo et al., 2016). The guidelines describe how to conduct matching-adjusted indirect comparisons (MAIC) in anchored and unanchored settings and simulated treatment comparison (STC) in an anchored setting. However, Swallow et al. (2015) reported bias with STC when used with binomial data such as hazard ratios. Remiro-Azocar et al. (2021) argue that the typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. Remiro-Azocar et al. (2022) describe how to fit a Bayesian marginalized model for binomial data which they referred to as multiple imputation marginalization (MIM). However, this method has not yet been extended to time-to-event data.
Questions remain about the application of population adjustment methods and their validity in HTA, particularly when applied to single-arm studies. The results from population adjusted indirect comparisons are typically not verifiable due to the actual treatment effect being unknown. Phillippo et al., (2016) argue that further research is needed to assess how population adjustment methods perform under a range of scenarios.
This research introduces a new approach to fitting a MIM for anchored and unanchored population adjusted indirect comparisons for time-to-event data using a series of frequentist analysis steps. This research will compare the results from the new method to those from an MAIC. The data used will be from actual trial data where there is sufficient evidence available to estimate a reliable treatment effect.
This is expected to be the one of the first studies to compare the results from population adjustment methods to those with a reliable estimate of the treatment effect based on real data. This study will assess the uncertainty caused by missing prognostic data or data reported in a different way compared to the study with the IPD. This will be assessed in both anchored and unanchored settings.
This research will provide guidance for whether population adjusted indirect comparisons are suitable for HTA and make recommendations on their application and what sensitivity analyses are needed when variables are not reported in the same way or are missing. It is expected this research will provide guidance on how baseline patient characteristics should be reported from clinical trials so that they can be used by future studies.
Specific Aims of the Project:
Evaluate the performance of two population-adjusted indirect comparison methods, matching-adjusted indirect comparison (MAIC), and multiple imputation marginalization (MIM), in anchored and unanchored settings
To describe a new method to perform multiple imputation marginalization (MIM) with time-to-event data for anchored and unanchored indirect treatment comparisons.
To assess the performance of this model compared to conducting a matching-adjusted indirect comparisons (MAIC) using real data where a reliable estimate of the treatment effect can be estimated.
To assess how sensitive model predictions are to patient baseline data not being reported in the same way or absent from the study with the trial-level data.
To make recommendations on how suitable population-adjusted indirect comparison methods are for health technology assessments in anchored and unanchored settings and what sensitivity analyses should be performed for a HTA submission.
Study Design: Methodological research
What is the purpose of the analysis being proposed? Please select all that apply.: Develop or refine statistical methods
Software Used: R
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: The intent-to-treat sample will be used from the COU-AA-301 trial for patients with metastatic castration resistant prostate cancer who have received at least 1 prior chemotherapy.
Primary and Secondary Outcome Measure(s) and how they will be categorized/defined for your study:
The primary outcome will be hazard ratios for overall survival for abiraterone versus enzalutamide. These will be derived from a network meta-analysis (NMA) which will include trial-level data from AFFIRM (enzalutamide versus placebo). The data from COU-AA-301 will be used to estimate unadjusted hazard ratios from Cox regression models and population adjusted hazard ratios. Hazard ratios will also be estimated for abiraterone versus enzalutamide directly using unanchored indirect methods.
The results will be compared to those from a NMA fitted to trials conducted in prostate cancer. The NMA will only contain trials with the following comparisons: abiraterone versus placebo, enzalutamide versus placebo and abiraterone versus enzalutamide. Preliminary research has identified 11 suitable trials in 4 different patient populations, where each population contains at least 1 study comparing abiraterone versus placebo and enzalutamide versus placebo. The network of evidence also includes 1 head-to-head comparison for abiraterone versus enzalutamide.
Main Predictor/Independent Variable and how it will be categorized/defined for your study:
All important prognostic variables will be used in this study. This will be for values at the start of treatment i.e. demographics and baseline disease characteristics. This is expected to include the following list of independent variables:
Age in years
Ethnicity ? White, Asian, other
Time since diagnosis (years)
Disease location - bone
Disease location - node
Disease location ? liver
No. of bone lesions
BPI-SF score for pain
Total Gleason score
Progression at study entry (PSA only vs. radiographic)
No. of previous hormonal treatments
No. of previous chemotherapy regimens
ECOG ? 0
ECOG ? 1
ECOG ? 2
Prostate-specific antigen (ng/ml)
Alkaline phosphatase (?g/dL)
This list is based on the prognostic effects identified by Armstrong et al. (2018) and those referred to by de Bono et al. (2011) and Scher et al. (2012).
Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: Other variables may be included depending on information in the clinical study report.
Statistical Analysis Plan:
Determine a hazard ratio for abiraterone versus enzalutamide
A network meta-analyses (NMA) which will only include abiraterone, enzalutamide, and placebo will be used to estimate a hazard ratio for abiraterone versus enzalutamide.
Previous NMAs, which included abiraterone and enzalutamide, have been published in the following populations:
Metastatic, castration resistant, prior chemotherapy: Chen et al. (2022)
Metastatic, castration resistant, chemotherapy nave: McCool et al. (2018)
Metastatic, castration sensitive, chemotherapy nave: Wang et al. (2021)
One additional study has been identified in the metastatic, castration resistant, chemotherapy nave population, which was a head-to-head trial by Izumi et al. (2022). For each trial the publication with the most suitable data will be found i.e., the longest follow-up prior to any treatment switching.
The data will be included in a single NMA, which will include a hierarchical exchangeable structure for population. A random-effects NMA will be conducted with informative priors for the heterogeneity distribution. The predicted hazard ratio for abiraterone versus enzalutamide in the metastatic, castration resistant, prior chemotherapy will be used to make comparison with those from population adjustment methods.
Determine prognostic effects
Random forest for survival data will be used to investigate the importance of prognostic variables, provide evidence for the shape of relationships, and search for interactions. This will be conducted using the randomForestSRC package (Ishwaran, 2022) in R. A Cox model will be fitted based on information from the random forest analyses.
Population adjustment methods
Matching adjusted indirect comparison
Data from important covariates will be used to weight the patient data from COU-AA-301 to match the population in the AFFIRRM trial. This will be conducted using the maic package in R, which follows the method described by Phillippo et al. (2016).
Sensitivity analyses will be conducted for missing data from the AFFIRM trial. This will include variables not reported or where they were not reported in the same way.
Multiple imputation marginalization
The method will use the following steps:
1) Generation of synthetic population datasets for the external data (multiple imputation)
Simulated covariate data sets that match the AFFIRM population will be generated using a multivariate mixed normal and binomial distribution using the method described by Fialkowski (2018). For the anchored analysis this will include a treatment effect. For the unanchored analyses models will only be fitted to the abiraterone data.
2) Marginal Model
A range of parametric survival models with covariates for different distributions will be fitted to the individual patient-level data (IPD) from COU-AA-301, and parameters simulated from these models.
Survival data will be simulated for the synthetic populations using the simulated model parameters following the method described by Brilleman et al. (2021) for a range of distributions.
Data will be re-censored by resampling from the original distribution from the IPD censored timepoints from COU-AA-301.
For each parametric model, and each data set, stratified Cox models will be used to predict the probability of survival for a sequence of timepoints. This will be used to produce Kaplan-Meier style charts. Cox models will be used to estimate hazard ratios for each simulated data set. Credible intervals will be derived by averaging over the confidence intervals across the results from each simulated data set for each model.
For the anchored analyses, the MAIC and MIM will produce adjusted hazard ratios for abiraterone versus placebo. These results will be used in an NMA to estimate the hazard ratio for abiraterone versus enzalutamide.
For the unanchored analyses, the hazard ratios will be for abiraterone versus enzalutamide.
Narrative Summary: This research aims to investigate the uncertainty and possible bias that may be associated with performing anchored and unanchored population-adjusted indirect comparisons in a setting where a reliable estimate of the actual treatment effect can be made. Individual patient-level data from COU-AA-301 and summary data from AFFIRM will be used to create data for the following scenarios: (1) COU-AA-301 and AFFIRM trials were the only available evidence (anchored indirect comparison) and (2) COU-AA-301 had been a single-arm trial (unanchored indirect comparison). This research will also investigate the impact of explanatory variables being reported in different ways or not reported.
Project start date: November 2022
Analysis completion date: June 2023
Date manuscript drafted and first submitted for publication: September 2023
Date results reported back to the YODA Project: August 2023
Products: Poster/presentation and manuscript
Research presentation at ISPOR Europe 2023: Poster or podium presentation
Manuscript to be submitted to Medical Decision Making, Value in Health, or BMC Medical Research Methodology.
Researchers involved in health technology assessments: pharmaceutical companies, agencies that support health technology assessments, and academics
Armstrong AJ, Lin P, Higano CS, Sternberg CN, Sonpavde G, Tombal B, et al. Development and validation of a prognostic model for overall survival in chemotherapy-naive men with metastatic castration-resistant prostate cancer. Annals of Oncology. 2018; 29: 2200?7.
Brilleman SL, Wolfe R, Moreno-Betancur M, Crowther MJ. Simulating survival data using the simsurv R package. J Stat Soft. 2021; 97(2):1-27.
Chen J, Zhang Y, Zhang X, Zhao J, Ni Y, Zhu S, et al. Comparison of systemic treatments for metastatic castration-resistant prostate cancer after docetaxel failure: A systematic review and network meta-analysis. Frontiers in Pharmacology. 2022; 12, Article 789319.
de Bono JS, Logothetis CJ, Molina A, Fizazi K, North S, Chu L, et al. Abiraterone and increased survival in metastatic prostate cancer. N Engl J Med. 2011; 364(21): 1995-2005.
Fialkowski AC. Simulation of correlated data with multiple variable types. SimMultiCorrData in R. 2018. Available at https://cran.r-project.org/web/packages/SimMultiCorrData/. Accessed 30 August 2022.
Ishwaran H, Kogalur, UB. Fast unified random forests for survival, regression, and classification (RF-SRC). randomForestSRC in R. 2022. Available at https://cran.r-project.org/web/packages/randomForestSRC/. Accessed 19 October 2022.
Izumi K, Shima T, Mita K, Kato Y, Kamiyama M, Inoue S, et al. Enzalutamide versus abiraterone plus prednisolone before chemotherapy for castration-resistant prostate Cancer: A multicenter randomized controlled trial. Europen Urology Open Science. 2022; 41: 16-23.
McCool R, Fleetwood K, Glanville J, Arber M, Goodall H, Naidoo S. Systematic review and network meta-analysis of treatments for chemotherapy-nave patients with asymptomatic/mildly symptomatic metastatic castration-resistant prostate cancer. Value in Health. 2018; 21: 1259-68.
Phillippo DM, Ades AE, Dias S, Palmer S, Abrams KR, Welton NR. NICE technical support document 18: Methods for population-adjusted indirect comparisons in submission to NICE. December 2016. Available at: http://nicedsu.org.uk/technical-support-documents/population-adjusted-in…. Accessed 30 August 2022.
Remiro-Azocar A, Heath A, Baio G. Marginalization of regression-adjusted treatment effects in indirect comparisons with limited patient-level data. Working Paper, 2022. Available at: https://arxiv.org/pdf/2008.05951.pdf. Accessed 30 August 2022.
Remiro-Azocar A, Heath A, and Baio G. Methods for population adjustment with limited access to individual patient data: A review and simulation study. Res Syn Meth.2021;12:750?75.
Swallow E, Signorovitch J, Yuan Y, Kalsekar, A. Indirect comparisons for single-arm trials or trials without common comparator arms. Presented at the ISPOR 20?th Annual International Meeting, Philadelphia, PA, US; 15 May 2015. Virtual.
Scher HI, Fizazi K, Saad F, Taplin M-E, Sternberg CN, Miller K, et al. Increased survival with enzalutamide in prostate cancer after chemotherapy. N Engl J Med. 2012; 367(13): 1187-1197.
Wang L, Paller C, Hong H, Rosman L, De Felice A, Brawley O, et al. Comparison of treatments for nonmetastatic castration-resistant prostate cancer: Matching-adjusted indirect comparison and network meta-analysis. JNCI J Natl Cancer Inst. 2022; 114(2): djab071.
Young R. Matching-adjusted indirect comparison. maic in R. 2022. Available at https://cran.r-project.org/web/packages/maic/. Accessed 30 August 2022.