Skip to main content


Research Proposal

Project Title: 
Heterogeneity in relative treatment efficacy by age, sex, socioeconomic status and comorbidity
Scientific Abstract: 

Using evidence synthesis, estimates of efficacy from clinical trials can be applied to standard treatment comparator (natural history) event rates from observational data to estimate effectiveness in target populations.
In order to extend this approach to estimate treatment effectiveness in people with additional secondary diseases (comorbidity), we need to determine whether treatment efficacy is similar in people with and without comorbidity.

To estimate the variation in efficacy by comorbidity within clinical trials, and summarise this for different drug-classes and wider groupings of related drug classes.

Study design

Trials of drugs to prevent or treat long-term medical conditions (38 trials from the YODA repository, ~200 trials from other repositories)

Main outcome measures
Outcomes common to trials of specific drug-classes, eg HbA1c in diabetes trials.

Statistical analysis
For each outcome, in each trial, we will model main effects and interactions with treatment allocation for age, sex and the 6 commonest comorbidities. We will use generalized linear models, or Cox regression as appropriate to the outcome.

In subsequent analyses, using the coefficients and variance-covariance matrix from these models as the dependent variables, we will estimate the mean (and variance) of each comorbidity-treatment interaction for specific drug-classes, and across drug-classes for wider related drug-groupings.

We will explore the effect of different scales and transformations on the extent of heterogeneity.

Brief Project Background and Statement of Project Significance: 

Multimorbid patients are less likely than those with single diseases to receive recommended drugs, even where there are no specific contraindications.[6–11] This difference may represent under treatment or may have arisen from uncertainty among clinicians as to the applicability of clinical guidelines, which rarely provide specific advice on managing multimorbidity.[3,12,13]

Where multimorbidity alters the (baseline) natural history of diseases, the effects of treatment are likely to differ. For example, while there is strong evidence that the benefits of dual antiplatelet therapy (DAPT) following myocardial infarction (versus a single antiplatelet) outweigh the risks,[14,15] this may not be true for patients with comorbid chronic obstructive pulmonary disease (COPD). Cardiovascular mortality is commoner in COPD than the general population, favouring DAPT.[16] However, non-cardiovascular mortality is also higher,[17] favouring single-antiplatelet therapy because of competing risks. Glucose and intensive risk factor control in diabetes[18,19] and anticoagulant use in atrial fibrillation[20] provide similar examples, where the net overall treatment benefits are uncertain for multimorbid patients.

Neither clinical trials nor observational studies fully address this problem.[21] Trialists do not report results stratified by combinations of important comorbidities.[12] Nor is it feasible to conduct trials sufficiently large to allow such comparisons for all indications.[5,21] Administrative healthcare databases are sufficiently large. However, while estimates of treatment effectiveness obtained from observational studies are generally comparable to those from clinical trials,[22] they have in several cases led to inaccurate conclusions.[23–25]

There is therefore a need to find alternative approaches to produce estimates of net overall treatment benefits for patients with multimorbidity.

In a Wellcome Trust funded project, we intend to develop and validate an approach using Bayesian Evidence Synthesis to estimate treatment effectiveness in multimorbidity. As with standard evidence synthesis, we plan to apply estimates of treatment efficacy from clinical trials to observational data representative of the target population. However, in most evidence synthesis it is assumed that estimates of relative treatment efficacy (RTE) from clinical trials can be applied, unmodified, to the target population. It is not known whether this assumption is valid in applying efficacy estimates to patients with multimorbidity.

In a limited number of large individual-patient meta-analyses (such as of aspirin for primary and secondary prevention) RTE was similar regardless of cardiovascular risk factors, age and sex.[27,28] However, for most drugs and conditions empirical evidence is lacking as to whether and how RTE differs between patients with and without comorbidity, especially for comorbidities which are not established causes or complications of the target disease.

Specific Aims of the Project: 

Therefore, in this proposal we aim to examine and quantify the variation in relative treatment efficacy by comorbidity. We intend to do so by examining such variation within clinical trials, for a range of drug-classes, within wider groupings of related drug-classes (eg blood glucose lowering drugs).

Stage One – Produce trial-level summary estimates
1. Assign drugs to broad drug-classes based on 5-character ATC codes
2. Assign related drug-classes to wider drug-groupings
3. Identify comorbidities within individual clinical trials according to the Clinical Classification Software scheme[29]
4. For each trial estimate interactions between treatment allocation and age, sex, socio-economic status and selected comorbidities
Stage Two – Analyse trial-level aggregated estimates
5. In Bayesian hierarchical generalized linear models, use the estimates from 4 alongside estimates from clinical trials obtained from other repositories to estimate average drug-class-level and drug-grouping level comorbidity-treatment interaction
6. Summarise 5 as a probability distribution for use in subsequent evidence synthesis; either as off-the-shelf informative priors for meta-analyses, or as inputs to probabilistic sensitivity analyses in which treatment effects are modelled

What is the purpose of the analysis being proposed? Please select all that apply.: 
Participant-level data meta-analysis
Participant-level data meta-analysis pooling data from YODA Project with other additional data sources
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

The purpose of this project is to produce summaries, for and across drug-classes, of the covariate-treatment interactions for different outcomes within drug-groupings formed of clinically-related drug-classes, for subsequent use by ourselves and others either as informative priors, or in probabilistic sensitivity analyses. The covariates of interest being comorbidities, age, sex and socio-economic status.

Consequently, trials were chosen to include drugs used to treat or prevent long term medical (and, as these are a common issue in the elderly, urological) conditions. Both short and long term indications were included (eg short term post MI therapy). We have excluded trials where the indication was for neoplastic, infectious, affective, psychotic or developmental disorders. Topical eye and primary prevention trials in the general adult population were also excluded. Small trials, and those with highly restrictive inclusion criteria were also excluded.

This led to us identifying 38 relevant trials in the YODA repository. Covariate-treatment interaction estimates from which will be analysed alongside estimates from similar trials obtained from other repositories

Main Outcome Measure and how it will be categorized/defined for your study: 

As our aim, is to ‘survey’ many trials to determine how much variation in treatment effect by comorbidity is plausible within different clinically-meaningful groupings, we intend to obtain measures which are similar across multiple trials. Therefore, within each indication we will select endpoints common to trials for that indication. For example, the majority of trials for rheumatoid arthritis include the American College of Rheumatology Criteria and Disease Activity Score endpoints.

In order to simplify comparisons, we will scale continuous outcomes and will also use log-transformation in order to examine heterogeneity by comorbidity on the relative scale. After examining trial documentation and datasets, but prior to performing any comparisons by treatment allocation, we will with our steering group agree on the final outcomes, transformations etc.

While we believe that this approach is suitable for our analysis, we recognise that it is not likely to be optimal for estimating the main effect for each trial. For that reason, we will not report the main effects (ie marginal across sub-groups) from individual trials.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

We will examine covariate-treatment interactions for each trial. The covariates included will be age, sex, socio-economic status and comorbid disease.

Age will be modelled as a continuous variable, where there is evidence of non-linearity we will use fractional polynomials. Sex will be defined as per the study documentation. We will define socio-economic status separately using educational attainment and income as proxies, collapsing the reported categories into a three-level variable.

Comorbidities will be categorised using the Agency for Healthcare Research and Quality clinical classifications system (CCS). [30] We will attempt to define in the trial datasets comorbid conditions from 94 of the CCS groups. Trial-specific operational definitions of each comorbidity will be defined using a combination of demographic, past medical history, lifestyle (eg smoking) and drug variables as well as information from trial protocols (ie inclusion and exclusion criteria). We anticipate that it will frequently not be possible to arrive at an operational definition for many of the comorbidities.

Each operational trial definition will be finalised prior to analysing outcome data.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 


Statistical Analysis Plan: 

Stage one
For each outcome, we will model main effects and interactions with treatment allocation for age, sex, socio-economic status (in 3-levels) and the 6 commonest comorbidities for that drug-grouping. Where a particular trial does not have a comorbidity variable defined we will record that variable as missing.

We are not aware of any study which has examined the effect of different scales on the extent of heterogeneity in treatment-comorbidity interactions between patients. Therefore, for continuous measures we will model outcomes on both the absolute and relative (log-transformed) scale, and for yes/no event outcome data we will model outcomes on the log-odds ratio, log-rate ratio and log-risk ratio scales. We will estimate hazard ratios using Cox regression, and for all other models will use Generalized Linear Models with appropriate link functions and error distributions.

Analyses will be conducted in R. We will obtain the coefficients and variance covariance matrices from each of these models. For simplicity, missingness will be treated using complete case analysis. We will however record and report, for each operational comorbidity definition, the proportion of participants for whom sufficient data was missing to prevent a participant being defined as having a comorbid disease.

In summary, therefore, the following results, all of which are trial-level aggregates/summaries, will be obtained from the above analyses, conducted on the YODA server:-
1. The proportion of participants with each combination of comorbidities
2. The proportion of participants with missing data included in the definition of each comorbidity
3. The coefficients and variance-covariance matrix of the regression models described above.

Stage Two
All subsequent modelling will be conducted on trial summary-level data within our institution.

In the second part of a two-step approach we will model the comorbidity-treatment interactions across multiple trials using the parameters described above as the dependent variables in Bayesian hierarchical generalized linear models (eg with a multivariate Gaussian likelihood). The main effects for treatments and covariates will have independent priors for each trial, while the priors for the covariate-treatment interactions will have hyper-priors, to allow sharing of information. We have successfully fit such models on simulated data.

For each covariate-treatment interaction we will report the mean effect for each drug-class and drug grouping, along with the between drug-class variance. We will first model each comorbidity in turn, after which we will simultaneously model multiple comorbidities. We will separately examine associations for datasets comprising trials where a trial drug is compared with placebo, an agent from a different drug-class, and an agent from the same drug-class.

An assumption of this modelling approach is that comorbidity-treatment interactions are exchangeable for trials within drug-classes, and for drug-classes within wider drug groupings. This assumption makes use of the structure inherent in the anatomic therapeutic classification system; thereby allowing borrowing of information across drug-classes (eg antithrombotic agents).

If there is no similarity in treatment-comorbidity interactions within our drug-groupings, the predicted comorbidity-treatment interaction for an unknown drug-class will have a wide uncertainty interval, indicating that little can be learned about one drug class comorbidity-treatment interaction based on knowledge about such an interaction for another drug-class within that grouping.

These models will be fit in the JAGS package. Alternative model specifications will be explored and compared using the deviance information criterion (DIC) and the Bayesian information criterion (BIC). Model convergence and autocorrelation will be assessed using diagnostic plots as well as summaries such as the Gelman-Rubin statistic.

Narrative Summary: 

Randomised clinical trials provide the best evidence about treatment effectiveness. Trial results can be applied to “target” populations, for example; if a diabetes trial shows that a drug reduces blindness by 20%, and other data shows that 8 out of 100 people with diabetes develop blindness, then 2 fewer people are expected to develop blindness per 100 treated.

We would like to use a similar approach for a new target, people with additional (co-morbid) diseases. To make sure this is valid, we will examine clinical trial data from YODA, and other clinical trial repositories, to see if trial participants with and without different co-morbid diseases experience similar treatment benefits.

Project Timeline: 

1. Defining covariates (age, sex and sex-treatment interactions) , identifying treatment allocation and outcome variables and standardising the latter - 6 months.
2. Modelling covariate-treatment interactions and obtaining coefficients and variance-covariance matrices - 6 months.
Analysing summaries (2) with summaries from other trial repositories - 2-years.

Dissemination Plan: 

The protocol will be registered on the PROSPERO register for systematic reviews (or similar) prior to accessing the data. An abstract summarising these results will be submitted to a relevant international conference within one year of completing the analysis and summary results will be made available via our institutional website. A manuscript reporting these findings will be submitted to an appropriate scientific journal such as Medical Decision Making, the Journal of Clinical Epidemiology or Trials.


1. Weiss CO, Boyd CM, Yu Q, Wolff JL, Leff B. Patterns of prevalent major chronic disease among older adults in the united states. J Am Med Assoc. 2007 Sep 12;298(10):1158–62.
2. Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012 Jul;380(9836):37–43.
3. Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. J Am Med Assoc. 2005 Aug 10;294(6):716–24.
4. Fabbri LM, Boyd C, Boschetto P, Rabe KF, Buist AS, Yawn B, et al. How to integrate multiple comorbidities in guideline development: article 10 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc. 2012 Dec;9(5):274–81.
5. Boyd CM, Kent DM. Evidence-based medicine and the hard problem of multimorbidity. J Gen Intern Med. 2014 Apr;29(4):552–3.
6. Bursi F, Vassallo R, Weston SA, Killian JM, Roger VL. Chronic Obstructive Pulmonary Disease after Myocardial Infarction in the Community. Am Heart J. 2010 Jul;160(1):95–101.
7. Quipourt V, Jooste V, Cottet V, Faivre J, Bouvier A-M. Comorbidities alone do not explain the undertreatment of colorectal cancer in older adults: a French population-based study. J Am Geriatr Soc. 2011 Apr;59(4):694–8.
8. Crump C, Winkleby MA, Sundquist K, Sundquist J. Comorbidities and mortality in persons with schizophrenia: a Swedish national cohort study. Am J Psychiatry. 2013 Mar;170(3):324–33.
9. Wang PS, Avorn J, Brookhart MA, Mogun H, Schneeweiss S, Fischer MA, et al. Effects of Noncardiovascular Comorbidities on Antihypertensive Use in Elderly Hypertensives. Hypertension. 2005 Aug 1;46(2):273–9.
10. Smith DJ, Martin D, McLean G, Langan J, Guthrie B, Mercer SW. Multimorbidity in bipolar disorder and undertreatment of cardiovascular disease: a cross sectional study. BMC Med. 2013;11:263.
11. Søgaard M, Thomsen RW, Bossen KS, Sørensen HT, Nørgaard M. The impact of comorbidity on cancer survival: a review. Clin Epidemiol. 2013;5(Suppl 1):3–29.
12. Boyd CM, Vollenweider D, Puhan MA. Informing Evidence-Based Decision-Making for Patients with Comorbidity: Availability of Necessary Information in Clinical Trials for Chronic Diseases. PLoS ONE. 2012 Aug 3;7(8):e41601.
13. Mutasingwa DR, Ge H, Upshur REG. How applicable are clinical practice guidelines to elderly patients with comorbidities? Can Fam Physician. 2011 Jul;57(7):e253–62.
14. Chen ZM, Jiang LX, Chen YP, Xie JX, Pan HC, Peto R, et al. Addition of clopidogrel to aspirin in 45,852 patients with acute myocardial infarction: randomised placebo-controlled trial. Lancet. 2005 Nov 5;366(9497):1607–21.
15. Effects of Clopidogrel in Addition to Aspirin in Patients with Acute Coronary Syndromes without ST-Segment Elevation. N Engl J Med. 2001;345(7):494–502.
16. MacNee W, Maclay J, McAllister D. Cardiovascular Injury and Repair in Chronic Obstructive Pulmonary Disease. Proc Am Thorac Soc. 2008 Dec 1;5(8):824–33.
17. McGarvey LP, John M, Anderson JA, Zvarich M, Wise RA. Ascertainment of cause-specific mortality in COPD: operations of the TORCH Clinical Endpoint Committee. Thorax. 2007 May;62(5):411–5.
18. Timbie JW, Hayward RA, Vijan S. Diminishing Efficacy of Combination Therapy, Response-Heterogeneity, and Treatment Intolerance Limit the Attainability of Tight Risk Factor Control in Patients with Diabetes. Health Serv Res. 2010 Apr;45(2):437–56.
19. Type 2 diabetes in adults: management | Guidance and guidelines | NICE [Internet]. [cited 2015 Dec 14]. Available from:
20. Lopes LC, Spencer FA, Neumann I, Ventresca M, Ebrahim S, Zhou Q, et al. Systematic Review of Observational Studies Assessing Bleeding Risk in Patients with Atrial Fibrillation Not Using Anticoagulants. PLoS ONE [Internet]. 2014 Feb 11 [cited 2014 Dec 31];9(2). Available from:
21. Weiss CO, Varadhan R, Puhan MA, Vickers A, Bandeen-Roche K, Boyd CM, et al. Multimorbidity and evidence generation. J Gen Intern Med. 2014 Apr;29(4):653–60.
22. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev. 2014;4:MR000034.
23. Hue TF, Cummings SR, Cauley JA, Bauer DC, Ensrud KE, Barrett-Connor E, et al. Effect of bisphosphonate use on risk of postmenopausal breast cancer: results from the randomized clinical trials of alendronate and zoledronic Acid. JAMA Intern Med. 2014 Oct 1;174(10):1550–7.
24. Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women’s Health Initiative randomized controlled trial. JAMA. 2002 Jul 17;288(3):321–33.
25. Criner GJ, Connett JE, Aaron SD, Albert RK, Bailey WC, Casaburi R, et al. Simvastatin for the Prevention of Exacerbations in Moderate-to-Severe COPD. N Engl J Med. 2014 May 18;370(23):2201–10.
26. Koné Pefoyo AJ, Bronskill SE, Gruneir A, Calzavara A, Thavorn K, Petrosyan Y, et al. The increasing burden and complexity of multimorbidity. BMC Public Health [Internet]. 2015 Apr 23 [cited 2015 May 21];15. Available from:
27. Aspirin in the primary and secondary prevention of vascular disease: collaborative meta-analysis of individual participant data from randomised trials. The Lancet. 2009 May;373(9678):1849–60.
28. Emberson J, Lees KR, Lyden P, Blackwell L, Albers G, Bluhmki E, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet Lond Engl. 2014 Nov 29;384(9958):1929–35.
29. Elixhauser A, Steiner C, Whittington C, McCarthy E. Clinical Classifications for Health Policy Research: Hospital Inpatient Statistics [Internet]. Agency for Health Care Policy and Research, Rockville, MD; 1995 [cited 2015 Dec 31]. Available from:
30. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998 Jan;36(1):8–27.
31. Agency for Healthcare Research and Quality. Clinical Classifications Software for ICD-10 Data [Internet]. 2012 [cited 2015 Dec 31]. Available from:
32. Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials [Internet]. NICE. 2011 [cited 2014 Jan 24]. Available from:

Supplementary Material: 

General Information

How did you learn about the YODA Project?: 

Request Clinical Trials

Associated Trial(s): 
What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation