Skip to main content


Research Proposal

Project Title: 
Comparative safety and effectiveness of cognitive enhancers for Alzheimer's dementia: a systematic review and IPD NMA
Scientific Abstract: 

Background: Alzheimer's dementia (AD) is the most common cause of dementia, and several organisations, such as the National Institute for Health and Care Excellence, suggest that management of patients with AD should be tailored to their needs. To date, little research has been conducted on the treatment effect in different subgroups of patients with AD.

Objective: To examine the comparative effectiveness and safety of cognitive enhancers for different patient characteristics.

Study design: Systematic review of randomised clinical trials of any duration comparing cognitive enhancers alone or in any combination against other cognitive enhancers, or placebo in adults with AD.

Participants: Adults (aged ≥18 years) diagnosed with AD

Main Outcome Measures: The primary outcome of interest is cognition according to the Mini-mental State Examination (MMSE), and the secondary outcome is overall serious adverse events.

Statistical Analysis: We will perform a Bayesian hierarchical random-effects meta-analysis combining the individual patient data (IPD) from each eligible study. If the identified treatment comparisons form a connected network diagram, we will perform an IPD network meta-analysis (NMA) to estimate subgroup effects for patients with different characteristics, such as AD severity and sex. We will combine aggregated data from studies that we will not be able to obtain IPD, with the IPD provided by the original authors, in a single model. We will use the PRISMA-IPD[1] and PRISMA-NMA[2] statements to report our findings.

Brief Project Background and Statement of Project Significance: 

Alzheimer's dementia (AD) is the most common cause of dementia, and has an insidious onset with progressive deterioration in cognition (eg, memory, thinking and perception), function, behaviour and mood. To date, 46.8 million people worldwide live with dementia. This number will almost double every 20 years, and it is estimated to reach 131.5 million by 2050.[3] A study showed that as age increases, the rates of AD increase overall for both men and women, but it is more prevalent in women (rate/100 years=2.50 (1.85–3.41)) than men (rate/100 years=1.89 (1.22–2.94)).[4] It is currently unclear if galantamine, rivastigmine or donepezil should be used by patients with severe AD, and whether memantine is the most optimal treatment for severe AD.[5] The use of acetylcholinesterase inhibitors and increased doses of donepezil in patients with dementia increase the risk of bradycardia, as well, cholinesterase inhibitors doubles the risk of hospitalisation for bradycardia in older patients.[6, 7] Also, the use of other medications may increase risk of adverse events. For example, cardiac medications like β-blockers may increase risk of bradycardia, and anti-inflammatories may increase risk for gastrointestinal bleeding.[6, 8-10]

To determine the relative effectiveness and safety of cognitive enhancers for patients with different patient characteristics (eg, mild-moderate AD vs severe AD, females vs males), we aim to conduct a systematic review and individual patient data (IPD) network meta-analysis (NMA). In AD, patients may respond differently to the medication based on severity of AD and sex, and hence severity and sex could be considered treatment effect modifiers. The optimal approach to tailor results to the patient characteristics is via using IPD. Tailoring the management of patients with AD is an issue that has been also brought up by several organisations,[11] including the Alzheimer's Society of Ontario[12] and the National Institute for Health and Care Excellence (NICE).[13] Also, the Alzheimer's Disease International (ADI) federation in their world Alzheimer report 2015 mention that there has been dramatically little research into the treatment effect across people of different age and sex.[3]

We previously attempted a systematic review and NMA of aggregated data, but we were unable to provide definitive conclusions regarding the influence of patient characteristics on the results.[14, 15] In this study we tailored results to age, AD severity, comorbidity and study duration via subgroup analysis. These results were similar to 4 Cochrane reviews examining cognitive enhancers for AD.[16-19] The reviews showed that donepezil, rivastigmine and galantamine, significantly improved cognition[16-19] against placebo, yet cholinisterase inhibitors overall and donepezil improved behaviour,[16, 17] cholinisterase inhibitors overall and rivastigmine improved function,[17, 18] and rivastigmine improved AD severity.[18] The use of IPD will increase power and will help explain the relationship between treatment effects and patient-level characteristics.

Specific Aims of the Project: 

The aim of this study is to examine the comparative effectiveness and safety of cognitive enhancers versus placebo or best supportive care by patient characteristics, such as AD severity and sex. We will use IPD-NMA to identify potential treatment effect modifiers, and estimate the most effective and safest treatments for patients with different characteristics. The outputs of our project are to provide clinicians, patients and caregivers with tailored evidence to inform their decision making, improving the health of patients living with AD.

What is the purpose of the analysis being proposed? Please select all that apply.: 
New research question to examine treatment effectiveness on secondary endpoints and/or within subgroup populations
New research question to examine treatment safety
Participant-level data meta-analysis
Participant-level data meta-analysis pooling data from YODA Project with other additional data sources
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

We have updated our previous review[15] using the following criteria:
• Population: Adults (aged ≥18 years) with AD diagnosed using various criteria (eg, Diagnostic and Statistical Manual of Mental Disorders, Nursing Minimum Data Set criteria) of any duration with either moderate AD.[20]
• Interventions: Cognitive enhancers (donepezil, rivastigmine, galantamine and memantine) alone or in any combination.
• Comparators: Cognitive enhancers, best supportive care alone or in any combination, and placebo.
• Outcomes: MMSE and overall serious adverse events.
• Study design: We will restrict to RCTs, and will exclude quasi-RCTs.
• Time: Studies of any duration conducted at any time.
• Other: Published studies written in any language will be included.
In case study publications reported data from the same study group (eg, companion reports), we included the most recent study.
Our systematic reviews identified 139 relevant studies. We will include IPD from the studies reported in the supplementary, as well as aggregate data from all remaining published studies.

Main Outcome Measure and how it will be categorized/defined for your study: 

The primary outcome of interest is cognition according to the MMSE (efficacy outcome, continuous variable), and the secondary outcome is overall serious adverse events (SAEs; safety outcome, dichotomous variable); both outcomes were reported by many of the included trials previously and for which NMA was possible. In particular, in our previous NMA using aggregated data, 60 RCTs with 15 862 patients contributed to a NMA for the MMSE outcome, and 51 RCTs with 19 329 patients contributed to a NMA for SAEs.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

We will use a data-driven approach. More specifically, all IPD variables provided will be entered in our NMA and in the meta-regression analysis we will start by including one dependent and one independent variable. Then significant moderators will simultaneously be entered into multiple regression models as long as the minimum number of cases per independent variable is 10. Our goal is to avoid over-fitting and provide reliable treatment effect estimates.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

The data we plan to abstract include study characteristics (eg, year of publication), aggregated patient characteristics (eg, number of patients), outcome results (eg, MMSE, SAE) and source of funding (categorised as: funded/authored by an employee of a drug manufacturer or other commercial organisation, government-sponsored/non-profit organisations, including universities and hospitals, no funding, funding unclearly reported, and funding not reported).[21] Two reviewers will abstract data independently, and all conflicts will be resolved through discussion. The year of publication and funding are potential effect modifiers. Therefore, these factors will be explored in a network meta-regression assuming a common fixed coefficient across treatment comparisons.

Statistical Analysis Plan: 

As with the original review, we will appraise the risk of bias using the Cochrane Risk of Bias tool.[22] We will draw a comparison-adjusted funnel plot[23] for both outcomes. Two review authors will also independently assess the quality of evidence in each NMA using the GRADE approach as extended for network meta-analysis.[24]

We will perform a Bayesian hierarchical random-effects meta-analysis for each treatment comparison, as we anticipate clinical and methodological between-study heterogeneity. We will perform a two-stage analysis, where at the first stage each individual will be analysed separately in each trial and at the second step the trial parameter estimates will be synthesised in a pairwise meta-analysis. All IPD from included studies will be first aggregated to study-level summary statistics using the R software (platform provided by the YODA project), and then these estimates will be introduced into the random-effects meta-analysis model. We will use the odds ratio for SAE[25] and the mean difference effect size for MMSE.[26] In case we are able to obtain IPD for a subset of trials, then we will use a two-part model with the same between-study variance in both parts and accounting for treatment-by-covariate interactions (including for example co-morbidities such as arrhythmias in the model[27]). The first part will entail the two-stage model described above using IPD only, whereas the second part will entail applying a pairwise meta-analysis with aggregate data.[27]
For a connected network of trials, the random-effects NMA model will be used. If possible, we will combine information across a network of trials using only IPD. If we are not successful in obtaining IPD for at least one study, we will combine both IPD and aggregated data in a single model. Again, a two-part analysis will be applied, considering the IPD reduced to aggregate data in the first part, and the aggregate data as identified in the published trials in the second part. Both IPD and aggregate data studies will share the same amount of heterogeneity. Information on patient-level covariates (eg, AD severity, sex) will be included in the model as secondary analyses. We will evaluate the consistency assumption using the design-by-treatment interaction model [28, 29] and the loop-specific method [30, 31] using aggregated data. If inconsistency is suggested, we will check the data for discrepancies and if none are identified, subgroup or meta-regression analyses will be performed.
We will estimate subgroup effects (eg, age, sex) using treatment-by-covariate interaction terms within studies and combining these across studies. We will apply 3 model specifications assuming that the regression coefficients are: a) different and unrelated across comparisons, b) different but related, sharing the same distribution, and c) identical across comparisons.[32, 33] We will compare the results of the models by evaluating the statistical significance of the regression coefficients for interactions, monitoring the reduction in the between-study variance, and using the Deviance Information Criterion[34] to compare the overall fit and parsimony of the models. We will rank the interventions for each outcome using the surface under the cumulative ranking curve.[35]

We will conduct multiple sensitivity analyses to examine the robustness of our results. We will: 1) restrict to studies with IPD only, 2) use different priors for the between-study variance, [36-38]3) restrict to RCTs with a low risk of bias, 4) use different imputation techniques for missing outcome data.[39, 40]

All pairwise meta-analyses and NMAs will be conducted using the Bayesian software OpenBUGS.[41] Two chains will be generated and convergence will be evaluated by their mixing, after discarding the first 10,000 iterations. We will use vague priors for all parameters of the models apart from the between-study variance for which we will use informative priors.[37, 38]

Narrative Summary: 

Alzheimer’s Dementia (AD) is the most common cause of dementia. Patients living with AD have a lower quality of life (deterioration in memory, thinking, perception, function, behaviour, and mood) and AD ultimately leads to death. Currently, there is no cure for AD, and patients may respond differently to the medication based on their characteristics (eg, severity of disease, sex). We aim to investigate the association between the cognitive enhancers for different patient characteristics and Mini-mental State Examination or overall serious adverse events. The findings of this study will help to improve guidelines for the management of patients with AD.

Project Timeline: 

Our study protocol was published on 7 December 2015 in an open access journal (see We started contacting the study authors to request for their IPD on 10 June 2016. By the time we receive the IPD we will collect, review and clean the data within 3-4 months. We anticipate that the statistical analyses will take another 3-4 months, depending on the complexity of the models and data. We will need approximately 2-3 months to prepare the manuscript and submit it. We expect the first submission of the manuscript will be approximately in March 2018.

Dissemination Plan: 

The findings of our study will fill an important knowledge gap in healthcare, and will be used to inform decision-making for patients suffering from this debilitating disease. The results of this systematic review and IPD-NMA will be of interest to stakeholders, including decision makers, guideline developers, clinicians, methodologists and patients. The dissemination of our findings will be knowledge user-driven and tailored to how and when knowledge users want to receive information. Team members will act as knowledge brokers, using their networks to facilitate dissemination, such as The Cochrane Collaboration, PRISMA-IPD, Drug Safety and Effectiveness Network (DSEN). We will also host a knowledge exchange event with our partners to discuss the results and facilitate dissemination. We will publish our findings in an open access journal, and present them at relevant meetings (Canadian Geriatrics Society; CGS), as well to newsletters of organisations (Alzheimer's Society of Ontario, CGS).


1. Stewart LA, Clarke M, Rovers M, et al. Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data: the PRISMA-IPD Statement. JAMA. 2015;313(16):1657-65.
2. Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777-84.
3. Prince M, Wimo A, Guerchet M, Ali GC, Wu YT, Prina M. World Alzheimer Report 2015. The global impact of dementia. An analysis of prevalence, incidence, cost and trends. London, UK; 2015.
4. Katz MJ, Lipton RB, Hall CB, et al. Age-specific and sex-specific prevalence and incidence of mild cognitive impairment, dementia, and Alzheimer dementia in blacks and whites: a report from the Einstein Aging Study. Alzheimer Dis Assoc Disord. 2012;26(4):335-43.
5. National Institute for Health and Clinical Excellence. Donepezil, galantamine, rivastigmine and memantine for the treatment of Alzheimer's disease. London, UK; 2011.
6. Park-Wyllie LY, Mamdani MM, Li P, Gill SS, Laupacis A, Juurlink DN. Cholinesterase inhibitors and hospitalization for bradycardia: a population-based study. PLoS Med. 2009;6(9):e1000157.
7. Hernandez RK, Farwell W, Cantor MD, Lawler EV. Cholinesterase inhibitors and incidence of bradycardia in patients with dementia in the veterans affairs new England healthcare system. J Am Geriatr Soc. 2009;57(11):1997-2003.
8. Manurung D, Trisnohadi HB. Beta blockers for congestive heart failure. PLoS Med. 2007;39(1):44-8.
9. Gheorghiade M, Colucci WS, Swedberg K. Beta-blockers in chronic heart failure. Circulation. 2003;107(12):1570-5.
10. Pahor M, Guralnik JM, Furberg CD, Carbonin P, Havlik R. Risk of gastrointestinal haemorrhage with calcium antagonists in hypertensive persons over 67 years old. Lancet. 1996;347(9008):1061-5.
11. Sindi S, Mangialasche F, Kivipelto M. Advances in the prevention of Alzheimer's Disease. F1000Prime Rep. 2015;7:50.
12. Williams AP, Peckham A, Rudoler D, Tam T, Watkins J. Formative evaluation of the Alzheimer society of Toronto counselling program. Alzheimer society of Toronto; 2013.
13. National Institute for Health and Clinical Excellence. Dementia: Supporting people with dementia and their carers in health and social care. London, UK; 2006.
14. Tricco AC, Vandervaart S, Soobiah C, et al. Efficacy of cognitive enhancers for Alzheimer's disease: protocol for a systematic review and network meta-analysis. Syst Rev. 2012;1:31.
15. Tricco AC, Ashoor HM, Rios P, et al. Comparative safety and effectiveness of cognitive enhancers for the treatment of Alzheimer’s disease: A rapidly updated systematic review and network meta-analysis Toronto, Canada; 2015.
16. Birks J, Harvey RJ. Donepezil for dementia due to Alzheimer's disease. Cochrane Database Syst Rev. 2006(1):Cd001190.
17. Birks J. Cholinesterase inhibitors for Alzheimer's disease. Cochrane Database Syst Rev. 2006(1):Cd005593.
18. Birks J, Grimley Evans J, Iakovidou V, Tsolaki M, Holt FE. Rivastigmine for Alzheimer's disease. Cochrane Database Syst Rev. 2009(2):Cd001191.
19. Loy C, Schneider L. Galantamine for Alzheimer's disease and mild cognitive impairment. Cochrane Database Syst Rev. 2006(1):Cd001747.
20. Burback D, Molnar FJ, St John P, Man-Son-Hing M. Key methodological features of randomized controlled trials of Alzheimer's disease therapy. Minimal clinically important difference, sample size and trial duration. Dement Geriatr Cogn Disord. 1999;10(6):534-40.
21. Cochrane Effective Practice and Organisation of Care Review Group (EPOC). Data Collection Checklist. Available from:
22. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from
23. Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta-analysis in STATA. PLoS One. 2013;8(10):e76654.
24. Puhan MA, Schunemann HJ, Murad MH, et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ. 2014;349:g5630.
25. Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med. 2000;19(24):3417-32.
26. Higgins JP, Whitehead A, Turner RM, Omar RZ, Thompson SG. Meta-analysis of continuous outcome data from individual patients. Stat Med. 2001;20(15):2219-41.
27. Sutton AJ, Kendrick D, Coupland CA. Meta-analysis of individual- and aggregate-level data. Stat Med. 2008;27(5):651-69.
28. Higgins JP, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods. 2012;3(2):98-110.
29. White IR, Barrett JK, Jackson D, Higgins JP. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Synth Methods. 2012;3(2):111-25.
30. Song F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003;326(7387):472.
31. Veroniki AA, Vasiliadis HS, Higgins JP, Salanti G. Evaluation of inconsistency in networks of interventions. Int J Epidemiol. 2013;42(1):332-45.
32. Donegan S, Williamson P, D'Alessandro U, Garner P, Smith CT. Combining individual patient data and aggregate data in mixed treatment comparison meta-analysis: Individual patient data may be beneficial if only for a subset of trials. Stat Med. 2013;32(6):914-30.
33. Donegan S, Williamson P, D'Alessandro U, Tudur Smith C. Assessing the consistency assumption by exploring treatment by covariate interactions in mixed treatment comparison meta-analysis: individual patient-level covariates versus aggregate trial-level covariates. Stat Med. 2012;31(29):3840-57.
34. Spiegelhalter DJ, Best NG, Carlin BP, Van Der Linde A. Bayesian measures of model complexity and fit. J R Stat Soc Series B Stat Methodol. 2002;64(4):583-639.
35. Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol. 2011;64(2):163-71.
36. Lambert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med. 2005;24(15):2401-28.
37. Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JP. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. Int J Epidemiol. 2012;41(3):818-27.
38. Rhodes KM, Turner RM, Higgins JP. Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data. J Clin Epidemiol. 2015;68(1):52-60.
39. Mavridis D, White IR, Higgins JP, Cipriani A, Salanti G. Allowing for uncertainty due to missing continuous outcome data in pairwise and network meta-analysis. Stat Med. 2015;34(5):721-41.
40. Spineli LM, Higgins JP, Cipriani A, Leucht S, Salanti G. Evaluating the impact of imputations for missing participant outcome data in a network meta-analysis. Clin Trials. 2013;10(3):378-88.
41. Lunn D, Spiegelhalter D, Thomas A, Best N. The BUGS project: Evolution, critique and future directions. Stat Med. 2009;28(25):3049-67.