Skip to main content

Colleague

2019-3980

Project Title: 
Impact of Ulcer Size and Extent of Inflammation On Ability To Achieve Endoscopic Healing In Crohn’s Disease: A SONIC post hoc Analysis
Specific Aims of the Project: 

This study of patients with Crohn’s disease from the SONIC trial (Study of Biologic and Immunomodulator Naïve Patients in Crohn’s Disease; ClinicalTrials.gov, NCT00094458) aims to evaluate the association of baseline endoscopic ulcerations (size, depth and location), disease activity, as measured by CD endoscopic index of severity (CDEIS) or Simple Endoscopic Score-CD (SES-CD), with the achievement of EH and CR at week 26.

Our hypothesis is that CD patients who are biologic and immunodulator naïve with baseline larger ulcer sizes and extensive inflammation are less likely to achieve EH and CR, regardless of treatment with either combination therapy, infliximab or azathioprine monotherapy at week 26.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Ongoing
Scientific Abstract: 

Background
Current and future medical therapies in Crohn’s disease (CD) need to demonstrate efficacy in achieving mucosal healing1. However, there is insufficient information regarding what degree of ulceration and inflammation affects endoscopic healing (EH) and clinical remission (CR).

Objective
This study aims to evaluate the association of baseline mucosal lesions and disease activity with the achievement of EH and CR at week 26.

Study Design
SONIC was a multicentre, randomised, double-blinded trial that randomized patients to infliximab, azathioprine, or combination therapy and evaluated corticosteroid-free CR at week 26. This post hoc analysis will assess the likelihood of achieving EH and CR at week 26 based on baseline endoscopic inflammation present.

Participants
Moderate-to-severe CD patients who are naive to immunomodulators and biologics, and had poor response to conventional therapies were eligible.

Main Outcome Measure(s)
The primary outcome measures will be EH, CR and corticosteroid-free CR at week 26. Baseline mucosal lesion features and inflammation will be assessed using endoscopic scoring systems.

Statistical Analysis
A multivariate logistic regression analysis will be used to examine the relationship between baseline endoscopic extent of disease and ability to achieve week 26 EH and CR. Known confounding factors for achieving endoscopic healing, such as treatment allocation and disease duration, will be adjusted for.

Brief Project Background and Statement of Project Significance: 

Crohn’s disease is a progressive, relapsing and remitting disease due to chronic transmural inflammation which can lead to complications such as strictures, fistulas, and abscess formation2. Current medical therapies and prospective therapies have adopted a ‘treat-to-target’ approach with the primary target being an ability to achieve EH, defined as an absence of ulcers on endoscopy1. EH in inflammatory bowel disease has become an important measure of treatment efficacy and a prognostic indicator of long-term adverse outcomes, such as hospitalizations and surgeries 3-5. However, there is currently insufficient information as to what degree of ulceration and inflammation of mucosal lesions across the different colonic segments and ileum affects EH and CR. This has clinical implications in moderate-to-severe CD as it determines if baseline characteristics and location of mucosal lesions, extent of inflammation are negative predictors of EH and CR.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

Data Source: Eligibility criteria and the SONIC (Study of Biologic and Immunomodulator Naïve Patients in Crohn’s Disease) study design were previously published in the clinical trial(ClinicalTrials.gov number: NCT00094458).

Inclusion Criteria: Patients eligible will be at least 21 years of age and have had CD for least 6 weeks, with moderate-to-severe disease activity (i.e. Crohn’s Disease Activity Index [CDAI] score 220-450 points). These patients will be either corticosteroid-dependent, being considered for a second course of systemic corticosteroids within 12 months, or will have had no response to either mesalamine (≥ 2.4 g/d) or budesonide (≥ 6 mg/d) after at least 4 weeks of treatment.

Exclusion Criteria: Patients who are ineligible include those with previous treatment to 6-mercaptopurine, methotrexate, or anti-TNF biologic agents. Previous abdominal surgery in the previous 6 months, symptomatic stricture, abscess, short gut, or ostomy. Patients with granulomatous infection, including tuberculosis, active infection with human immunodeficiency virus, hepatitis B or C, or opportunistic infection in the prior 6 months, multiple sclerosis or malignancy were excluded.

Narrative Summary: 

Endoscopic healing of the bowel has become a goal of treatment for patients with Crohn’s disease. Clinical trials conducted in Crohn’s disease for new therapies need to demonstrate the ability to heal the mucosa. However, nothing is known about the impact of lesion size and distribution on the ability to achieve endoscopic healing. This study proposes to look at the SONIC (Study of Biologic and Immunomodulator Naive Patients in Crohn's Disease; ClinicalTrials.gov, NCT00094458) database and determine whether patients who have more extensive inflammation at baseline and larger ulcer sizes are less likely to achieve endoscopic healing.

Project Timeline: 

Anticipated Project Start Date: To be started within the first week of database approval and acquisition in September 2019.

Analysis Completion Date: Research proposal to be finalized with data collection and analysis. Estimated data of completion will be November 2019.

Manuscript Draft Date: Manuscript draft estimated to be completed in December 2019 – January 2020.

Manuscript Submission Date: January – February 2020

Date Results Reported to YODA: March - April 2020

The dissemination of results, which may include but are not limited to abstracts and manuscripts will be reported to the YODA Project at the time of submission.

Dissemination Plan: 

Anticipated products include abstracts, which will be published or shared during scientific meetings, including Canadian Digestive Diseases Week, Digestive Disease Week, and European Crohn’s and Colitis Organisation. Additionally, a manuscript is expected to be completed for the research project and will be submitted for publication. Potential journals for submission include Clinical Gastroenterology and Hepatology, Journal of Crohn’s and Colitis, Inflammatory Bowel Diseases, and Digestive Diseases and Sciences. The dissemination of results, which may include but are not limited to abstracts, manuscripts, preprints, posters, and slide decks will be shared with the YODA Project at the time of submission.

Target audiences include clinicians and researchers interested in the advancement of the inflammatory bowel disease diagnostics and management.

Bibliography: 

1. Bouguen G, Levesque BG, Feagan BG, Kavanaugh A, Peyrin–Biroulet L, Colombel JF, Hanauer SB, Sandborn WJ. Treat to target: a proposed new paradigm for the management of Crohn's disease. Clinical Gastroenterology and Hepatology. 2015 Jun 1;13(6):1042-50.
2. Rieder F, Zimmermann EM, Remzi FH, Sandborn WJ. Crohn’s disease complicated by strictures: a systematic review. Gut. 2013;62:1072-1084.
3. Rutgeerts P, Diamond RH, Bala M, Olson A, Lichtenstein GR, Bao W, Patel K, Wolf DC, Safdi M, Colombel JF, Lashner B. Scheduled maintenance treatment with infliximab is superior to episodic treatment for the healing of mucosal ulceration associated with Crohn's disease. Gastrointestinal endoscopy. 2006 Mar 1;63(3):433-42.
4. Frøslie KF, Jahnsen J, Moum BA, Vatn MH, IBSEN Group. Mucosal healing in inflammatory bowel disease: results from a Norwegian population-based cohort. Gastroenterology. 2007 Aug 1;133(2):412-22.
5. Schnitzler F, Fidder H, Ferrante M, Noman M, Arijs I, Van Assche G, Hoffman I, Van Steen K, Vermeire S, Rutgeerts P. Mucosal healing predicts long‐term outcome of maintenance therapy with infliximab in Crohn's disease. Inflammatory bowel diseases. 2009 Sep;15(9):1295-301.
6. Best, W., Becktel, J., Singleton, J. et al. Development of a Crohns disease activity index. National Cooperative Crohn's Disease Study. Gastroenterology. 1976; 70: 439–444
7. Mary JY, Modigliani R. Development and validation of an endoscopic index of the severity for Crohn’s disease: a prospective multicentre study. Groupe d’Etudes Therapeutiques des Affections Inflammatoires du Tube Digestif [GETAID]. Gut 1989;30:983–9.
8. Daperno M, D’Haens G, Van Assche G, et al. Development and validation of a new, simplified endoscopic activity score for Crohn’s disease: the SES-CD. Gastrointest Endosc 2004;60:505–12.

What is the purpose of the analysis being proposed? Please select all that apply.: 
News research question to examine treatment effectiveness on secondary endpoints and/or within subgroup populations
Research that confirms or validates previously conducted research on treatment effectiveness
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

In SONIC, patients underwent a colonoscopy at baseline prior to randomization to treatment and mucosal ulcerations were detected in 325 patients. At week 26, of patients with mucosal ulcerations on baseline colonoscopy, a repeat colonoscopy was performed to determine EH. One of the main outcome measures will be EH at week 26. EH was defined as the absence of mucosal ulceration at week 26 in patients who had confirmed ulceration at baseline.

Additionally, CR at 26 will be an outcome measure defined as a Crohn’s Disease Activity Index (CDAI) score < 1506. Corticosteroid-free CR at week 26 will be measured as well, which is defined as CDAI score <150, budesonide <6mg/day, and no systemic corticosteroids in prior 3 weeks.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

To establish the baseline mucosal lesion characteristics and extent of inflammation of disease activity, multiple scoring systems derived from endoscopy (Crohn’s Disease Endoscopic Index of Severity [CDEIS]7 and Simple Endoscopic Score for Crohn’s Disease [SES-CD]8) will be used. Further description of CDEIS and SES-CD will be outlined below in ‘other variables of interest.’

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

The CDEIS scores six endoscopic variables [presence of deep ulcers, superficial ulcers, nonulcerated stenosis, and ulcerated stenosis; proportion of ulcerated surfaces; and surface involved by disease] that are assessed in each of five ileocolonic segments [rectum, sigmoid/left colon, transverse colon, right colon, and ileum] 7.

The SES-CD is a simple scoring system based on four endoscopic variables [presence and size of ulcers, proportion of surface covered by ulcers, proportion of affected surface, and presence and severity of stenosis] measured in the same five ileocolonic segments as the CDEIS8.

Statistical Analysis Plan: 

Continuous variables will be presented as means (and standard deviations [SD] or as medians and interquartile ranges [IQR]) if the distribution is skewed, and categorical or binary variables will be presented as proportions or percentages. Descriptive statistics will be used to summarize baseline demographics, disease characteristics and outcome parameters of CD patients with baseline mucosal lesions at screening colonoscopy. Proportions of patients achieving EH, CR, and corticosteroid-free CR will be compared between treatments with the use of the Fisher’s exact test.

Known baseline disease factors significantly associated with the different composite remission outcomes (i.e. EH, CR, and corticosteroid-free CR) at week 26 such as treatment allocation and disease duration will be included within a multivariate logistic regression analysis to help adjust for potential confounding factors.

Approved investigators will be granted access to participant-level data sets via a remote, secure, password-protected data sharing platform. All work on the data must take place within the secure platform. The platform will be easily accessible to researchers, and ongoing system monitoring and support will be available. Within the platform, researchers will have access to the following analytical tools: Stata, R, RStudio, and Open Office. If needed, researchers will be able to upload additional data sets to the secure platform, if the researcher has the rights/license to do so.

How did you learn about the YODA Project?: 
Software Used: 
Open Office
Associated Trials: 
<ol><li><a href="/node/159">NCT00094458 - C0168T67 - Multicenter, Randomized, Double-Blind, Active Controlled Trial Comparing REMICADE® (infliximab) and REMICADE plus Azathioprine to Azathioprine in the Treatment of Patients with Crohn’s Disease Naive to both Immunomodulators and Biologic Therapy (Study of Biologic and Immunomodulator Naive Patients in Crohn’s Disease)</a></li></ol>
Make Publicly Available : 
Year of Data Access: 
2019

2019-3978

Project Title: 
Safety of Risperidone and Paliperdone in schizophrenia and bipolar disorder diagnosed patients - a systematic review and meta-analysis
Specific Aims of the Project: 

The aims of the project are to examine whether the antipsychotic drugs Risperidone and Paliperdone increase the risk of SAEs for patients suffering from schizophrenia and other mental health problems like bipolar disorder, and to determine whether treatment-related factors are associated with their occurrence.

Main hypothesis: There is an overall significant difference in serious adverse events in the Risperidone or Paliperdone group compared to placebo group.

The same hypothesis test will be used for all identified serious adverse events.

Subgroup analysis will include diagnostic subgroup, age (under 18s), gender, drug combination, dosage from the patient safety listings.

Please find attached PROSPERO protocol (CRD42019140556) for further information.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation
Associated Trial(s): 

Application Status

Ongoing
Scientific Abstract: 

Background: Risperidone and Paliperidone are two mainstay anti-psychotic drugs for treating schizophrenia and other mental health problems like bipolar disorder. However, among the research community, there is rising concerns about serious adverse events such as 'gynecomastia' and rare muscle related side-effects ‘extrapyramidal effects’.

Objective: As existing evidence about the safety of both drugs is based upon data from journal publications, which likely to lead to under reporting of harms. We aim to do a more robust meta-analysis using CSRs.

Study Design: We will carry out a robust and exhaustive systematic review including a large meta-analysis of RCTs to evaluate the safety of risperidone and paliperidone for use in patients with schizophrenia or bipolar disorder.

Participants: Participants of RCTs of risperidone or paliperidone irrespective of dose, age or gender and involving patient populations with schizophrenia and bipolar disorder.

Main Outcome Measures: Serious adverse events or adverse events and death related incidences. Patient safety narratives and listings will be used to assess causality.

Statistical Analysis: Relative risks, risk differences and their 95% confidence intervals will be calculated and combined in traditional pairwise meta-analysis. Sensitivity analysis will also be performed using Peto-OR, and more advance methods such as the beta-binomial model and Bayesian meta-analysis which are considered better for handling heterogeneity when the event rate is rare.

Please find attached PROSPERO protocol (CRD42019140556)

Brief Project Background and Statement of Project Significance: 

Risperidone and Paliperdone are mainstay treatment for people with schizophrenia and bipolar disorder. However, amongst the research community, there have been rising concerns about serious adverse events such as 'gynecomastia' and 'extrapyramidal effects' that have been linked with the use of Risperidone and Paliperdone. The current evidence on the safety of both drugs is based upon data from journal publications, which are susceptible to high levels of reporting bias and publication bias. Clinical study reports offer an untapped source of data and are far better suited to assess the safety profiles of pharmacological interventions. Therefore, to reach a level of precision and confidence about these serious adverse events and rare outcomes, a more robust meta-analysis using CSRs is required. We plan to achieve this goal by using the data from CSRs on Risperidone and Paliperdone trials available at YODA, and by making additional freedom of information requests at the European medicines agency (EMA).

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

Study design: Systematic review and meta-analysis of RCTs.

Search strategy: We ran extensive searches in the electronic databases Cochrane Schizophrenia Groups Trials Register and CENTRAL, MEDLINE, EMBASE, BIOSIS, CINAHL, LILACS and PsycINFO. Additionally, we contacted all risperidone and paliperdone-marketing pharma companies for missing relevant data. The ‘ClinicalTrials.gov’ and ‘OpenTrials.net’ will be searched to identify any potential unpublished trials. Medical Reviews at the Drugs@FDA and European Public Assessment Reports were checked for any further missing data. For trials that were not accessible via YODA, the CSRs were request via the EMA.

Inclusion criteria: Participants of randomised controlled trials of risperidone or paliperdone treated for schizophrenia and bipolar disorder.

Please find attached PROSPERO protocol (CRD42019140556) for further information.

Narrative Summary: 

Risperidone and Paliperdone are antipsychotic-drugs approved for the treatment of schizophrenia in adults and adolescents, and for the short-term treatment of manic or mixed episodes of bipolar disorder. However, over the last decade there have been a rising number of cases of hormonal imbalances leading to breast tissue development and infertility in boys and girls. To date, meta-analysis of both drugs in schizophrenic patients have solely been based on published RCTs, involving adults, and analyzed using standard methods of meta-analysis. We propose a more robust assessment of the safety of both drugs, using more innovative methodologies involving Clinical Study Reports (CSRs).

Project Timeline: 

Start of project: 05/2019
First contact of data holders: 06/2019
Actual state of project: Identification of included RCTs from literature search and reported SAEs.

It is planned, that the data extraction and statistical analysis will start by 04/2020

Conference presentations and publication drafts are planned for the preceding months.

Dissemination Plan: 

We are performing a very large systematic review involving over 60,000 participants with a robust meta-analysis incorporating CSRs, narratives, patient safety listings and CRFs. The research question is a priory for patients with schizophrenia and bipolar indications and is in line with recent NIHR health technology assessment funding calls to research the safety of anti-psychotic interventions. https://www.nihr.ac.uk/funding-and-support/funding-opportunities/1941-cl.... Therefore, we anticipate that we would look to publish our results in a leading medical journal such as the BMJ or the Lancet in which collaborators with this study have already published. Furthermore, we expect our findings would be translated and implemented into national and international treatment guidelines and through policy involvement with mental health (with specific focus on schizophrenia and bipolar disorder).

Bibliography: 

• References related to risperidone and pailiperdone with the indications of interest:

1. Komossa, K., et al., Risperidone versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst Rev, 2011(1): p. Cd006626.
2. Leucht, S., et al., Second-generation versus first-generation antipsychotic drugs for schizophrenia: a meta-analysis. The Lancet. 373(9657): p. 31-41.
3. Alvir , J.M.J., et al., Clozapine-Induced Agranulocytosis -- Incidence and Risk Factors in the United States. New England Journal of Medicine, 1993. 329(3): p. 162-167.
4. Edwards, J.G., Risperidone for schizophrenia. BMJ, 1994. 308(6940): p. 1311-1312.
5. Bishop, J.R. and M.N. Pavuluri, Review of risperidone for the treatment of pediatric and adolescent bipolar disorder and schizophrenia. Neuropsychiatr Dis Treat, 2008. 4(1): p. 55-68.
6. Picchioni, M.M. and R.M. Murray, Schizophrenia. BMJ, 2007. 335(7610): p. 91-95.
7. Anderson, I.M., P.M. Haddad, and J. Scott, Bipolar disorder. BMJ, 2012. 345.
8. Samara, M.T., et al., Efficacy, Acceptability, and Tolerability of Antipsychotics in Treatment-Resistant Schizophrenia: A Network Meta-analysis. JAMA Psychiatry, 2016. 73(3): p. 199-210.
9. Leucht, S., et al., Second-generation versus first-generation antipsychotic drugs for schizophrenia: a meta-analysis. Lancet, 2009. 373(9657): p. 31-41.
10. Kay, S.R., A. Fiszbein, and L.A. Opler, The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull, 1987. 13(2): p. 261-76.
11. Woods, S.W., et al., Effects of Development on Olanzapine-Associated Adverse Events. Journal of the American Academy of Child & Adolescent Psychiatry, 2002. 41(12): p. 1439-1446.
12. Stanniland, C. and D. Taylor, Tolerability of atypical antipsychotics. Drug Saf, 2000. 22(3): p. 195-214.
13. Drugwatch. Risperdal: Schizophrenia, Bipolar ADHD Drug uses & side effects. Last accessed 20th October 2016. Available at: https://www.drugwatch.com/risperdal/.
14. Wang, B., et al., Did FDA Decisionmaking Affect Anti-Psychotic Drug Prescribing in Children?: A Time-Trend Analysis. PLoS ONE, 2016. 11(3): p. e0152195.
15. Gilbody, S.M., et al., Risperidone versus other atypical antipsychotic medication for schizophrenia. Cochrane Database Syst Rev, 2000(3): p. Cd002306.
16. Hughes, S., D. Cohen, and R. Jaggi, Differences in reporting serious adverse events in industry sponsored clinical trial registries and journal articles on antidepressant and antipsychotic drugs: a cross-sectional study. BMJ Open, 2014. 4(7).
17. Carpenter , W.T.J. and R.W. Buchanan Schizophrenia. New England Journal of Medicine, 1994. 330(10): p. 681-690.

• Methodological references when using clinical study reports:

1. Jones M, Jefferson T, Doshi P, Hodkinson A et al. When to include clinical study reports and regulatory documents in systematic reviews. BMJ Evidence-Based Medicine 2018; 23:210-217. http://dx.doi.org/10.1136/bmjebm-2018-110963
2. Hodkinson A, Jefferson T, Doshi P, Heneghan C. The use of clinical study reports to enhance the quality of systematic reviews: A survey of systematic review authors. Systematic Reviews (2018) 7:117 https://doi.org/10.1186/s13643-018-0766-x
3. Hodkinson A, Gamble C, Tudur Smith C. Reporting of harms outcomes: A comparison of journal publications with unpublished clinical study reports of orlistat trials. Trials 2016 17;207. https://doi.org/10.1186/s13063-016-1327-z
4. Rodgers Mark A, Brown Jennifer V E, Heirs Morag K, Higgins Julian P T, Mannion Richard J, Simmonds Mark C et al. Reporting of industry funded study outcome data: comparison of confidential and published data on the safety and effectiveness of rhBMP-2 for spinal fusion BMJ 2013; 346 :f3981
5. Jefferson T, Doshi P, Hodkinson A, et al. (2018). Interim guidance on the inclusion of clinical study reports and other regulatory documents in Cochrane Reviews – Interim report. Cochrane methods innovation fund 2. Available: https://methods.cochrane.org/methods-innovation-fund-2
6. Jefferson Tom, Jones Mark, Doshi Peter, Spencer Elizabeth A, Onakpoya Igho, Heneghan Carl J et al. Oseltamivir for influenza in adults and children: systematic review of clinical study reports and summary of regulatory comments BMJ 2014; 348 :g2545
7. Doshi P, Jefferson T.Clinical study reports ofrandomised controlled trials:an exploratory review ofpreviously confidential industryreports.BMJOpen2013;3:e002496. doi:10.1136/bmjopen-2012-002496
8. Eyding Dirk, Lelgemann Monika, Grouven Ulrich, Härter Martin, Kromp Mandy, Kaiser Thomas et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials BMJ 2010; 341 :c4737
9. Maund Emma, Tendal Britta, Hróbjartsson Asbjørn, Jørgensen Karsten Juhl, Lundh Andreas, Schroll Jeppe et al. Benefits and harms in clinical trials of duloxetine for treatment of major depressive disorder: comparison of clinical study reports, trial

What is the purpose of the analysis being proposed? Please select all that apply.: 
New research question to examine treatment safety
Summary-level data meta-analysis:
Summary-level data meta-analysis will pool data from YODA Project with other additional data sources
Participant-level data meta-analysis:
Participant-level data meta-analysis uses only data from YODA Project
Supplementary Material: 
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

The main outcome measure is the number(s) of serious adverse events in the treatment group and placebo group. The effect size measure will be the odd ratio, relative risk, risk difference and its 95% confidence intervals. We will calculate the number needed to treat to provide benefit/to induce harm, and its 95% CIs. All serious adverse events of interest will be assessed in the meta-analysis. In addition, rare adverse events will be analyzed in a sensitivity analysis involving more advanced methods including Peto-odds ratio, and more advanced methods like the beta-binomial model and Bayesian meta-analysis.

Longer-term outcomes will be assessed in a sensitivity analysis with the trials that had greater length of follow-up. Incidence rates will be calculated if the mean times are available.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

The state of the treatment (risperidone, paliperdone or placebo) will be the main predictor.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

Other potential predictors that will be addressed in the subgroup analysis include diagnostic subgroup (schizophrenia/bipolar), age (younger children or adolescents), gender, risperidone vs. paliperdone, combination of other drugs and dosage.

Statistical Analysis Plan: 

Because of the novelty and size of clinical study reports (including appendices listing data) we subdivided the extraction, appraisal, and analysis of the data into a two stage exercise. We included trials meeting our inclusion criteria (that is, had an appropriate study design) in stage 1. Trials not meeting our inclusion criteria (for example, open label studies) were not included in stage 1. In stage 1 we assessed the reliability and completeness of the identified trial data. This allowed us to identify missing important text or data. To aid us in determining completeness of the relevant parts of clinical study reports we constructed an extraction form based on the CONSORT-harms statement checklist and expert opinion from the research team.

We decided to only include data in stage 2 of the review (full analysis following standard Cochrane methods) if they satisfied the following three criteria:
1. Completeness: clinical study reports include identifiable CONSORT harms statement specified methods to enable replication of the study. Identifiable CONSORT harms statement specified results (safety results in the core report, tables of adverse and serious adverse events, appendices with serious adverse event narratives (E3 sections 12.3.1, 12.3.2 & 14.3.3) and individual participant safety listings (E3 section 16.2.7) and CRFs for SAEs and withdrawals for AEs (E3 Section 16.3.1)) should be available. A comparison table checklist will be used to support this decision.
2. Internal consistency: all parts (for example, denominators) of the same clinical study reports or unpublished reports are consistent.
3. External consistency: consistency of data as reported in regulatory documents, other versions of the same clinical study reports or unpublished reports, and other references, established by cross-checking.

The analysis will become clearer after stage 1 when we have assessed the state of the reports. An initial plan is outlined below:

Adverse events and Serious adverse events will be assessed by pooling the relative risk (RR) across trials. Effect estimates will be pooled across trials using Mantel-Haenszel fixed or random-effect meta-analysis dependent upon the number of studies reporting the outcome of interest. If there are less than five trials reporting the outcome, then we will use the fixed-effect approach as recommended in the Cochrane handbook. Initial sensitivity analysis was also performed pooling the relative difference instead of RR for rare events (Bradburn et al 2007, Sweating et al 2002). However, because adverse events are likely to be sparse, we will include the peto-odds ratio approach as this has been found to be more effective method for analysing rare event outcomes. We will also calculate the number needed to treat to provide benefit/to induce harm, and its 95% CIs.

Heterogeneity was assessed visually in the forest plots and the I² statistics will be compared between the CSR-based and the journal publication-based analyses to determine the magnitude of heterogeneity. I² values greater than 50% we interpreted as considerable levels of heterogeneity. Publication bias will be examined with funnel-plots (trim-and-fill), and the Cochrane risk of bias and GRADE assessment tool will be used to assess the quality of the studies.

How did you learn about the YODA Project?: 
Software Used: 
RStudio
Associated Trials: 
<ol><li><a href="/node/175">NCT00518323 - R076477PSZ3001 - A Randomized, Multicenter, Double-Blind, Weight-Based, Fixed-Dose, Parallel-Group, Placebo-Controlled Study of the Efficacy and Safety of Extended Release Paliperidone for the Treatment of Schizophrenia in Adolescent Subjects, 12 to 17 Years of Age</a></li><li><a href="/node/178">NCT00334126 - R076477SCH3015 - A Randomized, Double-blind, Placebo-controlled, Parallel Group Study to Evaluate the Efficacy and Safety of Paliperidone ER Compared to Quetiapine in Subjects With an Acute Exacerbation of Schizophrenia</a></li><li><a href="/node/179">NCT00086320 - R076477-SCH-301 - A Randomized, Double-blind, Placebo-controlled, Parallel-group Study With an Open-label Extension Evaluating Paliperidone Extended Release Tablets in the Prevention of Recurrence in Subjects With Schizophrenia</a></li><li><a href="/node/190">NCT00590577 - R092670PSY3007 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Dose Response Study to Evaluate the Efficacy and Safety of 3 Fixed Doses (25 mg eq., 100 mg eq., and 150 mg eq.) of Paliperidone Palmitate in Subjects With Schizophrenia</a></li><li><a href="/node/191">NCT00111189 - R092670PSY3001 - A Randomized Double-blind Placebo-controlled Parallel Group Study Evaluating Paliperidone Palmitate in the Prevention of Recurrence in Patients With Schizophrenia. Placebo Consists of 20% Intralipid (200 mg/mL) Injectable Emulsion</a></li><li><a href="/node/194">NCT00210548 - R092670PSY3003 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Dose-Response Study to Evaluate the Efficacy and Safety of 3 Fixed Doses (50 mg eq., 100 mg eq., and 150 mg eq.) of Paliperidone Palmitate in Subjects With Schizophrenia</a></li><li><a href="/node/195">NCT00101634 - R092670PSY3004 - A Randomized, Double-blind, Placebo-controlled, Parallel-group, Dose-response Study to Evaluate the Efficacy and Safety of 3 Fixed Doses (25 mg eq, 50 mg eq, and 100 mg eq) of Paliperidone Palmitate in Patients With Schizophrenia</a></li><li><a href="/node/196">NCT00391222 - RISBMN3001 - A Randomized, Double Blind, Placebo and Active Controlled Parallel Group Study to Evaluate the Efficacy and Safety of Risperidone Long-acting Injectable (LAI) for the Prevention of Mood Episodes in the Treatment of Subjects With Bipolar I Disorder</a></li><li><a href="/node/198">NCT00076115 - RIS-BIM-301 - Research on the Effectiveness of Risperidone in Bipolar Disorder in Adolescents and Children (REACH): A Double-Blind, Randomized, Placebo-Controlled Study of the Efficacy and Safety of Risperidone for the Treatment of Acute Mania in Bipolar I Disorder</a></li><li><a href="/node/199">NCT00132678 - RISBIM3003 - A Randomized, Double-blind, Placebo-controlled Study to Explore the Efficacy and Safety of Risperidone Long-acting Intramuscular Injectable in the Prevention of Mood Episodes in Bipolar 1 Disorder, With Open-label Extension</a></li><li><a href="/node/200">NCT00094926 - RIS-BIP-302 - A Prospective, Randomized, Double-blind, Placebo-controlled Study of the Effectiveness and Safety of RISPERDAL CONSTA Augmentation in Adult Patients With Frequently-relapsing Bipolar Disorder</a></li><li><a href="/node/296">NCT00397033 - R076477SCA3001 - A Randomized, Double-blind, Placebo-controlled, Parallel-group Study to Evaluate the Efficacy and Safety of Two Dosages of Paliperidone ER in the Treatment of Patients With Schizoaffective Disorder</a></li><li><a href="/node/297">NCT00412373 - R076477SCA3002 - A Randomized, Double-blind, Placebo-controlled, Parallel- Group Study to Evaluate the Efficacy and Safety of Flexible-dose Paliperidone ER in the Treatment of Patients With Schizoaffective Disorder</a></li><li><a href="/node/576">NCT00253162 - RIS-INT-69 - The Efficacy And Safety Of Flexible Dose Ranges Of Risperidone Versus Placebo Or Haloperidol In The Treatment Of Manic Episodes Associated With Bipolar I Disorder</a></li><li><a href="/node/622">NCT00299715 - R076477-BIM-3001 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Dose-Response, Multicenter Study to Evaluate the Efficacy and Safety of Three Fixed Doses of Extended-Release Paliperidone in the Treatment of Subjects With Acute Manic and Mixed Episodes Associated With Bipolar I Disorder</a></li><li><a href="/node/623">NCT00309699 - R076477-BIM-3002 - A Randomized, Double-Blind, Active- and Placebo-Controlled, Parallel-Group, Multicenter Study to Evaluate the Efficacy and Safety of Flexibly-Dosed, Extended-Release Paliperidone Compared With Flexibly-Dosed Quetiapine and Placebo in the Treatment of Acute Manic and Mixed Episodes Associated With Bipolar I Disorder</a></li><li><a href="/node/624">NCT00309686 - R076477-BIM-3003 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Multicenter Study to Evaluate the Efficacy and Safety of Flexibly-Dosed Extended-Release Paliperidone as Adjunctive Therapy to Mood Stabilizers in the Treatment of Acute Manic and Mixed Episodes Associated With Bipolar I Disorder</a></li><li><a href="/node/625">NCT00752427 - R076477-SCH-702 - 24 week extension of NCT00085748: A Randomized, 6-Week Double-Blind, Placebo-Controlled Study With an Optional 24-Week Open-Label Extension to Evaluate the Safety and Tolerability of Flexible Doses of Paliperidone Extended Release in the Treatment of Geriatric Patients With Schizophrenia</a></li><li><a href="/node/626">NCT00077714 - R076477-SCH-304 - A Randomized, Double-blind, Placebo- and Active-controlled, Parallel-group, Dose-response Study to Evaluate the Efficacy and Safety of 2 Fixed Dosages of Paliperidone Extended Release Tablets and Olanzapine, With Open-label Extension, in the Treatment of Patients With Schizophrenia</a></li><li><a href="/node/627">NCT00083668 - R076477-SCH-305 - A Randomized, Double-blind, Placebo- and Active-controlled, Parallel-group, Dose-response Study to Evaluate the Efficacy and Safety of 3 Fixed Dosages of Paliperidone Extended Release (ER) Tablets and Olanzapine, With Open-label Extension, in the Treatment of Patients With Schizophrenia</a></li><li><a href="/node/628">NCT00074477 - R092670-SCH-201 - A Randomized, Double-Blind, Placebo-Controlled Study to Evaluate the Efficacy and Safety of 50 and 100 Mg-eq of Paliperidone Palmitate in Patients With Schizophrenia</a></li><li><a href="/node/638">NCT00078039 - R076477-SCH-303 - Trial Evaluating Three Fixed Dosages of Paliperidone Extended-Release (ER) Tablets and Olanzapine in the Treatment of Patients With Schizophrenia</a></li><li><a href="/node/704">NCT00085748 - R076477-SCH-302 - A Randomized, 6-Week Double-Blind, Placebo-Controlled Study With an Optional 24-Week Open-Label Extension to Evaluate the Safety and Tolerability of Flexible Doses of Paliperidone Extended Release in the Treatment of Geriatric Patients With Schizophrenia</a></li><li><a href="/node/857">NCT00088075 - RIS-SCH-302/CR003370 - A Randomized, Double-Blind, Placebo-Controlled Clinical Study of the Efficacy and Safety of Risperidone for the Treatment of Schizophrenia in Adolescents</a></li><li><a href="/node/859">NCT00253149 - RIS-USA-102/CR006040 - The Safety And Efficacy Of Risperdal (Risperidone) Versus Placebo Versus Haloperidol As Add-On Therapy To Mood Stabilizers In The Treatment Of The Manic Phase Of Bipolar Disorder</a></li><li><a href="/node/860">NCT00253136 - RIS-USA-121/CR006055 - Risperidone Depot (Microspheres) vs. Placebo in the Treatment of Subjects With Schizophrenia</a></li><li><a href="/node/863">NCT00257075 - RIS-USA-239/CR006052 - The Efficacy And Safety Of Flexible Dosage Ranges Of Risperidone Versus Placebo In The Treatment Of Manic Episodes Associated With Bipolar I Disorder</a></li><li><a href="/node/866">RIS-USA-72 - The safety and efficacy of risperidone 8 mg qd and 4 mg qd compared to placebo in the treatment of schizophrenia</a></li><li><a href="/node/867">NCT01529515 - R092670PSY3012  - A Randomized, Multicenter, Double-Blind, Relapse Prevention Study of Paliperidone Palmitate 3 Month Formulation for the Treatment of Subjects With Schizophrenia</a></li><li><a href="/node/868">NCT01193153 - R092670SCA3004 - A Randomized, Double-Blind, Placebo-Controlled, Parellel-Group Study of Paliperidone Palmitate Evaluating Time to Relapse in Subjects With Schizoaffective Disorder </a></li><li><a href="/node/869">NCT01662310 - R076477-SCH-3041 - Paliperidone Extended Release Tablets for the Prevention of Relapse in Subjects With Schizophrenia: A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group Study</a></li><li><a href="/node/870">NCT00490971 - R076477BIM3004 - A Randomized, Double-Blind, Active- and Placebo-controlled, Parallel-group, Multicenter Study to Evaluate the Efficacy and Safety of Extended-Release Paliperidone as Maintenance Treatment After an Acute Manic or Mixed Episode Associated With Bipolar I Disorder</a></li><li><a href="/node/872">NCT00105326 - R076477-SCH-1010/CR002281 - A Double-blind, Placebo-controlled, Randomized Study Evaluating the Effect of Paliperidone ER Compared With Placebo on Sleep Architecture in Subjects With Schizophrenia</a></li><li><a href="/node/1032">NCT00645307 - R076477-SCH-701 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group Study With an Open-Label Extension Evaluating Extended Release OROS® Paliperidone in the Prevention of Recurrence in Subjects With Schizophrenia - Open Label Phase</a></li><li><a href="/node/3772">NCT01299389 - PALM-JPN-4 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Fixed-Dose, Multicenter Study of JNS010 (Paliperidone Palmitate) in Patients With Schizophrenia</a></li><li><a href="/node/3807">NCT00236379 - RIS-USA-275 - A Six-month, Double-blind, Randomized, International, Multicenter Trial to Evaluate the Glucoregulatory Effects of Risperidone and Olanzapine in Subjects With Schizophrenia or Schizoaffective Disorder</a></li><li><a href="/node/3853">NCT00992407 - RISSCH4178 - A Randomized, Open-label, Active-controlled Study to Evaluate Social Functioning of Long Acting Injectable Risperidone and Oral Risperidone in the Treatment of Subjects With Schizophrenia or Schizoaffective Disorder</a></li><li><a href="/node/3857">NCT00236457 - RIS-INT-62 - Randomized, Multi-center, Open Label Trial Comparing Risperidone Depot (Microspheres) and Olanzapine Tablets in Patients With Schizophrenia or Schizoaffective Disorder</a></li><li><a href="/node/3974">NCT00061802 - RIS-SCP-402 - A Randomized, Double Blind Study to Evaluate the Efficacy and Safety of Two Atypical Antipsychotics vs. Placebo in Patients With an Acute Exacerbation of Either Schizophrenia or Schizoaffective Disorder</a></li></ol>
Make Publicly Available : 
Year of Data Access: 
2019

2019-3943

Project Title: 
Defining a therapeutic drug window for infliximab induction therapy in pediatric patients with moderate-to-severe Crohn’s disease
Specific Aims of the Project: 

Specific Aim 1:
To investigate the association between serum infliximab concentration at weeks 2, 6 and 10 with clinical remission at week 10.

Specific Aim 2:
To investigate the association between serum infliximab concentration at weeks 2, 6 and 10 with primary non-response at week 10.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Ongoing
Scientific Abstract: 

Background: Infliximab is an effective treatment for Crohn’s disease (CD). Recent exposure-response relationship studies have revealed a positive correlation between high serum infliximab concentration and favorable therapeutic outcomes, although there are limited data regarding induction therapy and pediatric patients with CD.
Objective: To define the therapeutic window for adequate serum infliximab concentration associated with clinical response and remission following induction therapy in pediatric patients with moderate-to-severe CD.
Study Design: Post-hoc analysis of the REACH randomized controlled trial.
Participants: Patients with moderate-to-severe CD who received infliximab induction therapy (5mg/Kg at weeks 0, 2 and 6) (n=112).
Main outcome measure(s): Association between infliximab concentration at weeks 2, 6 and 10 with primary non-response (defined as lack of clinical response) or clinical remission assessed at week 10.
Statistical Analysis: Descriptive statistics will be provided with medians and interquartile range for continuous variables and frequency and percentage for categorical variables. A receiver operating characteristic analysis will be performed for infliximab concentrations to trace thresholds associated with outcomes of interest. Infliximab concentrations will be compared between groups with the Mann-Whitney U and Kruskal Wallis test, as appropriate. Univariate and multivariate analyses will be performed to identify variables associated with outcomes of interest.

Brief Project Background and Statement of Project Significance: 

Two pivotal randomized-controlled trials clearly showed high clinical response/remission rates after induction infliximab treatment in pediatric patients with inflammatory bowel disease (IBD). [1, 2] Serum infliximab concentrations have been related to favorable therapeutic outcomes in IBD, such as clinical, biochemical and endoscopic response. [3-7] Nevertheless, there are limited data on the therapeutic window and the role of therapeutic drug monitoring during induction infliximab therapy in pediatric patients with IBD. As pharmacological treatment options in pediatric patients with IBD remain limited, emphasis has to be given to rational decision-making and optimization of therapies utilizing a therapeutic drug monitoring (TDM)-based therapeutic approach. This project by defining the adequate drug concentration for better therapeutic outcomes can be the first step for the application of reactive and proactive TDM towards a more personalized infliximab therapy in pediatric patients with moderate-to-severe CD. This could potentially improve care and reduce the substantial social and economic burden to the community by preventing future CD-related hospitalizations and surgeries.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

Post-hoc analysis of the REACH RCT regarding patients who received induction infliximab therapy (n=112). [1]

Narrative Summary: 

Two pivotal randomized-controlled trials clearly showed high clinical response/remission rates after induction infliximab treatment in pediatric patients with inflammatory bowel disease (IBD). Serum infliximab concentrations have been related to favorable therapeutic outcomes in IBD, such as clinical, biochemical and endoscopic response. Nevertheless, there are limited data on the therapeutic window and the role of therapeutic drug monitoring during induction infliximab therapy in pediatric patients with IBD. The aim of the study is to investigate the association between serum infliximab concentrations and clinical response or remission in patients with moderate-to-severe Crohn’s disease.

Project Timeline: 

It is estimated that it will take 6-7 months to review the appropriate data. Statistical analyses will take another 2-3 months, while manuscript preparation will take approximately another 1-2 months. Consequently, the whole project will be completed in 9-12 months.

Dissemination Plan: 

Presentation of the results to national and international medical congresses including Digestive Disease Week (DDW), Advances in IBD (AIBD), American College of Gastroenterology (ACG), European Crohn’s and Colitis Organization (ECCO) and publication of the data in a high impact medical journal such as the American Journal of Gastroenterology, Clinical Gastroenterology and Hepatology, or the Journal of Crohn’s and Colitis.

Bibliography: 

1. Hyams J, Crandall W, Kugathasan S, et al. Induction and maintenance infliximab therapy for the treatment of moderate-to-severe Crohn's disease in children. Gastroenterology. 2007;132:863-73.
2. Hyams J, Damaraju L, Blank M, et al. Induction and maintenance therapy with infliximab for children with moderate to severe ulcerative colitis. Clin Gastroenterol Hepatol. 2012;10:391-9.
3. Adedokun OJ, Xu Z, Padgett L, et al. Pharmacokinetics of infliximab in children with moderate-to-severe ulcerative colitis: results from a randomized, multicenter, open-label, phase 3 study. Inflamm Bowel Dis 2013;19:2753-62.
4. Papamichael K, Rakowsky S, Rivera C, et al. association between serum infliximab trough concentrations during maintenance therapy and biochemical, endoscopic, and histologic remission in Crohn's disease. Inflamm Bowel Dis 2018;24:2266-2271
5. Papamichael K, Rakowsky S, Rivera C, et al. Infliximab trough concentrations during maintenance therapy are associated with endoscopic and histologic healing in ulcerative colitis. Aliment Pharmacol Ther. 2018 Feb;47(4):478-484.
6. Adedokun OJ, Sandborn WJ, Feagan BG, et al. Association between serum concentration of infliximab and efficacy in adult patients with ulcerative colitis. Gastroenterology 2014;147:1296-1307.
7. Singh N, Rosenthal CJ, Melmed GY, et al. Early infliximab trough levels are associated with persistent remission in pediatric patients with inflammatory bowel disease. Inflamm Bowel Dis 2014;20(10):1708-13.

What is the purpose of the analysis being proposed? Please select all that apply.: 
News research question to examine treatment effectiveness on secondary endpoints and/or within subgroup populations
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

• Main outcome measures of interest include:
1. Clinical remission, defined as a pediatric Crohn’s disease activity index (PCDAI)≤10 at week 10.
2. Primary non-response, defined as lack of clinical response (decrease from baseline to week 10 in the total PCDAI score of at least 15 points and a total PCDAI score of no more than 30 points at week 10).
• Secondary outcome measures of interest include:
1. Change from baseline in quality of life based on the IMPACT III Questionnaire score at week 10. (The IMPACT III scores range from 35 to 175, with higher scores indicating better quality of life).
2. Change from baseline of the erythrocyte sedimentation rate (ESR) at week 10.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

Main predictor/independent variables associated with outcomes of interest include:
• Serum infliximab concentrations at weeks 2, 6 and 10 associated with outcomes of interest.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

Other variables associated with outcomes of interest include:
• gender
• race
• involved GI area
• age
• concomitant corticosteroids at baseline
• concomitant immunomodulators (thiopurines/methotrexate) at baseline
• disease duration
• weight
• height
• PCDAI at baseline
• ESR at baseline

Statistical Analysis Plan: 

Descriptive statistics will be provided with medians and interquartile range (IQR) for continuous variables and frequency and percentage for categorical variables. A receiver operating characteristic (ROC) analysis will be performed for infliximab concentrations to trace thresholds associated with outcomes of interest. Optimal thresholds will be chosen by using the Youden index, which maximizes the sum of the specificity (SP) and sensitivity (SN) of the ROC curve. SN, SP, positive predictive value, and negative predictive value will be also calculated. Infliximab concentrations at weeks 2, 6 and 10 will be compared between groups with the Mann-Whitney U test. Serum infliximab concentrations will be categorized also into quartiles. Rates of clinical remission and primary non-response at week 10 will be compared across infliximab serum concentration quartiles with the chi-square test (linear-by-linear association). The Kruskal-Wallis and the chi-square test will be used to compare continuous or discrete variables, respectively, across quartile groups. Univariate and multivariate logistic regression analyses will be performed to identify variables independently associated with outcomes of interest. The results will be expressed as odds ratio (OR) with 95% confidence intervals, followed by the corresponding P-value. Results will be considered statistically significant when P<0.05.

How did you learn about the YODA Project?: 
Software Used: 
STATA
Associated Trials: 
<ol><li><a href="/node/158">NCT00207675 - C0168T47 - A Randomized, Multicenter, Open-label Study to Evaluate the Safety and Efficacy of Anti-TNF a Chimeric Monoclonal Antibody (Infliximab, REMICADE) in Pediatric Subjects With Moderate to Severe CROHN'S Disease</a></li></ol>
Make Publicly Available : 
Year of Data Access: 
2019

2019-3941

Project Title: 
Factors moderating estimates of antipsychotic efficacy in schizophrenia
Specific Aims of the Project: 

The project has two overarching aims:
- To compare the psychometric performance of brief unidimensional subscales (PANSS-6 and BPRS-6) derived from commonly used schizophrenia rating scales (PANSS and BPRS). This will be done i) through Rasch analysis (item response theory analysis) investigating the scalability and transferability of the subscales and their comprehensive counterparts and ii) through contrasting drug-placebo separation between the unidimensional subscales and their multidimensional counterparts.
- To investigate the importance of putative patient-level effect modifiers (e.g., BMI, age, sex, dose, baseline symptom profile/severity, adverse event proneness) with regards to efficacy as measured by brief unidimensional subscales. To ensure that our results are comparable to previous investigations we well repeat all analyses using the full rating scales as effect parameters.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation
Associated Trial(s): 

Application Status

Ongoing
Scientific Abstract: 

Background Brief unidimensional subscales for schizophrenia have demonstrated superior psychometric properties as compared to full rating scales. They are also faster to administer and therefore more feasible to implement in clinical practice. In the current study we will compare the performance of two of these subscales to that of their comprehensive counterparts, as well as investigate to what extent patient-level predictors moderate the response to antipsychotic treatment, using patient-level data.
Objective i) To assess the performance of brief unidimensional subscales, BPRS-6 and PANSS-6, as compared to the full BPRS and PANSS rating scales in treatment trials in schizophrenia and ii) to investigate to what extent patient-level predictors (e.g., age, sex, baseline symptom profile) moderate treatment efficacy as measured by brief unidimensional subscales, single items, and full scales.
Study Design Individual patient-level data meta-analysis of acute-phase treatment trials in schizophrenia.
Participants Patients with schizophrenia treated with an established antipsychotic or placebo in acute-phase treatment trials.
Main Outcome Measure(s) Between-treatment contrasts of endpoint scores, and categorical derivates thereof (e.g., response), on brief unidimensional subscales (BPRS-6 and PANSS-6), single items and comprehensive rating scales (BPRS and PANSS).
Statistical Analysis Continuous outcome measures will be analyzed using Mixed Models for Repeated Measurements (MMRM) methodology, categorical outcome measures will be analyzed using generalized linear mixed models.

Brief Project Background and Statement of Project Significance: 

Schizophrenia is a severe mental disorder characterized by abnormalities in thinking, perception, emotions and social function (1, 2). These characteristics have a profound negative impact on those living with schizophrenia (3, 4), as underlined by the fact that life expectancy of individuals with schizophrenia is approximately 20 years shorter than that of the background population (5).

While pharmacological treatments for schizophrenia have existed for more than 60 years, prescription decisions largely follow a “trial and error”-methodology (6, 7). It thus often takes several failed trials before effective antipsychotic treatment is initiated. Considering the vast humanitarian and societal impact of schizophrenia (8), and that duration of untreated psychosis predicts poor long-term outcomes (9), this gap in knowledge is highly unsatisfactory.

One reason that may have contributed to the difficulty in implementing personalized medicine for schizophrenia is the multidimensional rating scales used to evaluate disease severity. The traditional take has been that a good rating scale should offer exhaustive syndromal coverage. This position, while reasonable, is not without drawbacks since many psychiatric symptoms (e.g., insomnia or hypersomnia, somatic and psychic anxiety, concentration difficulties, loss of energy, gastrointestinal symptoms or weight changes) are not illness-specific, and can also be side effects of treatment. Similarly, the clinical impact of different symptoms, measured by the patient as functional impairment, varies greatly even though the symptoms may contribute the same amount of points to the rating scale sum-score (10).

In schizophrenia, the rating scale most widely used as outcome measure in clinical studies is the Positive and Negative Syndrome Scale (PANSS) (11). It has been demonstrated, across several patient populations (acute-, chronic- and treatment-resistant schizophrenia), that the full 30-item PANSS is highly multidimensional (12-14). However, a unidimensional 6-item subscale of PANSS (PANSS-6), which covers the following core positive and negative symptoms of schizophrenia: delusions, conceptual disorganisation, hallucinations, blunted affect, passive/apathetic social withdrawal and lack of spontaneity & flow of conversation, has demonstrated solid psychometric properties (12-14), including an increased sensitivity to the beneficial effects of some antipsychotics compared to the full PANSS (12). Similar findings have been reported for a shortened version (BPRS-6) of the Brief Psychiatric Rating Scale (BPRS), which is also commonly used in schizophrenia trials (15).
We aim to evaluate the psychometric performance of the BPRS-6 and the PANSS-6 as compared to the full BPRS and PANSS, respectively. If this evaluation supports the shorter scales as being psychometrically superior, then we will assess the impact of individual level predictors (e.g., age, sex, BMI, baseline symptom severity/profile) of interest. We hope that by using unidimensional subscales – which should be more consistent across different patient populations – we will have better power to identify clinically relevant effect modifiers.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

The data used in this study will come from the clinical trial programs of risperidone and paliperidone, with data access being provided by YODA. We will include all double-blind, acute-phase, placebo- and/or actively controlled trials of these compounds in schizophrenia or schizoaffective disorder, which utilized either PANSS, or BPRS, or both. Studies will be included regardless of treatment duration or participant age. Comparison arms against other established antipsychotics (e.g., haloperidol. olanzapine) will be included whereas any comparator arms against non-antipsychotic drugs will be excluded.

Narrative Summary: 

Treatments for schizophrenia are assessed using rating scales. By comparing ratings between patients given different treatments (for example an antipsychotic or placebo) it is possible to assess which treatment is more efficacious.
In this study we wish to evaluate the performance of abridged versions (BPRS-6 and PANSS-6) of common rating scales for schizophrenia (BPRS and PANSS). We also aim to investigate if patient-level factors (e.g., age, sex, baseline symptom profile) moderate antipsychotic efficacy. The goals are i) to improve the rating of schizophrenia, thus bettering future clinical trials in this condition, and ii) to facilitate personalized medicine in schizophrenia.

Project Timeline: 

We anticipate that the project can start as soon as we get data access approved by YODA. After that we believe that we will need approximately 3-4 months of preparatory work to familiarize ourselves with the data sets, assess the feasibility of the various analyses, and prepare the final analysis set(s). We believe that the analyses for the first part of the project will take roughly one month, whereas those for the second part are estimated to take roughly two months. During this process we will continuously iterate a draft of the corresponding article. We expect that finalizing each manuscript will take roughly one month after all analysis work is completed. We plan to publish at least one paper per objective, and thus expect the first manuscript to be ready for publication approximately six months after gaining data access, and the second manuscript approximately nine months after obtaining data access. We will report our results to YODA immediately prior to submitting for publication/presentation.

Dissemination Plan: 

Publications stemming from the first part of the project are – due to the technical nature – likely to be primarily of interest for psychiatrists and researchers in psychiatry. As such, we will attempt to publish manuscripts stemming from this part firstly in high-impact specialist journals (e.g., American Journal of Psychiatry, Lancet Psychiatry, JAMA Psych, Molecular Psychiatry), and secondarily in lower ranked specialist journals.
Publications stemming from work related to the second objective (predictors of response and comparative efficacy of different antipsychotics) may be of importance for the general medical community. If any such findings do emerge we will pursue publication in a high-impact general medicine journal (e.g., NEJM, JAMA, The Lancet, BMJ, PLoS Med). If this is not feasible then we will follow the same publication plan as above.
We will further communicate our findings in symposia and poster form (if accepted) at relevant international conferences (2-3 times yearly), as well as through lectures and public events organized at our local universities (Aarhus university and the University of Gothenburg).

Bibliography: 

1. World Health O. The ICD-10 classification of mental and behavioural disorders : clinical descriptions and diagnostic guidelines. Geneva: World Health Organization; 1992.
2. Association AP. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5®). 2013.
3. Bobes J, Garcia-Portilla MP, Bascaran MT, Saiz PA, Bousono M. Quality of life in schizophrenic patients. Dialogues Clin Neurosci. 2007;9(2):215-26.
4. Salomon JA, Vos T, Hogan DR, Gagnon M, Naghavi M, Mokdad A, et al. Common values in assessing health outcomes from disease and injury: disability weights measurement study for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2129-43.
5. Tiihonen J, Lonnqvist J, Wahlbeck K, Klaukka T, Niskanen L, Tanskanen A, et al. 11-year follow-up of mortality in patients with schizophrenia: a population-based cohort study (FIN11 study). Lancet. 2009;374(9690):620-7.
6. Arango C, Kapur S, Kahn RS. Going beyond "trial-and-error" in psychiatric treatments: OPTiMiSE-ing the treatment of first episode of schizophrenia. Schizophr Bull. 2015;41(3):546-8.
7. Lieberman JA, Stroup TS, McEvoy JP, Swartz MS, Rosenheck RA, Perkins DO, et al. Effectiveness of antipsychotic drugs in patients with chronic schizophrenia. N Engl J Med. 2005;353(12):1209-23.
8. Chong HY, Teoh SL, Wu DB, Kotirum S, Chiou CF, Chaiyakunapruk N. Global economic burden of schizophrenia: a systematic review. Neuropsychiatr Dis Treat. 2016;12:357-73.
9. Penttila M, Jaaskelainen E, Hirvonen N, Isohanni M, Miettunen J. Duration of untreated psychosis as predictor of long-term outcome in schizophrenia: systematic review and meta-analysis. Br J Psychiatry. 2014;205(2):88-94.
10. Fried EI, Nesse RM. The impact of individual depressive symptoms on impairment of psychosocial functioning. PLoS One. 2014;9(2):e90311.
11. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13(2):261-76.
12. Ostergaard SD, Foldager L, Mors O, Bech P, Correll CU. The validity and sensitivity of PANSS-6 in treatment-resistant schizophrenia. Acta Psychiatr Scand. 2018.
13. Ostergaard SD, Foldager L, Mors O, Bech P, Correll CU. The Validity and Sensitivity of PANSS-6 in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Study. Schizophr Bull. 2018;44(2):453-62.
14. Ostergaard SD, Lemming OM, Mors O, Correll CU, Bech P. PANSS-6: a brief rating scale for the measurement of severity in schizophrenia. Acta Psychiatr Scand. 2016;133(6):436-44.
15. Bech P, Austin SF, Timmerby N, Ban TA, Moller SB. A clinimetric analysis of a BPRS-6 scale for schizophrenia severity. Acta Neuropsychiatr. 2018;30(4):187-91.
16. Hieronymus F, Nilsson S, Eriksson E. A mega-analysis of fixed-dose trials reveals dose-dependency and a rapid onset of action for the antidepressant effect of three selective serotonin reuptake inhibitors. Transl Psychiatry. 2016;6(6):e834.

What is the purpose of the analysis being proposed? Please select all that apply.: 
News research question to examine treatment effectiveness on secondary endpoints and/or within subgroup populations
Research that confirms or validates previously conducted research on treatment effectiveness
Participant-level data meta-analysis:
Participant-level data meta-analysis uses only data from YODA Project
Research on clinical trial methods
Research on clinical prediction or risk prediction
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

For the first part of the project, i.e., comparing the performance between brief unidimensional subscales and their comprehensive counterparts, the primary outcome measures will be those of a standard Rasch analysis (e.g., Andersen´s likelihood ratio test, Wald tests, tests of differential item functioning). We will also assess differences in responsiveness to change by contrasting endpoint effect sizes (standardized mean differences, SMDs, for continuous outcome parameters; odds ratios, ORs, for categorical outcome parameters e.g., response or remission) between rating scales, both for drug-placebo comparisons and for drug-drug comparisons.
For the second part of the project, i.e., assessing individual-level predictors, the main outcome measures will be the parameter estimates and levels of significance for the predictors (e.g., BMI, age, sex, baseline severity) when added to simplified models that do not control for these factors, as well as the differences in endpoint effect sizes between models that do and do not control for the respective factors.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

The main predictor variables are: treatment, BMI, age, sex, baseline symptom severity (assessed either by BPRS or PANSS, or by their brief unidimensional counterparts), baseline symptom profile (assessed by individual BPRS and PANSS symptoms), early symptomatic improvement, and the presence of adverse events of interest. Predictor variables will primarily be included as continuous covariates, with potential interactions with treatment and/or trial, being checked in all analyses. Since specific levels for certain predictors may only be covered in certain studies (e.g., adolescents not being represented in studies focused on adults and vice versa), some comparisons will necessarily be indirect. In such cases we will conduct also stratified analyses for each population.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

Depending on data availability/feasibility we may contrast outcome as measured by PANSS-6, PANSS, BPRS-6, and BPRS to outcome as measured by CGI or an established quality of life scale (e.g., QOLS, SQLS).

Statistical Analysis Plan: 

The scalability and transferability of the BPRS-6, BPRS, PANSS-6 and PANSS will be investigated using Rasch analysis (e.g., as implemented in the eRm, ltm, and difR packages in R). We will use conventional tests of differential item functioning, scalability and transferability, as well as graphical checks of model fitness to determine the performance of the different rating instruments. Sensitivity analyses – in addition to the tests of differential item functioning – in relevant subpopulations (e.g., studies conducted in different regions, studies on adolescents) may be included if deemed appropriate.
Continuous outcome measures will be analyzed using Mixed Models for Repeated Measurement (MMRM) methodology. The dependent variables will be all post-baseline evaluations up until the endpoint evaluation, or up until the last common evaluation point included in the majority of eligible studies (for maximum coverage). The models will include fixed factors for treatment, trial, time (usually in weeks), and the interaction between treatment and time. The decision to treat trial as a fixed factor is motivated by feasibility and based on previous experiences with using MMRM models in trials on major depression in which models specifying a random intercept and/or slope for trial tend to have convergence issues (16). Baseline severity on the corresponding effect measure will be included as a covariate. The within-subjects correlations over time will be modelled using an unstructured (co)variance matrix. If this fails to converge an autoregressive (co)variance matrix with heterogeneous variance will be tried, and if this fails to converge an autoregressive (co)variance matrix with heterogeneous variance will be fitted. Should models using all three different (co)variance matrices fail to converge we will explore other solutions, e.g., other (co)variance structures, and/or excluding certain studies and/or some time-points from the primary analyses. Denominator degrees of freedom will be estimated using the Kenward-Roger approximation. Effects on continuous outcome measures will be reported as standardized mean differences to maximize comparability between different rating instruments.
Categorical outcome measures will be analyzed in an analogous fashion using Generalized Linear Mixed Models and effects on categorical outcome measures (e.g., response, remission) will be reported as odds ratios.
Covariates and predictor variables of interest (see above) will primarily be included as continuous covariates while assessing potential interactions with treatment and/or trial. These analyses, however, will necessitate somewhat of a case-by-case methodology based on data availability/overlap between studies. We thus foresee that we may need to conduct also stratified analyses, or analyses of derivate measures (e.g., categorical derivatives, factor scores), as sensitivity analyses.
Considering the available sample size we judge that we will have adequate power to detect any clinically relevant effect modifiers for risperidone and paliperidone. For reference drugs with sparse data, however, power may be inadequate. This will be dealt with on a case-by-case basis and communicated in the resulting publications.
The issue of missing data will be partially mitigated by the use of MMRM and generalized linear mixed models. Sensitivity analyses will be done on the intention-to-treat population using LOCF methodology in order to assess the impact of this methodological decision. In cases where an outcome of interest or a potential effect modifier can be time-dependent (e.g., early side effects, early rating scale-assessed improvement or decline) we will conduct sensitivity analyses using different capture windows.

How did you learn about the YODA Project?: 
Software Used: 
RStudio
Associated Trials: 
<ol><li><a href="/node/173">NCT01009047 - R076477PSZ3003 - A Randomized, Multicenter, Double-Blind, Active-Controlled, Flexible-Dose, Parallel-Group Study of the Efficacy and Safety of Prolonged Release Paliperidone for the Treatment of Symptoms of Schizophrenia in Adolescent Subjects, 12 to 17 Years of Age </a></li><li><a href="/node/175">NCT00518323 - R076477PSZ3001 - A Randomized, Multicenter, Double-Blind, Weight-Based, Fixed-Dose, Parallel-Group, Placebo-Controlled Study of the Efficacy and Safety of Extended Release Paliperidone for the Treatment of Schizophrenia in Adolescent Subjects, 12 to 17 Years of Age</a></li><li><a href="/node/178">NCT00334126 - R076477SCH3015 - A Randomized, Double-blind, Placebo-controlled, Parallel Group Study to Evaluate the Efficacy and Safety of Paliperidone ER Compared to Quetiapine in Subjects With an Acute Exacerbation of Schizophrenia</a></li><li><a href="/node/181">NCT00589914 - R092670PSY3006 - A Randomized, Double-Blind, Parallel-Group, Comparative Study of Flexible Doses of Paliperidone Palmitate and Flexible Doses of Risperidone Long-Acting Intramuscular Injection in Subjects With Schizophrenia</a></li><li><a href="/node/190">NCT00590577 - R092670PSY3007 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Dose Response Study to Evaluate the Efficacy and Safety of 3 Fixed Doses (25 mg eq., 100 mg eq., and 150 mg eq.) of Paliperidone Palmitate in Subjects With Schizophrenia</a></li><li><a href="/node/192">NCT00210717 - R092670PSY3002 - A Randomized, Double-Blind, Parallel Group, Comparative Study of Flexibly Dosed Paliperidone Palmitate (25, 50, 75, or 100 mg eq.) Administered Every 4 Weeks and Flexibly Dosed RISPERDAL CONSTA (25, 37.5, or 50 mg) Administered Every 2 Weeks in Subjects With Schizophrenia</a></li><li><a href="/node/194">NCT00210548 - R092670PSY3003 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Dose-Response Study to Evaluate the Efficacy and Safety of 3 Fixed Doses (50 mg eq., 100 mg eq., and 150 mg eq.) of Paliperidone Palmitate in Subjects With Schizophrenia</a></li><li><a href="/node/195">NCT00101634 - R092670PSY3004 - A Randomized, Double-blind, Placebo-controlled, Parallel-group, Dose-response Study to Evaluate the Efficacy and Safety of 3 Fixed Doses (25 mg eq, 50 mg eq, and 100 mg eq) of Paliperidone Palmitate in Patients With Schizophrenia</a></li><li><a href="/node/197">NCT00034749 - RIS-USA-231 - The Efficacy and Safety of Risperidone in Adolescents With Schizophrenia: a Comparison of Two Dose Ranges of Risperidone</a></li><li><a href="/node/296">NCT00397033 - R076477SCA3001 - A Randomized, Double-blind, Placebo-controlled, Parallel-group Study to Evaluate the Efficacy and Safety of Two Dosages of Paliperidone ER in the Treatment of Patients With Schizoaffective Disorder</a></li><li><a href="/node/297">NCT00412373 - R076477SCA3002 - A Randomized, Double-blind, Placebo-controlled, Parallel- Group Study to Evaluate the Efficacy and Safety of Flexible-dose Paliperidone ER in the Treatment of Patients With Schizoaffective Disorder</a></li><li><a href="/node/548">NCT00249132 - RIS-INT-3 - A Canadian multicenter placebo-controlled study of fixed doses of risperidone and haloperidol in the treatment of chronic schizophrenic patients</a></li><li><a href="/node/626">NCT00077714 - R076477-SCH-304 - A Randomized, Double-blind, Placebo- and Active-controlled, Parallel-group, Dose-response Study to Evaluate the Efficacy and Safety of 2 Fixed Dosages of Paliperidone Extended Release Tablets and Olanzapine, With Open-label Extension, in the Treatment of Patients With Schizophrenia</a></li><li><a href="/node/627">NCT00083668 - R076477-SCH-305 - A Randomized, Double-blind, Placebo- and Active-controlled, Parallel-group, Dose-response Study to Evaluate the Efficacy and Safety of 3 Fixed Dosages of Paliperidone Extended Release (ER) Tablets and Olanzapine, With Open-label Extension, in the Treatment of Patients With Schizophrenia</a></li><li><a href="/node/628">NCT00074477 - R092670-SCH-201 - A Randomized, Double-Blind, Placebo-Controlled Study to Evaluate the Efficacy and Safety of 50 and 100 Mg-eq of Paliperidone Palmitate in Patients With Schizophrenia</a></li><li><a href="/node/638">NCT00078039 - R076477-SCH-303 - Trial Evaluating Three Fixed Dosages of Paliperidone Extended-Release (ER) Tablets and Olanzapine in the Treatment of Patients With Schizophrenia</a></li><li><a href="/node/704">NCT00085748 - R076477-SCH-302 - A Randomized, 6-Week Double-Blind, Placebo-Controlled Study With an Optional 24-Week Open-Label Extension to Evaluate the Safety and Tolerability of Flexible Doses of Paliperidone Extended Release in the Treatment of Geriatric Patients With Schizophrenia</a></li><li><a href="/node/857">NCT00088075 - RIS-SCH-302/CR003370 - A Randomized, Double-Blind, Placebo-Controlled Clinical Study of the Efficacy and Safety of Risperidone for the Treatment of Schizophrenia in Adolescents</a></li><li><a href="/node/858">RIS-USA-1 (RIS-USA-9001) - Risperidone versus haloperidol versus placebo in the treatment of schizophrenia</a></li><li><a href="/node/860">NCT00253136 - RIS-USA-121/CR006055 - Risperidone Depot (Microspheres) vs. Placebo in the Treatment of Subjects With Schizophrenia</a></li><li><a href="/node/866">RIS-USA-72 - The safety and efficacy of risperidone 8 mg qd and 4 mg qd compared to placebo in the treatment of schizophrenia</a></li><li><a href="/node/871">NCT00524043 - R076477SCH4012 - A Randomized, Double-Blind, Placebo- and Active-Controlled, Parallel-Group Study to Evaluate the Efficacy and Safety of a Fixed Dosage of 1.5 mg/Day of Paliperidone Extended Release (ER) in the Treatment of Subjects With Schizophrenia</a></li><li><a href="/node/3772">NCT01299389 - PALM-JPN-4 - A Randomized, Double-Blind, Placebo-Controlled, Parallel-Group, Fixed-Dose, Multicenter Study of JNS010 (Paliperidone Palmitate) in Patients With Schizophrenia</a></li></ol>
Make Publicly Available : 
Year of Data Access: 
2019

2019-3938

Project Title: 
Machine learning prediction of remission in patients given augmented treatment in major depression
Specific Aims of the Project: 

The aims of this project are two-fold: 1) improving our machine learning model to include prediction of augmentation treatment utility and 2) to improve understanding of subtypes of treatment resistant depression (TRD). We aim to expand our model to include treatment efficacy predictions for patients who have been unresponsive to classical antidepressant medication. By training our model with data that includes the outcomes of antidepressant augmentation with risperidone or concerta for treatment resistant depression, we hope that we will be able to effectively identify patients with TRD who will likely benefit from this treatment. Ideally, this would reduce the arduous trial-and-error process that these patients typically go through.
Through the training process, our algorithm will also identify a range of factors that it has identified as important for predicting outcome. This knowledge, in and of itself, would add to the scientific literature surrounding treatment resistant depression, by providing a starting point for classifying different subtypes of TRD based on symptom features that may determine response to treatment augmentation with antipsychotics or stimulants. It allows us to identify features that not only predict outcome but also act as mediators or moderators.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Ongoing
Scientific Abstract: 

Background: Globally, clinical depression affects over 320 million people and is the leading cause of disability worldwide. While numerous treatment options do exist, most patients spend months to years undergoing an arduous trial-and-error process before finding a treatment that works for them.
Objective: We will train machine learning models to help determine when adjuncts to antidepressants would be useful in an effort to reduce time to recovery. All models will be published and made open source and not directly commercialized.
Study design: We will use patient data to train our deep learning model to predict the outcome of methylphenidate hydrochloride and risperidone augmentation of SSRI/SNRI treatment for treatment-resistant Major Depressive Disorder.
Participants: Patients diagnosed with major depressive disorder who have failed to respond to at least one antidepressant treatment and were given risperidone or methylphenidate augmentation.
Main outcome measures: Models will be trained to maximize specific model metrics: AUC, PPV, NPV, sensitivity, specificity. These models are then tested to see if they provide projected improvement in population remission on a held out sample of the data and if this is significantly different from random allocation in a series of bootstrapped samples.
Statistical analyses: To predict different clinical outcomes, our custom high-level deep learning pipeline, Vulcan, supports deep learning, random forest, and other models as well as a feature selection pipeline (i.e variance thresholding, recursive feature elimination).

Brief Project Background and Statement of Project Significance: 

Major depressive disorder occurs in an estimated 10%-15% of the population, and remains one of the highest public health concerns. While there is a range of pharmacological treatment options, nearly one third of patients fail to respond to adequate doses of antidepressant agents. Treatment-resistant depression (TRD) is defined as major depressive disorder (MDD) with symptoms that fail to respond to treatment with at least two different classes of antidepressant medications (Al-Harbi, 2012). These patients require further treatment that may involve switching antidepressant medication, combining different antidepressants, the addition of a medication which is not an antidepressant, or somatic therapies (Philip, Carpenter, Tyrka, & Price, 2010). This is done on a trial-and-error basis, which results in a long and difficult process for the patient.

Both risperidone, an antipsychotic agent, and methylphenidate, a stimulant used to treat ADHD, have been used as options for augmentation of antidepressant monotherapy for TRD. In general, risperidone augmentation has been found to be more effective than placebo, although results are mixed. Methylphenidate has been found to be effective in elderly patients, but results are less positive in other populations (Barowsky & Schwartz, 2006; Philip et al., 2010). It is clear that these augmentation strategies are not effective for everyone, and further investigation is required in order to determine who may benefit the most from these treatments. Due to the complexity of TRD, and the heterogeneity of symptoms, and medication side effects, there is likely a complex interaction of various factors that impact the efficacy of methylphenidate and risperidone augmentation.

In order to unravel these interactions, a machine learning approach that can interpret data sets with a large number of variables may be the most effective solution to the challenge of predicting efficacy of treatment augmentation. To address this challenge, we have developed our pipeline, Vulcan AI, which provides a comprehensive pipeline for high-dimensional data visualization, data preprocessing, rapid modular network prototyping, training, evaluation, and model interpretability. We use a deep neural network to analyze our data, allowing us to capture the complex, non-linear relationships likely to be present in psychiatric data (e.g. mediation and moderation effects, which are often unknown a priori). When data is present in insufficient quantities for deep learning to be useful, Vulcan can be used to implement random forests, regression, and gradient boosting. By training the algorithm with data that includes the outcomes of antidepressant augmentation with methylphenidate and risperidone augmentation, we should be able to identify patients with TRD that will benefit from these treatments, reducing the arduous trial-and-error process inherent in the treatment of depression that does not respond to first line monotherapy.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

All patients with primary diagnoses of major depression and without bipolar disorder will be included.

Narrative Summary: 

Globally, clinical depression affects over 320 million people and is the leading cause of disability worldwide. As such a widespread disorder that is incredibly symptomatically homogenous, two thirds of patients fail to respond to the first treatment and require numerous treatment changes. As such, the treatment selection process in depression is essentially an educated “guess and check” approach. Using machine learning, we aim to personalize this process by using patient data to predict the most effective treatments for each individual so that patients can reach recovery faster. In particular, we aim to predict which patients will benefit the most from the addition of adjuvant treatments.

Project Timeline: 

Assuming a 12 month timeline, we will spend the first three months familiarizing ourselves with the data and building initial models, as well as installing needed packages on the online portal (which we have already checked is possible). We will then spend two months refining and finalizing the models. We will have completed the draft of our manuscript three months later (8 months into the project) and will circulate to co-authors and submit it by month 9 of the project; that way, we still have three months to use the data to complete any revisions.

Dissemination Plan: 

We plan to include all of our findings, in manuscripts to be submitted to peer-reviewed journals. Some appropriate journals may include JAMA Psychiatry, American Journal of Psychiatry, and Translational Psychiatry. In addition, our code for Vulcan AI is open source, available on GitHub (https://github.com/Aifred-Health/Vulcan). We will also be disseminating the final models via a Git-like service and will be submitting our work to conferences. We also generally post all of our work on pre-print servers.

Bibliography: 

Al-Harbi, K. S. (2012). Treatment-resistant depression: therapeutic trends, challenges, and future directions. Patient Preference and Adherence, 6, 369–388. https://doi.org/10.2147/PPA.S29716
Barowsky, J., & Schwartz, T. L. (2006). An Evidence-Based Approach to Augmentation and Combination Strategies for. Psychiatry (Edgmont), 3(7), 42–61.
Philip, N. S., Carpenter, L. L., Tyrka, A. R., & Price, L. H. (2010). Pharmacologic Approaches to Treatment Resistant Depression: A Re-examination for the Modern Era. Expert Opinion on Pharmacotherapy, 11(5), 709–722. https://doi.org/10.1517/14656561003614781

What is the purpose of the analysis being proposed? Please select all that apply.: 
Preliminary research to be used as part of a grant proposal
Develop or refine statistical methods
Research on clinical prediction or risk prediction
Supplementary Material: 
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

Accuracy of remission prediction as per the area under the receiver operating curve (AUROC); remission will be defined as a subthreshold score on the main outcome questionnaire used in the study. We will also look at our model’s ability to produce projected improvements on the population remission rate.
Other measures will include the model's sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

We do not define a priori predictors; these will be selected by our feature selection pipeline.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

We do not define a priori predictors; these will be selected by our feature selection pipeline.

Statistical Analysis Plan: 

As noted, we are using machine learning methods- mostly deep learning, but also random forests, logistic regression, and gradient boosting. We plan to first analyze the data from the two studies we are requesting and the study protocols to determine if the studies can be pooled or not, based on whether or not the study protocols were sufficiently similar and if similar data was collected between the two studies. After this, we will pass the data (pooled or separately, depending on if the data can be pooled) through our feature selection pipeline, which will use recursive feature elimination, variance thresholding, and feature importance thresholding to select the features most predictive of remission. Following this we will train four models: a deep learning model, a random forest model, a logistic regression, and a gradient boosting approach, and analyze resulting model metrics to determine which approach provides the highest AUC. Data will be split into training and test splits, which models trained and internally cross-validated (k= 10) prior to being tested on a held-out sample. Another held-out sample will be used to measure model predictions regarding population remission rate. This step consists of a naive, hypothetical analysis followed by a conservative, non-hypothetical analysis. The naive analysis consists of predicting remission rates with different treatments for each patient in the held-out dataset, and then averaging their sample remission rate and comparing it to the population remission rate. This provides a blue-sky estimate of model utility in differential treatment benefit prediction, but remains hypothetical. We then produce 1000 bootstrapped samples and look at the patients who happened to be randomized to the treatment our model suggests they should have received and see if these patients do better in terms of remission rate compared to the general patient population; this allows for us to determine if the model could significantly improve remission rates.

A NOTE ON USE OF MACHINE LEARNING- REPLY TO REVIEWER COMMENTS
Thank you for the question about the sample size and dimensionality. With respect to dimensionality, we have had success in building models with good predictive and differential predictive performance using models with less than 50 features- in fact between 17 and 31. As these are mostly symptom scales, which will be present in this data, we do believe that we will be have sufficient dimensionality. We have also found that we can use machine learning analyses on smaller than usually anticipated datasets without overfitting- with the ability to fit models to datasets with ~600 people. The 2 trials listed here are likely too small in isolation for deep learning, and as such our plan is to use less advanced, more traditional machine learning algorithms to start- beginning with linear regression (for continuous outcomes like final score) or logistic regression (using remission/non-remission) and then moving to gradient boosting and random forests. We will try and see at that point, once we have a few good traditional baseline models, how deep learning perform, and also how some of our previous deep learning algorithms trained on larger datasets (including one with an augmentation treatment) performs when predicting this data (we will not mix the data from this project with data from previous projects). After we publish the models from this project in an open-source manner, we will also explore federated learning approaches- how to have models trained on different data but in the same problem space work together to provide predictions. As such our work will still be able to employ machine learning approaches, but these will be selected with the sample size and risk of overfitting in mind.

How did you learn about the YODA Project?: 
Software Used: 
I am not analyzing participant-level data / I will not be using these software for analyses in the secure platform
Please clarify:: 
We have already confirmed that we will be able to install the packages we need to run Vulcan in the Yoda environment
Associated Trials: 
<ol><li><a href="/node/1100">NCT00246233 - 42603MDD3001 (CON-CAN-3) - A Double-blind, Placebo-controlled, Randomized Trial to Evaluate the Safety, Tolerability and Efficacy of CONCERTA® (Methylphenidate Hydrochloride) Augmentation of SSRI/SNRI Monotherapy in Adult Patients With Major Depressive Disorder.</a></li><li><a href="/node/2061">NCT00044681 - RIS-INT-93 - A Study to Evaluate the Efficacy, Safety and Maintenance Effect of Risperidone Augmentation of SSRI Monotherapy in Young and Older Adult Patients With Unipolar Treatment-Resistant Depression</a></li></ol>
Make Publicly Available : 
Year of Data Access: 
2019

2019-3936

Project Title: 
Unbiased Treatment Efficacy Detection Methods with Patient Centered Outcomes
Specific Aims of the Project: 

There are 4 specific aims of the project: 1. Theoretical statistical exposition of methods under development. 2. Simulation studies of proposed methods. 3. Evaluation of proposed methods using randomized clinical trial data. 4. Publication of proposed methods, findings, and software from simulation and empirical studies.

Aim 1: See attached manuscript for outline of the theoretical exposition (equations are in LaTeX and won’t render here).

Aim 2: Simulation studies demonstrate the improved estimates of treatment efficacy compared to other methods. This work hypothesizes that the proposed method will return estimates with smaller bias and RMSE than standard approaches. This will lead to greater power to detect treatment arm separation.

Aim 3: After showing that the procedure can recover the true parameters in a simulation study, the utility of the proposed method will be demonstrated using data from a clinical trial. The clinical trial should be a survival analysis design, which makes it highly likely to feature MNAR drop-out mechanisms. Applying the proposed method should then yield different estimates than an unadjusted IRT model.

Aim 4: This work will be published in a statistics journal. Additionally, all software code written will be made available online.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Ongoing
Scientific Abstract: 

Background: Patient Centered Outcomes (PCO) are often incorporated in trials via patient/clinician/caregiver reported questionnaires. In the survival analysis context, the data is typically considered to be missing not at random (MNAR), which requires the use of statistical approaches that properly adjust for this type of missing data.
Objective: This research presents an item response theory (IRT) framework for adjusting PCO-based scores for MNAR drop-out. The model allows the IRT scores and the drop-out mechanism to be modeled simultaneously. This restores conditional independence and corrects for bias in the estimates of treatment efficacy that occur under MNAR drop-out.
Study Design: A simulation study was designed, run, and analyzed to illustrate the improved estimates of treatment efficacy using this approach. An empirical example using clinical trial data demonstrates the utility of the procedure.
Participants: Patients from a clinical trial with a survival analysis design will be included in the analysis.
Main Outcome Measures: In the simulation study, the bias and root mean squared error (RMSE) of the estimated separation in the treatment arms will be computed. In the empirical data example, the treatment arm separation will be estimated with the proposed procedure and without adjustment. Differences in the estimated separation and resulting inferences will be reported.
Statistical Analysis: This will compare the estimates of treatment efficacy from the proposed longitudinal IRT model with the estimates of treatment efficacy from a standard IRT model.

Brief Project Background and Statement of Project Significance: 

Statistical models in the psychometrics literature have been slow to penetrate the field of Patient Centered Outcomes (PCO). Latent variable models, of which IRT is a sub-category, offer advantages in modeling data with measurement error.1 Although there has been interest in applying the IRT framework to PCO data, there have been barriers to widespread adoption. Specifically, there is a lack of easily implemented procedures for dealing with MNAR data. Little methodological work has been done to address this issue since the initial research on the topic.2 Until these methods are developed and validated, IRT-based estimates of treatment efficacy using PCO will be biased under MNAR data. This work addresses a gap in the statistical literature by developing a psychometric model that can accommodate MNAR data. Adapting modern psychometric methods for the field of PCO is a crucial step in addressing the lack of sensitivity in many PCO measures.3

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

An appropriate data source requires (1) missing not at random as part of the design, (2) a substantial proportion of drop-out, and (3) sufficient sample size to estimate the models (note that the two Abiraterone trials or the two Daratumumab trials could be combined to increase sample size).

The requested studies meet these criteria. However, a useful dataset to illustrate the proposed method would also show some difference in the treatment effect across methods. This would highlight the potential benefit of the method. Furthermore, because the method is new, it is unclear what the limitations of the procedure are. Having several trials to evaluate the method would be the most productive way of understanding the benefits and limitations of the proposed procedure. Crucially, this could be included in the manuscript to offer important guidance for researchers planning to use the method.

The following questionnaires will be used:
Paliperidone palmitate: PANSS, PSP, SDS
Abiraterone: FACT-P, BFI
Galantamine: Mini-Mental State Examination, Disability Assessment in Dementia
Daratumumab: EORTC-QLQ-C30 and EQ-5D-5L

Narrative Summary: 

Patient Centered Outcomes (PCO) are often incorporated in trials via patient/clinician/caregiver reported questionnaires. In the survival analysis context, such as cancer trials, patients drop-out of the study for reasons related to disease severity. This type of drop-out leads to incorrect conclusions about patient quality of life. This research develops statistical approaches that are needed that appropriately adjust for this missingness and yield correct inferences. More robust development of these methods will help detect which interventions have a positive impact on patient quality of life. This, in turn, will help guide patient centered drug development.

Project Timeline: 

A draft manuscript has been attached as a file below. A technical exposition has been sketched out, and initial simulation evidence already compiled. The application to the clinical trial data will help guide the simulation study conditions. The project will start once the data have been transferred.
Timeline:
Data management: 1 month
Application to clinical trial, creating tables/figures, writing up results: 3 months
Full simulation study, creating tables/figures, writing up results: 3 months
Completing writing, revisions: 3 months
Submission to journal: 10 months from data transfer

Note: typically journal reviewers will ask for additional work to be done. Given the 6-18 month review times for popular statistical journals, an extension will almost certainly be necessary.

Dissemination Plan: 

Please see attached file for a draft of a manuscript. This is being prepared for the journal Statistics in Medicine. The goal is to highlight psychometric methods that can and should be utilized in medical applications. Just as importantly, software code will be disseminated. The code will be included as an Appendix to the manuscript, available online. The same code will also be posted to a github account, making it easily searchable. Please note that this software code will NOT include any sensitive information regarding the requested dataset.

Bibliography: 

1. de Ayala RJ. The Theory and Practice of Item Response Theory (Methodology in the Social Sciences). New York: The Guilford Press; 2009.
2. Douglas JA. Item response models for longitudinal quality of life data in clinical trials. Stat Med. Nov 15 1999;18(21):2917-2931.
3. Lawrence Gould A, Boye ME, Crowther MJ, et al. Joint modeling of survival and longitudinal non-survival data: current methods and issues. Report of the DIA Bayesian joint modeling working group. Statistics in Medicine. 2015;34(14):2181-2195.

What is the purpose of the analysis being proposed? Please select all that apply.: 
Develop or refine statistical methods
Supplementary Material: 
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

The purpose of this study is to evaluate the proposed statistical methodology that is being developed. This is done via simulation study, where procedures can be evaluated in terms of the quality of the estimates. The quality of the estimates can be directly compared because the true values are known to the researcher. The estimates are evaluated via two measures, bias and RMSE. These are standard measures in the field.

For an illustration of the statistical methods using empirical clinical trial data (i.e., data shared by YODA), the patient quality of life score will be estimated for each treatment arm, using the proposed statistical procedure and the existing procedure. The resulting estimates will be compared, with the percent difference reported. Importantly, any differences in inferences will also be reported.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

IRT models use questionnaire items to evaluate the construct of interest. The proposed approach goes one step further and also incorporates patient drop-out. That is, the patient data is re-coded to reflect the timepoint at which they dropped out.

Note here that the proposed method is the reverse of the typical survival analysis. For example, in oncology trials, the typical approach compares time to drop-out across treatment arms, with an adjustment for patient health-related quality of life. The work here does the opposite: the longitudinal IRT model compares quality of life across treatment arms, with an adjustment for drop-out. This assumes that the drop-out is related to the patient quality of life – that is, it assumes the data is MNAR due to the survival analysis design. The ultimate purpose of the proposed method is to compare patient health-related quality of life across the treatment arms, without the bias that occurs from ignoring the MNAR drop-out mechanism.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

Standard practice is to include basic demographic information, such as age and sex, in the statistical model. This ensures that potential confounds have been controlled for, which allows for valid interpretation of the model output.

Statistical Analysis Plan: 

The proposed use of this dataset is to illustrate a newly developed statistical method. This illustration will be part of a manuscript that provides a technical exposition of the methods, as well as a simulation study to show the performance of the method. A minimal amount of descriptive statistics will be computed, with the main focus being a comparison of the estimated treatment effect using the proposed method versus standard methods. To accommodate this analysis, descriptives such as the proportion of drop-out at each time point, as well as the average score at each timepoint (stratified across treatment arms) will be computed. This will help to show how the method makes adjustments for missing data and how that impacts the model-based estimates of treatment efficacy.

How did you learn about the YODA Project?: 
Software Used: 
R
Associated Trials: 
<ol><li><a href="/node/181">NCT00589914 - R092670PSY3006 - A Randomized, Double-Blind, Parallel-Group, Comparative Study of Flexible Doses of Paliperidone Palmitate and Flexible Doses of Risperidone Long-Acting Intramuscular Injection in Subjects With Schizophrenia</a></li><li><a href="/node/304">NCT00638690 - COU-AA-301 - A Phase 3, Randomized, Double-Blind, Placebo-Controlled Study of Abiraterone Acetate (CB7630) Plus Prednisone in Patients With Metastatic Castration-Resistant Prostate Cancer Who Have Failed Docetaxel-Based Chemotherapy</a></li><li><a href="/node/1043">NCT00679627 - GALALZ3005 - A Randomized, Double-Blind, Placebo-controlled Trial of Long-term (2-year) Treatment of Galantamine in Mild to Moderately-Severe Alzheimer's Disease</a></li><li><a href="/node/1115">NCT00887198 - COU-AA-302 - A Phase 3, Randomized, Double-blind, Placebo-Controlled Study of Abiraterone Acetate (CB7630) Plus Prednisone in Asymptomatic or Mildly Symptomatic Patients With Metastatic Castration-Resistant Prostate Cancer</a></li><li><a href="/node/2196">NCT02076009 - 54767414MMY3003 - Phase 3 Study Comparing Daratumumab, Lenalidomide, and Dexamethasone (DRd) vs Lenalidomide and Dexamethasone (Rd) in Subjects With Relapsed or Refractory Multiple Myeloma</a></li><li><a href="/node/2201">NCT02136134 - 54767414MMY3004 - Phase 3 Study Comparing Daratumumab, Bortezomib and Dexamethasone (DVd) vs Bortezomib and Dexamethasone (Vd) in Subjects With Relapsed or Refractory Multiple Myeloma</a></li></ol>
Make Publicly Available : 
Year of Data Access: 
2019

2019-3904

Project Title: 
Development of a Clinical Prediction Tool for Treatment Outcomes in Ustekinumab-treated Patients with Moderate Severe Crohn's Disease
Specific Aims of the Project: 

Specific aim #1: To identify factors predictive of remission with ustekinumab in Crohn's disease patients through post-hoc analyses of phase II and III RCTs of ustekinumab in Crohn's disease
Specific aim #2: Develop and validate a prediction model and clinical decision support tool capable of identifying Crohn's disease patients most or least likely to respond to ustekinumab

Hypothesis: Factors for the ustekinumab specific decision support tool may be overlapping with our prior vedolizumab decision support tool, however, the differential weighting within the prediction model and tool will help differentiate a patient as being optimal or not ideal for one of the two therapies.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Incomplete Not Reviewed
Scientific Abstract: 

Background: Clinical decision support tools provide an opportunity to personalize treatment decisions based on individual patient characteristics.
Objective: Identify factors predictive of achieving remission with ustekinumab in Crohn's disease, develop a logistic regression prediction model, and transform this model into an easy to use clinical decision support tool which will be validated using a routine practice cohort of ustekinumab treated Crohn's disease patients.
Study Design: Individual participant level pooled analysis of RCTs of ustekinumab in patients with Crohn's disease
Participants: Patients enrolled in phase II or III RCTs of ustekinumab in moderate-severe Crohn's disease
Main Outcome Measures: clinical remission between weeks 3 and 16
Secondary Outcome Measures: Endoscopic remission at week 8; Measured ustekinumab drug exposure over 16 weeks, Rapidity in symptom reduction
Statistical Analysis: Logistic and Cox Proportional Hazard modeling will be used to identify predictors of clinical remission and endoscopic remission. Identified predictors will then be weighted and combined into a single clinical decision support tool using a multiplication of the beta coefficient. Performance during validation in the routine practice cohort will be done using ROC curves and sensitivity/specificity.

Brief Project Background and Statement of Project Significance: 

Multiple therapeutic options exist for the management of Crohn's disease, with biologics representing one of the most widely used and most expensive for our healthcare system. Despite the widespread use and availability for nearly 2 decades, the personalization of these agents is not feasible due to a lack of decision support tools availability. Clinical prediction models and clinical decision support tools utilize patient characteristics to provide an individualized prediction of expected treatment effectiveness. These are now integral components of 'precision medicine' and the development of these for biologics in Crohn's disease are essential. Our group has successfully developed and validated a clinical decision support tool for vedolizumab in Crohn's disease (Dulai et al. Gastro 2018) and in the current proposal we aim to leverage the phase 2 and 3 clinical trial programs for ustekinumab to similarly build and validate a clinical decision support program for ustekinumab in Crohn's disease. The significance of this work lies in the development of a tool which will help to identify the Crohn's disease patients most or least likely to benefit from ustekinumab. Coupled with our prior tool for vedolizumab in Crohn's disease, the two tools together could be used to personalize decisions for both agents. From a scientific perspective, it will help identify clinical factors associated with response to infliximab, which can then be used to further understand how these drugs may be effective. From a clinical perspective, information generated from this study on treatment response to ustekinumab, will be generalizable and directly applicable to patient care, informing clinical guidelines and offering potential for promoting value-based in patients with Crohn's disease.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

• Trials of ustekinumab in Crohn's disease
Inclusion criteria: • Patients (adults or pediatric) with moderate-severe Crohn's disease • Treated with ustekinumab or placebo for induction and/or maintenance Exclusion criteria • Patients lost to follow-up or did not participate in trial after randomization (without receiving any dose of the medication)

Narrative Summary: 

Multiple biologics and small molecule inhibitors are available for the treatment of Crohn's disease, however, the optimal selection of these agents is not well informed. Using logistic regression analyses, we propose to identify factors predictive of remission with ustekinumab and transform these factors into an easy to use clinical decision support tool. This will be developed using a combination of phase 2 and 3 clinical trial data for ustekinumab in Crohn's disease, and subsequently validated in a routine practice cohort of patients treated across multiple sites in the US.

Project Timeline: 

o Project start date: July 1, 2019
o Analysis completion date: November 1, 2019
o Manuscript drafted: January 1, 2020
o Manuscript submitted for publication: January 31, 2020
o Date results reported back to YODA: January 31, 2020

Dissemination Plan: 

We anticipate the generation of a manuscript from this work with the target audience being clinical gastroenterologists. Potential submission journals include Gastroenterology, Gut, American Journal Gastroenterology, Clinical Gastroenterology & Hepatology, and Inflammatory Bowel Disease

Bibliography: 

Dulai PS, Boland BS, Singh S, et al. Development and Validation of a Scoring System to Predict Outcomes of Vedolizumab Treatment in Patients With Crohn's Disease. Gastroenterology. 2018 Sep;155(3):687-695.e10. doi: 10.1053/j.gastro.2018.05.039. Epub 2018 May 30.

What is the purpose of the analysis being proposed? Please select all that apply.: 
Research on clinical prediction or risk prediction
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

Clinical remission (CDAI < 150) between weeks 3-16 of the trial
Endoscopic remission (SES-CD defined) at week 8

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

Main predictor/independent variable will be exposure to ustekinumab vs. placebo

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

Key confounding variables of interest in our study are:
o Biochemical measures of disease severity – baseline C-reactive protein as a categorical variable (<0.5mg/dl or ?0.5mg/dl)
o Co-interventions – concomitant use of immunomodulators like azathioprine, 6-mercaptopurine or methotrexate (yes vs. no), concomitant use of steroids (yes vs. no), dose of ustekinumab
o Factors known to modify pharmacokinetics of biologics in Crohn's disease – baseline albumin as a categorical variable (<3.5g/dl vs. ?3.5g/dl), sex (males vs. females), weight
o Disease factors – disease location (ileal, colonic), disease duration

Statistical Analysis Plan: 

A multivariable Cox proportional hazard regression prediction model will be built in the ustekinumab trial datasets for the outcome of clinical remission between weeks 3 and 16. All baseline variables found to have a p value <0.15 on univariable analyses will be included in the multivariable model, after an assessment for co-linearity and clinical importance or interpretability, and a backward model selection approach will be used with a p value threshold of 0.15 for inclusion in the final model. Interaction terms will be assessed individually and included in the final model if they have a p value of < 0.10 on univariable and multivariable analyses. This model will then undergo external validation in the consortium established by Dr. Dulai (outside the YODA platform). Discriminative ability will be assessed by receiver operating characteristic (ROC) curve analysis and presented as area under the ROC curve (AUC). Calibration will be tested by the Hosmer–Lemeshow goodness-offit test after splitting the sample into quintiles. This test assesses whether or not the observed event rates match expected event rates in subgroups of the model population, with P-values <0.05 indicating evidence of poor fit. The overall performance of the models will be evaluated with the Nagelkerke R2 and the Brier score. Nagelkerke R2 is a measure between 0 and 1, with 0 denoting that the model does not explain any variation and 1 denoting that it perfectly explains the observed variation. The Brier score is a measure between 0 and 1 of prediction with the mean squared difference between the predicted probability and the actual outcome. A lower Brier score indicates better performance, and the Brier score for a model can range from 0 for a perfect model to 0.25 for a noninformative model. The final prediction model will then transformed into a CDST by multiplying the regression coefficient of each predictor in the model by a factor of 10, rounding to the nearest value, and removing the intercept. The ustekinumab trial cohort subjects will then be split into quartiles using the CDST, and cut-points will be determined for patients who have a low (lowest quartile of clinical remission rates), intermediate (middle two quartiles of clinical remission rates), or high (highest quartile of clinical remission rates) probability of responding to ustekinumab. These cut-points will then be applied to the ustekinumab trial population to understand how the probability of achieving clinical remission with ustekinumab compared with placebo-treated participants. Finally, the CDST cut-points will be applied to the real-world consortium, and the sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio of the scoring tool will be calculated to identify patients who had a low or high probability of achieving clinical remission with ustekinumab.

How did you learn about the YODA Project?: 
Software Used: 
R
Associated Trials: 
<ol><li><a href="/node/1129">NCT00771667 - C0743T26 - A Phase 2b, Multicenter, Randomized, Double-blind, Placebo-controlled, Parallel Group Study to Evaluate the Efficacy and Safety of Ustekinumab Therapy in Subjects With Moderately to Severely Active Crohn's Disease Previously Treated With TNF Antagonist Therapy</a></li><li><a href="/node/1133">NCT01369329 - CNTO1275CRD3001 - A Phase 3, Randomized, Double-blind, Placebo-controlled, Parallel-group, Multicenter Study to Evaluate the Safety and Efficacy of Ustekinumab Induction Therapy in Subjects With Moderately to Severely Active Crohn's Disease Who Have Failed or Are Intolerant to TNF Antagonist Therapy (UNITI-1)</a></li><li><a href="/node/1134">NCT01369342 - CNTO1275CRD3002 - A Phase 3, Randomized, Double-blind, Placebo-controlled, Parallel-group, Multicenter Study to Evaluate the Safety and Efficacy of Ustekinumab Induction Therapy in Subjects With Moderately to Severely Active Crohn's Disease (UNITI-2)</a></li><li><a href="/node/1361">NCT01369355 - CNTO1275CRD3003 - A Phase 3, Randomized, Double-blind, Placebo-controlled, Parallel-group, Multicenter Study to Evaluate the Safety and Efficacy of Ustekinumab Maintenance Therapy in Subjects With Moderately to Severely Active Crohn's Disease</a></li><li><a href="/node/3381">NCT00265122 - C0379T07 - A Multicenter, Randomized, Phase 2a Study of Human Monoclonal Antibody to IL-12p40 (CNTO 1275) in Subjects With Moderately to Severely Active Crohn's Disease</a></li></ol>
Make Publicly Available : 

2019-3865

Project Title: 
Alternative Data Presentation For Treatment Outcomes in Psoriasis
Specific Aims of the Project: 

The aims of this project are to:
-Explore whether alternative data presentation and analysis methods, such as spider and waterfall plots, could be utilized to add to the conclusions and information provided from psoriasis biologic clinical trials.
-Explore whether a Bayesian aggregate n-of-1 approach could limit sample size and enable the design of more cost-effective.

We hypothesize that incorporating spider and waterfall plots will prove valuable for illustrating individual patient response and population response distributions, and could be utilized to improve clinical decision-making. We also believe implementing a Bayesian aggregate n-of-1 approach to psoriasis biologic clinical trials could provide similar conclusions to traditional larger trials.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Ongoing
Scientific Abstract: 

Background:
Psoriasis clinical trials have traditionally used Psoriasis Area and Severity Index (PASI) scores as a primary outcome and reported proportions of patients achieving PASIs reduction in a frequentist statistical model (group comparison). A more informative method of data presentation for the clinician would be in terms of probability for treatment success in the best and worst cases.
Objective:
-Can alternative data presentations and analyses improve conclusions and information provided from psoriasis biologic clinical trials?
-Can a Bayesian aggregate n-of-1 approach limit sample sizes and enable the design of more cost-effective trials?
Study Design:
We will re-analyze raw psoriasis clinical trial data and generate waterfall plots illustrating PASI or Dermatology Quality of Life Index (DLQI) improvements, spider plots assessing long-term drug efficacy, and a Bayesian aggregate n-of-1 format to generate a posterior probability distribution for the probability of achieving a PASI75 score.
Participants:
We will include patients with plaque psoriasis that were included in the phase III trials: NCT00267969, NCT00307437, NCT00454584, NCT01059773, NCT01550744, NCT02203032.
Main Outcome Measure(s):
1) Distribution of individual PASI and DLQI responses in participants receiving biologic treatment for psoriasis.
2) Modeling of the sample size effect using n-of-1 design and Bayesian hierarchical model meta-analysis.
Statistical Analysis:
Bayesian analysis n-of-1 design as described by 1-3.Alternative data models will be prepared using R suite.

Brief Project Background and Statement of Project Significance: 

Clinical trial data form the empirical foundation for modern evidence-based medicine. Through this process, drugs are rigorously studied for their safety and efficacy before recommendations may be formed to guide clinical decision-making. Just as the therapeutic options for psoriasis continue to develop and improve, so too must the analysis and data presentation models used to generate and illustrate trial results.

Current PASI score and frequentist methods do not visualize the entire spectrum of individual patient responses. Current developments in other fields such as in oncology advance towards alternative methods of data visualization encompassing the responses in the entire study population. Examples are the spider and waterfall plots that illustrate each patient’s unique drug response rates as well as population response distributions. Waterfall plots demonstrate the entire cohort’s response distribution to treatment by illustrating each patient’s individual response, effectively resulting in a waterfall-like figure 4,5. Spider plots have been used in oncology publications to graph tumor size changes longitudinally over time 4. We hope to extend this functionality into the field of dermatology and assess long-term biologic efficacy for treatment and maintenance of psoriasis. Ultimately, we aim to explore the usage of such alternative data presentation methods, improve the external validity of psoriasis drug studies, and optimize patient-care by providing practitioners with more complete results and conclusions. Furthermore, the implementation of more individual-centered data presentation may facilitate future studies exploring personalized medicine and enable researchers to inquire into what factors lead to a single patient’s unique responses to various psoriasis biologics.

Bayesian aggregate n-of-1 formats have previously been used in studying rare disease treatments where gathering high-quality treatment evidence is challenging due to limited resources, insufficient patient populations, and considerable heterogeneity 1. In extending this methodology into psoriasis biologic clinical trials, we hope to enable the design of smaller and more cost-effective studies with comparable outcomes to larger clinical trials. We will make use of previous published protocols as a reference when designing our own Bayesian aggregate n-of-1 analyses 1-3.

This study is original and the proposed methods for data analysis and visualization have not been attempted before for psoriasis.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

We will include patients with plaque psoriasis that were included in phase III clinical trials. Both placebo-controlled trials and trials with active comparators will be included. The list of trials are as follows: NCT00267969, NCT00307437, NCT00454584, NCT01059773, NCT01550744, NCT02203032. There are no other exclusion criteria.

Narrative Summary: 

Modern patient care is built upon evidence-based medicine and relies on clinical trial data. However, traditional outcomes such as proportions of patients achieving a primary outcome lack clinical relevance for physicians. This project will investigate the usage of alternative methods of data analysis and presentation focusing on capturing individual patient outcomes, such as: waterfall plots and spider plots. We will also explore a Bayesian aggregate n-of-1 design in psoriasis biologic clinical trials to describe individualistic therapy outcomes. We hypothesize these methods will provide more clinically relevant information and facilitate psoriasis treatment decisions for clinicians.

Project Timeline: 

The project will begin as soon as we have access to the clinical trial data. We will aim to have applied, and have access, to the raw clinical trial data by the end of April. Next, we will re-analyze the data and prepare waterfall plots illustrating PASI or DLQI improvements and spider plots assessing long-term drug efficacy – this should be completed by the end of June. We hope to have the Bayesian aggregate n-of-1 data prepared by the end of July. Both of the above steps constitute our planned analyses. August and September will be spent interpreting our prepared data presentation models and preparing the discussion and conclusions. The final six months of our 12-month data usage will be dedicated to preparing the manuscript for publication. We plan to have a draft ready for publication by the end of December 2019.

Dissemination Plan: 

We plan to publish the results in peer-reviewed dermatological journals. Potentially suitable target journals are, but not restricted to: J Am Acad Dermatol, Br J Dermatol, JAMA Dermatology.

Philip will apply to present the results, once published, at the Canadian Dermatology Association 2020 conference, the Skin Research Group of Canada 2020 conference, the Society of Investigative Dermatology 2020 conference, and the University of Alberta 2020 Day of Medical Research Symposium.

Bibliography: 

1. Stunnenberg, B. C. et al. Effect of Mexiletine on Muscle Stiffness in Patients With Nondystrophic Myotonia Evaluated Using Aggregated N-of-1 Trials. JAMA 320, 2344–2353 (12 11, 2018).
2. Chen, X. & Chen, P. A comparison of four methods for the analysis of N-of-1 trials. PLoS One 9, e87752 (2014).
3. Stunnenberg, B. C. et al. Combined N-of-1 trials to investigate mexiletine in non-dystrophic myotonia using a Bayesian approach; study rationale and protocol. BMC Neurol. 15, (2015).
4. Chia, P. L., Gedye, C., Boutros, P. C., Wheatley-Price, P. & John, T. Current and Evolving Methods to Visualize Biological Data in Cancer Research. J. Natl. Cancer Inst. 108, (08 2016).
5. Gillespie, T. W. Understanding Waterfall Plots. J Adv Pract Oncol 3, 106–111 (2012).

What is the purpose of the analysis being proposed? Please select all that apply.: 
Research that confirms or validates previously conducted research on treatment effectiveness
Preliminary research to be used as part of a grant proposal
Participant-level data meta-analysis:
Participant-level data meta-analysis uses only data from YODA Project
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

The main outcomes are 1) individual reduction of PASI scores over time and individual reduction in DLQI scores and 2) absolute values pf PASI and DLQI at all timepoints during treatment.
Secondary outcomes are 1) proportions of patients achieving PASI and DLQI reductions by 50%, 75%, 90%, and 100%, 2) proportion of patients achieving predetermined PASI values 0, <2, <5, 5, and 3) proportion of patients achieving DLQI values of 0, <2, and <5.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

-Failure of previous biologic therapy (yes/no).

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

-Sex (male/female)
-Body weight, body mass index, or presence of obesity (as available)
-Presence or absence of psoriatic arthritis (yes/no)

Statistical Analysis Plan: 

-Individual data (primary and secondary outcomes) will be plotted using R-scripting to visualize individual responses using waterfall plots (single endpoint) and spider plots (multiple endpoint in time) for different studied drugs and placebo. Further statistical analysis will be descriptive by analysing the overall shape of the curves and presence or absence of the visually obvious thresholds in the level of population responses.
-Depending on the results of this analysis, we would like to reserve the right for other types of data presentation, if applicable (e.g. movies showing longitudinal variability in clinical responses).
-For the Bayesian analysis, the independent variables listed above will be used as priors. In this analysis, we will include patients who received more than one block of therapy (e.g. active treatment and placebo). Individual patients will be treated as trials and analysed using meta-analysis methodology described by Chen and Chen2. We will use modelling with different numbers of patients (n=10, 15, 25, 50, 100) to analyse the sample size of individual patient trials that give equivalent statistical results to the frequentist analysis (methodology is described in reference 1). The response will be presented as the probability curve of achieving a predetermined outcome: PASI75 or PASI90 (this outcome will be the same as the primary or secondary outcomes in the original study).

How did you learn about the YODA Project?: 
Software Used: 
R
Associated Trials: 
<ol><li><a href="/node/1686">NCT00267969 - C0743T08 - A Phase 3, Multicenter, Randomized, Double-blind, Placebo Controlled Trial Evaluating the Efficacy and Safety of Ustekinumab (CNTO 1275) in the Treatment of Subjects With Moderate to Severe Plaque-type Psoriasis </a></li><li><a href="/node/1696">NCT00307437 - C0743T09 - A Phase 3, Multicenter, Randomized, Double-blind, Placebo-controlled Trial Evaluating the Efficacy and Safety of CNTO 1275 in the Treatment of Subjects With Moderate to Severe Plaque-type Psoriasis</a></li><li><a href="/node/3366">NCT01550744 - CNTO1275PSO3009 - A Phase 3b, Randomized, Double-blind, Active-controlled, Multicenter Study to Evaluate a "Subject-tailored" Maintenance Dosing Approach in Subjects With Moderate-to-Severe Plaque Psoriasis</a></li><li><a href="/node/3371">NCT02203032 - CNTO1959PSO3003 - A Phase 3, Multicenter, Randomized, Double-blind Study to Evaluate the Efficacy and Safety of Guselkumab for the Treatment of Subjects With Moderate to Severe Plaque-type Psoriasis and an Inadequate Response to Ustekinumab</a></li><li><a href="/node/3391">NCT00454584 - C0743T12 - A Phase 3, Multicenter, Randomized Study Comparing CNTO 1275 and Etanercept for the Treatment of Moderate to Severe Plaque Psoriasis</a></li><li><a href="/node/3411">NCT01059773 - CNTO1275PSO4004 - An Exploratory Trial to Assess Naturalistic Safety and Efficacy Outcomes in Patients With Moderate to Severe Plaque Psoriasis Transitiioned to Ustekinumab From Previous Methotrexate Therapy (TRANSIT)</a></li></ol>
Make Publicly Available : 
Year of Data Access: 
2019

2019-3846

Project Title: 
Prediction of response to neoadjuvant androgen deprivation therapy in patients with high risk and locally advanced prostate cancer
Specific Aims of the Project: 

Neoadjuvant ADT has been shown to be effective in a select group of patients. Until now, cohorts included in clinical trials have been too small to effectively evaluate the factors associated with a benefit from nADT. Our hypothesis is that there are certain clinical (patient/tumor-specific) factors associated with a benefit from nADT.
The aim of this project is to calculate a nomogram helping urologists to evaluate whether a patient diagnosed with high-risk prostate cancer would benefit receive nADT or not. This Nomogram may help to improve long-term survival in patients with a high chance of benefitting from nADT.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Incomplete Not Reviewed
Scientific Abstract: 

Backgronund: Neoadjuvant therapy is the standard of care for a variety of solid tumors, such as bladder, breast or colorectal cancer, and has been shown to provide a significant long-term survival benefit. Recent studies have shown that the latter seems to apply to neoadjuvant androgen deprivation therapy (nADT) in patients diagnosed with high-risk prostate cancer (PCa), as well. During recently published Phase II trials re-examining the question of nADT using newly developed drugs, some patients drastically responded to neoadjuvant therapy, while others only showed limited or no tumor response.
Objective: Consequently, this study aims to establish a predictive model to distinguish between patients benefitting from nADT and patients not benefitting from nADT using preoperative staging information and biomarkers.
Study-Design: Retrospective Data Analysis
Outcome measures: The primary outcome of this study will be to evaluate factors predicting pathologic complete response (minimal residual disease ≤ 0.5 after radical prostatectomy) to nADT in patients diagnosed with PCa categorized as “high risk”.
Statistical analysis: We will calculate a multivariable Cox proportional hazards regression model including the covariates mentioned above using pCR as our outcome. The result will then be shown in nomogram format to illustrate all independent predictors of pCR. Internal model validity will be evaluated by bootstrapping. The optimism of our model will be defined as the decrease in model performance in new subjects compared with performance in the sample under study.

Brief Project Background and Statement of Project Significance: 

Neoadjuvant therapy is the standard of care for a variety of solid tumors, such as bladder, breast or colorectal cancer, and has been shown to provide a significant long-term survival benefit (1-4). Recent studies have shown that the latter seems to apply to neoadjuvant androgen deprivation therapy (nADT) in patients diagnosed with high-risk prostate cancer (PCa), as well (5). Older studies, however, failed to observe this benefit resulting in nADT not being a guideline-recommended standard of care. These numbers, however, refer to patients undergoing ADT using luteinizing hormone-releasing hormone (LHRH) agonists and first-generation anti-androgens (6-7). Since then, new hormonal agents have been developed and successfully used in the treatment of metastatic PCa. Those drugs have been shown to play a potential role as neoadjuvant agents for PCa with promising results in certain patients (8). ADT bears a variety of potential side effects, which include, but are not limited to, cardiovascular disease, osteoporosis, cognitive decline and dementia, depression, and metabolic syndrome (9-12). During recently published Phase II trials re-examining the question of nADT, some patients drastically responded to neoadjuvant therapy, while others only showed limited or no tumor response (8).
In order to maximize therapy effectiveness on the one hand and reduce the incidence of those potential ADT side effects, on the other hand, it remains to be determined, which patients are most likely to benefit from nADT. Consequently, this study aims to establish a predictive model to distinguish between patients benefitting from nADT and patients not benefitting from nADT using preoperative staging information and biomarkers.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

All data will be derived from a pooled dataset consisting of different phase II trials examining nADT. Only patients undergoing nADT will be included in the analysis. We will perform a pooled analysis including all patients, as well as a subgroup analysis for the respective drugs under investigation.

Narrative Summary: 

Neoadjuvant therapy is the standard of care for a variety of solid tumors, such as bladder, breast or colorectal cancer, and has been shown to provide a significant long-term survival benefit. Recent studies have shown that the latter seems to apply to neoadjuvant androgen deprivation therapy (nADT) in patients diagnosed with high-risk prostate cancer (PCa), as well. Substances used for nADT in clinical trials are, amongst others, arbiraterone acetate or enzalutamide. We are planning to establish a predictive model to distinguish between patients benefitting from nADT and patients not benefitting from nADT using readlily available data from clinical trials.

Project Timeline: 

The anticipated start date will be May 30th, 2019. Data Analysis is expected to take approximately 2-3 months. Drafting the manuscript will take approximately 2 months. After a revision period of approximately 2 months, we plan on submitting the manuscript. Altogether, we plan on finishing this project by January 2020.

Dissemination Plan: 

The target audience for this project will be medical oncologists and urologists. Suitable journals for the manuscript will be "European Urology", "Journal of Clinical Oncology", "JAMA", "Journal of Urology".

Bibliography: 

1. Sauer, R., Becker, H., Hohenberger, W. et al.: Preoperative versus postoperative chemoradiotherapy for rectal cancer. N Engl J Med, 351: 1731, 2004
2. Meeks, J. J., Bellmunt, J., Bochner, B. H. et al.: A systematic review of neoadjuvant and adjuvant chemotherapy for muscle-invasive bladder cancer. Eur Urol, 62: 523, 2012
3. Vetterlein, M. W., Wankowicz, S. A. M., Seisen, T. et al.: Neoadjuvant chemotherapy prior to radical cystectomy for muscle-invasive bladder cancer with variant histology. Cancer, 123: 4346, 2017
4. Mauri, D., Pavlidis, N., Ioannidis, J. P.: Neoadjuvant versus adjuvant systemic treatment in breast cancer: a meta-analysis. J Natl Cancer Inst, 97: 188, 2005
5. McClintock, T. R., von Landenberg, N., Cole, A. P. et al.: Neoadjuvant Androgen Deprivation Therapy Prior to Radical Prostatectomy: Recent Trends in Utilization and Association with Postoperative Surgical Margin Status. Ann Surg Oncol, 26: 297, 2019
6. Shelley, M. D., Kumar, S., Wilt, T. et al.: A systematic review and meta-analysis of randomised trials of neo-adjuvant hormone therapy for localised and locally advanced prostate carcinoma. Cancer Treat Rev, 35: 9, 2009
7. Gleave, M. E., Goldenberg, S. L., Chin, J. L. et al.: Randomized comparative study of 3 versus 8-month neoadjuvant hormonal therapy before radical prostatectomy: biochemical and pathological effects. J Urol, 166: 500, 2001
8. McKay, R. R., Montgomery, B., Xie, W. et al.: Post prostatectomy outcomes of patients with high-risk prostate cancer treated with neoadjuvant androgen blockade. Prostate Cancer Prostatic Dis, 21: 364, 2018
9. Patil, T., Bernard, B.: Complications of Androgen Deprivation Therapy in Men With Prostate Cancer. Oncology (Williston Park), 32: 470, 2018
10. Dinh, K. T., Reznor, G., Muralidhar, V. et al.: Association of Androgen Deprivation Therapy With Depression in Localized Prostate Cancer. J Clin Oncol, 34: 1905, 2016
11. Nead, K. T., Gaskin, G., Chester, C. et al.: Androgen Deprivation Therapy and Future Alzheimer's Disease Risk. J Clin Oncol, 34: 566, 2016
12. Nead, K. T., Sinha, S., Nguyen, P. L.: Androgen deprivation therapy for prostate cancer and dementia risk: a systematic review and meta-analysis. Prostate Cancer Prostatic Dis, 20: 259, 2017
13. D'Amico, A. V., Whittington, R., Malkowicz, S. B. et al.: Pretreatment nomogram for prostate-specific antigen recurrence after radical prostatectomy or external-beam radiation therapy for clinically localized prostate cancer. J Clin Oncol, 17: 168, 1999
14. Gianni, L., Pienkowski, T., Im, Y. H. et al.: Efficacy and safety of neoadjuvant pertuzumab and trastuzumab in women with locally advanced, inflammatory, or early HER2-positive breast cancer (NeoSphere): a randomised multicentre, open-label, phase 2 trial. Lancet Oncol, 13: 25, 2012
15. Peintinger, F., Sinn, B., Hatzis, C. et al.: Reproducibility of residual cancer burden for prognostic assessment of breast cancer after neoadjuvant chemotherapy. Mod Pathol, 28: 913, 2015
16. Barentsz, J. O., Weinreb, J. C., Verma, S. et al.: Synopsis of the PI-RADS v2 Guidelines for Multiparametric Prostate Magnetic Resonance Imaging and Recommendations for Use. Eur Urol, 69: 41, 2016
17. Hens, N., Aerts, M., Molenberghs, G.: Model selection for incomplete and design-based samples. Stat Med, 25: 2502, 2006
18. Steyerberg, E. W.: Clinical prediction models : a practical approach to development, validation, and updating. New York, NY: Springer, pp. xxviii, 497 p., 2009

What is the purpose of the analysis being proposed? Please select all that apply.: 
Research on clinical prediction or risk prediction
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

The primary outcome of this study will be to evaluate factors predicting pathologic complete response (pCR) to nADT in patients diagnosed with PCa categorized as “high risk” by D’Amico et al (13). PCR is defined as either minimal residual disease (MRD) ≤ 0.5 cm or residual cancer burden (RCB) ≤ 0.25 cm3 after radical prostatectomy, serving as a proxy for an improved long-term survival (8,14,15).

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

We will evaluate a variety of independent variables. There is no main predictor yet.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

Baseline characteristics for each patient will include age at diagnosis, race, body mass index (BMI), smoking status, and Charlson Comorbidity Index (CCI). Tumor characteristics will include preoperative PSA level, clinical T-Stage, primary and secondary Gleason score, as well as the maximum percentage of tumor infiltration and preoperative androgen receptor (AR) expression of the initial biopsy. Radiographic tumor characteristics will include MRI-derived tumor size, location, radiographic T-stage, and Prostate Imaging Reporting and Data System (PIRADS) score Version II (16), for all patients available

Statistical Analysis Plan: 

We will perform a descriptive analysis of baseline characteristics by generating medians and interquartile ranges (IQR) for continuous variables and frequencies and proportions for categorical variables.
We will calculate a multivariable Cox proportional hazards regression model including the covariates mentioned above using pCR as our outcome. In order to reach the most informative and parsimonious model, we will perform backward selection according to the Aikaike information criterion (17). The result will then be shown in nomogram format to illustrate all independent predictors of pCR. Internal model validity will be evaluated by bootstrapping. We will estimate and test the coefficients of our final model in using n = N (patient number of the original sample) bootstrap samples. The optimism of our model will be defined as the decrease in model performance in new subjects compared with performance in the sample under study (Difference in coefficients in the original and bootstrap sample) (18).

How did you learn about the YODA Project?: 
Software Used: 
STATA
Associated Trials: 
<ol><li><a href="/node/3774">NCT00924469 - COU-AA-201-DFCI - A Phase 2 Open-Label, Randomized, Multi-center Study of Neoadjuvant Abiraterone Acetate (CB7630) Plus Leuprolide Acetate and Prednisone Versus Leuprolide Acetate Alone in Men With Localized High Risk Prostate Cancer</a></li></ol>
Make Publicly Available : 

2019-3829

Project Title: 
Predicting Individual Treatment Effects for Doxil/Caelyx in Malignant Ovarian Cancer
Specific Aims of the Project: 

• Extend the PITE approach for use with trials in which survival is the primary outcome. The PITE framework has limited utility in cancer research until it can be shown to be effective at predicting treatment effects on individual survival.
• Develop an approach using PITE to obtain predictions of individual treatment effects & predictive intervals under each arm in a multi-arm trial. This aim increases the utility of PITEs by obtaining individual predictions & predictive intervals for multiple interventions.
• Establish & demonstrate a procedure for selecting a statistical model to estimate PITE in oncology trials.
The overarching goal is to develop tests of heterogeneity and then to obtain predictions useful to oncologists and their patients.

What type of data are you looking for?: 
Individual Participant-Level Data, which includes Full CSR and all supporting documentation

Application Status

Approved Pending DUA Signature
Scientific Abstract: 

Background: While personalized medicine has been shown to have great promise for the treatment of cancers, new statistical approaches which have promise for finding a greater individual differences in the effects of treatment have yet to be widely implemented.
Objective. This study will utilized the predicted individual treatment effect (PITE) framework to test for evidence of individual differences in the effects of Doxil for the treatment of ovarian cancer and to examine the predictors which contribute most to observed differences.
Study Design. This is a secondary data analysis of the Doxil clinical trial.
Participants. All subjects included in the primary analyses for the clinical trial will be included.
Main Outcome. The primary outcome will be progression free survival.
Analyses. We propose to utilize the predicted individual treatment effect (PITE) framework in which available baseline covariates are utilized to predict the treatment effect for each individual. We will conduct a permutation test to evaluate whether there is overall evidence for individual differences in treatment effects. If so, we will describe the range of differential effects, conduct cross validation analyses, and assess variable importance to provide information about the primary contributors to the individual differences in the effects of the intervention.

Brief Project Background and Statement of Project Significance: 

Randomized controlled trials (RCTs) are typically designed to assess the average causal effect of a treatment.(1,2) Implicit in this approach is the assumption that the estimated treatment effect is reasonable for at least most individuals in the population. One of the challenges in oncology is the vast amount of heterogeneity within and between types of cancers, heterogeneity between individuals, and even heterogeneity between tumor sites within the same patient.(3) Thus, the very nature of cancer casts doubt on the appropriateness of assessing only the average effects of treatment. This is the rationale behind the ideological movement toward personalized medicine and targeted therapies.(4)

Precision medicine can be greatly facilitated by statistical methods for evaluating heterogeneity in treatment effects. We note that in oncology the control condition is likely to be standard care or an alternative treatment arm rather than a no-treatment control. The PITE approach allows predicting treatment effects for patients that have not originally been part of the clinical trial. The PITE approach is a significant contribution because it results in predictions of the effect of the intervention for any individual for whom the covariates can be measured.

We have developed and is in the process of evaluating a test for the presence of any individual differences in the treatment effect given all available covariates. This permutation based test(5) evaluates whether the variance (individual differences) of the PITE observed in a given RCT is greater than would be expected due to chance. This test for heterogeneity in treatment effects is a significant contribution to oncology as it provides a tool to assess the presence of individual differences in treatment effects in any RCT. It has been found that more than half of RCTs in oncology fail to show the expected effects(6), a possible reason for this is that while a given treatment benefits some individuals, it does not have a greater effect than standard care on average, and may do worse for some other individuals. The PITE approach with its test for individual differences in treatment effects and patient level predictions has implications for treatments that do not show benefits on average, it can be used to test for evidence that the intervention benefits some patients and it allows them to be identified.

We have utilized the PITE approach to test for heterogeneity in treatment effects and predict individual response to treatment for both binary and continuous outcomes.(7) We propose to use applied data and simulations to better understand what survival models can be effectively used for PITE and the conditions under which those models will be successful. We will also extend this work to use methods for predicting survival outcomes from the machine learning literature which can better deal with a large number of predictors. This non-trivial extension of the PITE approach to predicting individual differences in the effects of treatments on survival time will be highly significant for the application of PITEs to oncology.

Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

All subjects included in the primary analyses for the clinical trial will be included. Our analyses rely on having as many observations as possible. If baseline outcome data are missing, we will perform multiple imputation.

Narrative Summary: 

Research protocols that are used to test cancer treatments typically focus on estimating average treatment response. More than half of RCTs (randomized control trials) in oncology fail to show expected effects. This might be, at least partially, a result of RCTs' focus on average effects rather than individual effects. This proposal is based on the premise that individual differences in the effects of oncology treatments are to be expected and that medically relevant predictions of individual response may often be obtained using the full constellation of available data. We propose to demonstrate a method for obtaining predicted individual treatment effects (PITEs) in an oncology RCT.

Project Timeline: 

Project start date: August 15, 2019
Analysis completion date: February 15, 2020 – Because the proposed analyses are quite time consuming and some decisions may need to be guided by simulations we anticipate this taking 6 months.
Draft of manuscript: May 15, 2020
Submit manuscript for publication: June 15, 2020
Results returned to YODA: July 15, 2020

Dissemination Plan: 

We anticipate that the results we be published as a demonstration of the PITE method in journals that focus on applied statistics in medicine. One such journal might be Statistics in Medicine. If the results are particularly telling, we would want to submit a paper for publication in an oncology journal, emphasizing the practical implications for physician diagnostic practices and patient decision-making.

Bibliography: 

1. Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol. 1974;(66):688-701
2. Holland PW. Statistics and causal inference. J Am Stat Assoc. 1986;81(396):945-960.
3. Venook AP, Arcila ME, Benson AB, et al. NCCN work group report: Designing clinical trials in the era of multiple biomarkers and targeted therapies. J Natl Compr Cancer Netw. 2014;12(11):1629-1649. doi:10.1007/s00394-015-0841-1.A.
4. Mitikiri ND, Reese ES, Hussain A, Onukwugha E, Pritchard D, Dubois R. The emerging relevance of heterogeneity of treatment effect in clinical care: A study using stage IV prostate cancer as a model. J Comp Eff Res. 2013;2(6):605.
5. Higgins JJ. Introduction to Modern Nonparametric Statistics. Duxbury Press; 2003.
6. Gan HK, You B, Pond GR, Chen EX. Assumptions of expected benefits in randomized phase III trials evaluating systemic treatments for cancer. J Natl Cancer Inst. 2012;104(8):590-598. doi:10.1093/jnci/djs141.
7. Lamont A, Lyons MD, Jaki T, et al. Identification of predicted individual treatment effects in randomized clinical trials. Stat Methods Med Res. March 2016:962280215623981. doi:10.1177/0962280215623981.

What is the purpose of the analysis being proposed? Please select all that apply.: 
Develop or refine statistical methods
Research on clinical prediction or risk prediction
Supplementary Material: 
Submit Data Request: 
Main Outcome Measure and how it will be categorized/defined for your study: 

The endpoints to be analyzed in this research are the same as in the trial: survival time, survival, tumor size (if available), and safety.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

We plan to use all baseline covariates available.

Statistical Analysis Plan: 

The primary purpose of this study is to demonstrate the PITE method using the DOXIL/CAELYX trial. This method has been previously published and additional methodological papers on the PITE method with application to amyotrophic lateral sclerosis (ALS) are currently under review. For the proposed study we will implement the PITE approach using a beta R package being developed by our team. The effect of interest in this study is individual predictions of treatment effects, our analyses will include: 1) an assessment of significant variation in the effects of pegylated liposomal doxorubicin in the requested RCT data using a permutation test which compares the variability of the estimated PITEs to chance variability (the distribution of which is derived using random permutations of treatment status); 2) given that there is significant variability of treatment effects, we will estimate the predicted treatment effect for each individual in the original trial, a major outcome of the study is the description of the type of individual differences predicted by the PITE approach; 3) finally, we will estimate predictive intervals for the PITEs and describe both the individual level predictions as well as the differences in precision across individuals. These steps are described in more detail below.
Our analyses begin by testing for evidence of significant variability in PITEs across individuals in in the requested RCT data. Because the permutation test for assessing variability in PITE is a model based test, it requires PITE estimates to assess significant variability in PITE. The PITE approach is a general framework for estimating individual variability designed to work with any predictive algorithm or model. Thus, our first task is determining the predictive method to use to estimate PITE. This trial includes a moderate number of baseline covariates and some evidence of differential treatment effects. Because there are a moderate number of covariates and most are categorical, we expect that a linear model with 2-way interactions specified within both treatment and control conditions (functionally this is a 3-way interaction because all variables interact with treatment) may be the most efficient model. However, we will also consider Bayesian Additive Regression Trees and LASSO models with interactions. Simulations in which the baseline covariates and treatment status are utilized with outcome data generated different types of heterogeneity in treatment effects will be used to determine which predictive method to use.
Once the predictive model or algorithm to be used to generate PITEs has been identified, PITEs are estimated: 1) generate a predictive algorithm for the outcome under both treatment and control conditions; 2) obtain predictions for each individual of their outcome under each condition; 3) PITE is the difference between the two predictive models. Once PITEs are obtained, the first question, whether there is significant variability in the PITEs, is assessed by comparing the observed variability of the PITEs to the distribution of variability in PITEs given that treatment is unrelated to all predictors and outcomes (this distribution is obtained by permuting treatment status many times and obtaining PITE estimates for each). Given significant variability in PITE estimates, the observed distribution of PITEs will be described, answering the second aim of this study. Finally, we will apply previously derived methods (currently under review) to assess the variability of the PITEs. Both the PITE estimates and the predictive intervals will be described in the resulting research reports.

Although a research team has been involved in developing the PITE approach, (7) analysis of these data will be conducted exclusively by team members at the University of New Mexico. No other team members will be involved at this stage. Further, we have not secured access to other oncology data sources and have dropped this from the proposal.

How did you learn about the YODA Project?: 
Software Used: 
R
Associated Trials: 
<ol><li><a href="/node/646">NCT00653952 - 30-57 - A Phase 3, Randomized, Open-Label, Comparative Study of CAELYX® versus Paclitaxel HCl in Patients with Epithelial Ovarian Carcinoma Following Failure of First-Line, Platinum-Based Chemotherapy</a></li><li><a href="/node/647">30-49 - A Phase 3, Randomized, Open-Label, Comparative Study of DOXIL/CAELYX® versus Topotecan HCl in Patients with Epithelial Ovarian Carcinoma Following Failure of First-Line, Platinum-Based Chemotherapy</a></li></ol>
Make Publicly Available : 

Pages