Skip to main content

2020-4535

Research Proposal

Project Title: 
Development of a Novel Disease Activity Index for Crohn’s Disease
Scientific Abstract: 

Background: A composite index comprised of patient-reported symptoms and objective measures of inflammation that meets regulatory guidelines does not exist for CD. A sensitive disease activity index would be a significant advance by providing a tool to evaluate the efficacy of new therapeutics in early drug trials.
Objective: The objective of this study is to develop and validate a novel continuous CD activity index consisting of a potential combination of clinical, endoscopic, histologic, and/or biomarker items.
Study Design: Data collected during the conduct of 2 trials that evaluated the efficacy and safety of ustekinumab in CD (UNITI-1 [NCT01369329] and UNITI-2 [NCT01369342]) will be used for this study. The best subsets of clinical, endoscopic, histologic, and biomarker items that predict treatment assignment or CD activity after induction therapy will be selected for the new index, which will be validated using the UNITI-2 dataset
Participants: All participants enrolled in the UNITI-1 and UNITI-2 trials.
Main Outcome measure(s): CDAI, IBDQ, SES-CD, GHAS, and CD biomarkers
Statistical Analysis: Regression models with methods to select the best subsets of predictors will be used to identify items in the UNITI-1 dataset that are strongly and independently associated with treatment or CD activity to be included in a novel index. Longitudinal validity of the new index will be evaluated by determining the correlation between index scores and biomarkers. Standardized effect size will be used to assess responsiveness. UNITI-2 data will be used to validate the index.

Brief Project Background and Statement of Project Significance: 

A large number of drug candidates for CD are currently in development (1) which has strained patient, investigator, and regulatory resources. Consequently, there is a need for optimized trial designs that reduce the cost and time required to bring effective therapies to market. Development of novel, efficient outcome measures is an attractive strategy for streamlining drug development.

The Crohn’s Disease Activity Index (CDAI), which has been used as a regulatory endpoint in clinical trials for over 4 decades, is an empirically-derived instrument consisting of patient-reported items, physical signs, and the hematocrit, which are predictive of a physician’s global assessment of disease activity (2). Although the CDAI is a continuous outcome measure, it is usually analyzed and reported as a binary outcome measure (e.g., scores ≤ 150 points for clinical remission). In recent years, treatment of CD has shifted from control of symptoms to attainment of both clinical and endoscopic remission, as endoscopic remission is associated with better long-term outcomes (3-5).

The importance of resolution of endoscopic disease as a distinct treatment target is underscored by the weak to moderate correlation between the CDAI and validated endoscopic scores (6, 7). Although symptom control is important, this discordance calls into question the validity of the CDAI as the primary measure of therapeutic efficacy, especially in early phase trials where sample sizes are smaller and there may be limited statistical power to detect treatment differences between multiple dose arms (8). In recognition of this issue, US and European regulatory agencies have mandated the use of symptom-defined remission (using patient-reported symptoms of the CDAI) and endoscopic response (defined as a 50% reduction in the Simple Endoscopic Score for CD) as coprimary outcomes in registration trials (9-11); however, this co-primary endpoint is unlikely to be a sensitive indicator of treatment efficacy as it is based upon a dichotomous definition of the two endpoints.

In early phase drug development, where signal detection is paramount and sample size is limited, the use of a continuous, composite outcome measure has the potential for increased statistical efficiency (12). A single composite measure may minimize the risk of missing an efficacy signal, lower drug development costs by allowing for smaller, more efficient early phase trials, and accelerate the delivery of effective therapies to patients (11).

Recently, a composite measure of ulcerative colitis (UC) disease activity consisting of clinical, endoscopic, and histologic variables was developed (UC-100) and externally validated as a tool for early drug development in UC (13). The model on which the UC-100 score is based shows greater responsiveness to detect change than traditional endpoints. The use of the UC-100 score as a continuous endpoint in trials may reduce sample size requirements. The aim of the current study is to develop an analogous tool for use in early phase CD trials, lowering drug development costs and accelerating bringing new therapies to market.

Specific Aims of the Project: 

The primary aim of this study is to develop a novel continuous CD activity index consisting of a potential combination of clinical, endoscopic, histologic, and/or biomarker items. Following development of the novel index, the aims of this study are too:
1. Assess the longitudinal validity of the novel index, by evaluating the correlation between changes in the index scores and changes in biomarker levels and health-related quality of life
2. Validate the new index using the UNITI-2 dataset

What is the purpose of the analysis being proposed? Please select all that apply.: 
Develop or refine statistical methods
Research on clinical trial methods
Research on clinical prediction or risk prediction
Software Used: 
STATA
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study: 

This study will use subject-level data from the baseline and end of induction visits from the UNITI-1 and UNITI-2 trials. Data will be collected for all subjects, regardless of treatment, with clinical (CDAI), endoscopic (Simple Endoscopic Score for CD [SES-CD]), histologic (Global Histological Disease Activity Score [GHAS]), health-related quality of life (Inflammatory Bowel Disease Questionnaire [IBDQ]) and biomarker (C-reactive protein [CRP], fecal calprotectin [FCP], hemoglobin, and platelet concentration) data at both visits.

Main Outcome Measure and how it will be categorized/defined for your study: 

The main outcome of this study will be a novel composite disease activity index for CD, potentially comprised of clinical, endoscopic, histologic, and biomarker outcomes assessed in the UNITI trials. The longitudinal validity of the new index will be evaluated by determining the correlation between changes in the new index scores following ustekinumab induction therapy and change in disease biomarkers (CRP and FCP) and the IBDQ. The strength of the correlations will be interpreted based on Cohen’s benchmarks, where correlations of 0.1, 0.3, and 0.5 suggest small, medium, and large correlations, respectively (14). Index responsiveness, defined using treatment assignment as the criterion, will be quantified using the standardized effect size,15 where effect sizes of 0.2, 0.5, and 0.8 will be interpreted as small, moderate, and large responsiveness, respectively.

Main Predictor/Independent Variable and how it will be categorized/defined for your study: 

For index model development, CDAI, IBDQ, SES-CD and GHAS scores and CRP, FCP, hemoglobin, and platelet levels will be considered for inclusion. Regression models with methods to select the best subsets of predictors will be used to identify the items that are strongly and independently associated with the dependent variables outlined in the Statistical Analysis Plan section, for inclusion in the model.

Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: 

In addition to CDAI scores, component subscores (e.g., stool frequency and abdominal pain subscores) will be required for this study.

Statistical Analysis Plan: 

Dependent variables for model building include: 1) Treatment assignment (ustekinumab or placebo); and 2) CDAI score. Potential exploratory dependent variables include: 1) Stool frequency subscore < 1.5 and CRP < 5 mg/L, and 2) Abdominal pain subscore < 1 and CRP < 5 mg/L (or variations thereof).

Model Development. Summary statistics, including sample distributions for each variable, will be explored, followed by exploratory bivariable analyses to determine associations between candidate index items and the candidate dependent variables. Multivariable relations among independent variables will be explored using principal components analysis with varimax rotation.

Logistic regression models will be used for binary dependent variables and linear regression models for continuous dependent variables. Model assumptions will be evaluated using methods specific to the type of dependent variable selected.

Exploratory bivariate analyses between the dependent variables and each selected item will be performed to guide item coding. Items will be coded as continuous if a linear relationship is demonstrated between change in the item score and change in the dependent variable. If a linear relationship is not evident, the bivariate relationships will be used to collapse item levels. Multivariable regression models will be built using either 1) the best subsets of variables with Akaike Information criteria, or 2) backwards elimination using items with a P < .15. The final model will be used to develop score points for items that are deemed clinically relevant (16, 17).

Binary Dependent Variable. If the dependent outcome is binary (i.e., treatment assignment), index performance will be assessed in terms of discrimination, prediction error, and calibration. Discrimination will be measured using the area under the curve of the receiver operating characteristic (AUROC)(17). Prediction error will be quantified with Nagelkerke’s R2 and the Brier Score(18, 19). The Hosmer-Lemeshow goodness-of-fit test will be used to evaluate calibration (20). Test values with P > .05 will be considered indicative of adequate model fit. Internal index validation will be conducted using bootstrap resampling with 1000 replications to determine reproducibility (21).

Continuous Dependent Variable. If the dependent variable is continuous (i.e., CDAI score), the stability of the final model will be assessed and calibrated using the bootstrap method with 2000 replicates (22). This involves obtaining the optimism-corrected performance (e.g., R2, calibration intercept, and slope) as the difference between the apparent performance in the study sample and the average optimism, which will be obtained using the bootstrap method with 2000 replications. Within each replication, the bootstrap sample will be used to develop a model using backward elimination, with apparent performance estimated. This model will be tested on data points in the original sample not included in the bootstrap sample to obtain test performance and the optimism calculated. Average optimism of 2000 replications will then be obtained.

Longitudinal Validity. The change in new index scores will be correlated with change in disease biomarkers (e.g., CRP and FCP levels) and change in IBDQ scores from baseline. Correlations will be interpreted based on Cohen’s benchmarks (14).

External Index validation. The UNITI-2 dataset will be used to evaluate model performance. In the case of poor performance, the model will be adjusted based on the UNITI-2 dataset using methods described in the literature (23).

Outcome development. The coefficients from the final model will be standardized by dividing by the smallest coefficient and then rounding to allow for simple calculation of the new index.

Responsiveness testing. Index responsiveness, defined using treatment assignment as the criterion, will be quantified using standardized effect size (15).

Narrative Summary: 

Management of Crohn’s disease (CD) has focused on controlling symptoms; however, therapies that treat not only the symptoms but also underlying disease process result in better long-term outcomes. A sensitive measure of CD that includes both disease symptoms and objective measures of disease activity does not exist. The purpose of this study is to develop a novel index for CD that includes measurements of symptoms as well as endoscopic and microscopic measures. A new sensitive measure of CD would be a significant advance by providing a tool that could be used in early drug trials for determining the effectiveness of new drug therapies.

Project Timeline: 

This study is expected to take less than 1 year following receipt of the clinical trial data to submission of a manuscript for publication. Following receiving the data, it will take approximately 3 to 4 months for model development and an additional 4 months to validate and potentially adjust the model using the UNITI-2 dataset. A manuscript will be prepared and submitted for publication approximately 10 months after receiving the trial data.

Assuming this request is approved and data is received in February 2021, results will be reported back to the YODA project and a manuscript submitted in December 2021.

Dissemination Plan: 

The results of this study will be presented at international gastroenterology conferences, such as: Digestive Diseases Week, European Crohn’s and Colitis Organisation’s annual meeting, and United European Gastroenterology Week and ultimately published in a top-ranked gastroenterology journal, such as the American Journal of Gastroenterology, Gut, or Gastroenterology.

Bibliography: 

1. Duijvestein M, Battat R, Vande Casteele N, D'Haens GR, Sandborn WJ, Khanna R, et al. Novel therapies and treatment strategies for patients with inflammatory bowel disease. Curr Treat Options Gastroenterol. 2018 Mar;16(1):129-46.
2. Best WR, Becktel JM, Singleton JW, Kern F, Jr. Development of a Crohn's disease activity index. National Cooperative Crohn's Disease Study. Gastroenterology. 1976 Mar;70(3):439-44.
3. Baert F, Moortgat L, Van Assche G, Caenepeel P, Vergauwe P, De Vos M, et al. Mucosal healing predicts sustained clinical remission in patients with early-stage Crohn's disease. Gastroenterology. 2010 Feb;138(2):463-8; quiz e10-1.
4. Froslie KF, Jahnsen J, Moum BA, Vatn MH, Group I. Mucosal healing in inflammatory bowel disease: results from a Norwegian population-based cohort. Gastroenterology. 2007 Aug;133(2):412-22.
5. Shah SC, Colombel JF, Sands BE, Narula N. Systematic review with meta-analysis: mucosal healing is associated with improved long-term outcomes in Crohn's disease. Aliment Pharmacol Ther. 2016 Feb;43(3):317-33.
6. Khanna R, Zou G, D'Haens G, Rutgeerts P, McDonald JW, Daperno M, et al. Reliability among central readers in the evaluation of endoscopic findings from patients with Crohn's disease. Gut. 2016 Jul;65(7):1119-25.
7. Khanna R, Nelson SA, Feagan BG, D'Haens G, Sandborn WJ, Zou GY, et al. Endoscopic scoring indices for evaluation of disease activity in Crohn's disease. Cochrane Database Syst Rev. 2016 8:CD010642.
8. Peyrin-Biroulet L, Reinisch W, Colombel JF, Mantzaris GJ, Kornbluth A, Diamond R, et al. Clinical disease activity, C-reactive protein normalisation and mucosal healing in Crohn's disease in the SONIC trial. Gut. 2014 Jan;63(1):88-95.
9. European Medicines Agency. Guideline on the development of new medicinal products for the treatment of Crohn’s disease (Revision 2). 2018 Available from: https://www.ema.europa.eu/documents/scientific-guideline/guideline-devel....
10. US Food and Drug Administration (FDA) Guidance for Industry: Patient-Reported Outcome Measures Use in Medical Product Development to Support Labeling Claims. In: Department of Health and Human Services (US) FaDA, Center for Drug Evaluation and Research (CDER), ed. Rockville, MD: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformati..., . 2009.
11. Williet N, Sandborn WJ, Peyrin-Biroulet L. Patient-reported outcomes as primary end points in clinical trials of inflammatory bowel disease. Clin Gastroenterol Hepatol. 2014 Aug;12(8):1246-56 e6.
12. Montori VM, Permanyer-Miralda G, Ferreira-Gonzalez I, Busse JW, Pacheco-Huergo V, Bryant D, et al. Validity of composite end points in clinical trials. BMJ. 2005 Mar 12;330(7491):594-6.
13. Jairath V, Jeyarajah J, Zou G, Parker CE, Olson A, Khanna R, et al. A composite disease activity index for early drug development in ulcerative colitis: development and validation of the UC-100 score. Lancet Gastroenterol Hepatol. 2019 Jan;4(1):63-70.
14. Cohen J. Statsitical power for the behavioral sciences. New York, NY: Routledge Academic. 1988.
15. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Control Clin Trials. 1991 12(4 Suppl):142S-58S.
16. Labarere J, Renaud B, Fine MJ. How to derive and validate clinical prediction models for use in intensive care medicine. Intensive Care Med. 2014 Apr;40(4):513-27.
17. Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KG. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol. 2003 May;56(5):441-7.
18. Nagelkerke NJD. A note on a general definition of the coefficient of determination. Biometrika. 1991 78(3):691-2.
19. Brier SS. Analysis of contingency tables under cluster sampling. Biometrika. 1980 67(3):591-6.
20. Hosmer DWL, S. A goodness-of-fit test for the multiple logistic regression model. Commun Stat - Theory Methods. 1980 9(10):1043-69.
21. Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc. 1983 78(382):316-31.
22. Efron B, Tibshirani R. Improvements on Cross-Validation: The .632+ Bootstrap Method. J Am Stat Assoc. 1997 92(438):548-60.
23. Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004 Aug 30;23(16):2567-86.