Conflict of Interest
Request Clinical TrialsAssociated Trial(s):
- NCT00036439 - A Randomized, Placebo-controlled, Double-blind Trial to Evaluate the Safety and Efficacy of Infliximab in Patients With Active Ulcerative Colitis
- NCT00096655 - A Randomized, Placebo-controlled, Double-blind Trial to Evaluate the Safety and Efficacy of Infliximab in Patients With Active Ulcerative Colitis
- NCT00336492 - A Phase 3, Randomized, Open-label, Parallel-group, Multicenter Trial to Evaluate the Safety and Efficacy of Infliximab (REMICADE) in Pediatric Subjects With Moderately to Severely Active Ulcerative Colitis
- NCT00487539 - A Phase 2/3 Multicenter, Randomized, Placebo-controlled, Double blind Study to Evaluate the Safety and Efficacy of Golimumab Induction Therapy, Administered Subcutaneously, in Subjects with Moderately to Severely Active Ulcerative Colitis
- NCT01551290 - A Phase 3, Multicenter, Randomized, Double-Blind, Placebo-Controlled Study Evaluating the Efficacy and Safety of Infliximab in Chinese Subjects With Active Ulcerative Colitis
- NCT00488631 - A Phase 3 Multicenter, Randomized, Placebo-controlled, Double-blind Study to Evaluate the Safety and Efficacy of Golimumab Maintenance Therapy, Administered Subcutaneously, in Subjects With Moderately to Severely Active Ulcerative Colitis
- NCT00488774 - A Phase 2/3 Multicenter, Randomized, Placebo-controlled, Double-blind Study to Evaluate the Safety and Efficacy of Golimumab Induction Therapy, Administered Intravenously, in Subjects With Moderately to Severely Active Ulcerative Colitis
- NCT01863771 - A Safety and Effectiveness Study of Golimumab in Japanese Patients With Moderately to Severely Active Ulcerative Colitis
- NCT01988961 - A Study to Evaluate the Accuracy of a Subset of the Length-109 Probe Set Panel (a Genetic Test) in Predicting Response to Golimumab in Participants With Moderately to Severely Active Ulcerative Colitis
Request Clinical Trials
Data Request StatusStatus: Ongoing
Project Title: Validation of an artificial intelligence model on central read endoscopic disease severity in patients with ulcerative colitis
Endoscopic evaluation of disease severity of ulcerative colitis has shown high inter- and intra-observer variance. However, it remains an essential inclusion criterion and an outcome in many medical trials. Central reading has therefore become necessary to unify the assessment of disease severity, ensuring unbiased evaluation and reducing the observer variabilities. However, even central reading has demonstrated observer variance. The cost associated with central reading is high, with the risk of patients not being included in trials due to delayed assessment. We have developed an artificial intelligence (AI) model that can reliably score endoscopic disease severity during live endoscopy and on recordings or pictures from procedures.
We aim to evaluate endoscopic videos or images from all trials conducted in ulcerative colitis patients and compare the model?s disease severity classification with the central read scores.
Endoscopic videos or images from 10 trials (4.679 patients) will be requested. All videos or images will be analysed by the AI model and compared to the original scores.
Patients diagnosed with ulcerative colitis
The area under the receiver operating characteristic (AUROC) >0.90 & accuracy >0.85.
Sensitivity and specificity will be reported as secondary evaluation metrics. We will also investigate the false positives and negatives of the AI model to evaluate its pitfalls.
We will apply the following metrics/analysis: AUROC, accuracy, sensitivity, specificity, Cohens Kappa, and adjusted Cohens Kappa.
Brief Project Background and Statement of Project Significance:
A key component in the care of UC patients is the monitoring of disease activity and timely response with treatment. Disease activity is defined by a combination of clinical and biochemical markers as well as endoscopy to evaluate mucosal inflammation. Current guidelines recommend mucosal healing, most commonly defined as a Mayo endoscopic subscore (MES) of 0?1, as the treatment target(2). However, endoscopic assessment suffers from substantial intra- and inter-observer variation limiting the reliability of individual assessments(3).
Studies of clinicians scoring endoscopic images have shown inter-observer kappa values between 0.45?0.75[1?3]. In light of these results, central reading emerged to standardize endoscopic evaluation for clinical trials and other research projects. However, although central reading improved the evaluation process, the results showed inter-observer reliability of only 0.79 and intra-observer reliability to be 0.89, measured using weighted kappa values [1,4].
We have developed a new automated way of evaluating endoscopic images from UC patients for both clinical and academic purposes. We have demonstrated that our deep learning (DL) model is highly capable of distinguishing MES 0 from 1?3, MES 0?1 from 2?3, and differentiating between all four MES levels of activity.
We believe our model can support both clinical and academic assessment of disease severity in UC patients with an accuracy of 88. An application example could be in an RCT study using only the AI model as a central reader. Such a protocol reduces the time and expenditures associated with evaluating the endoscopic disease severity compared to a human central reader.
Furthermore, this can help standardize the field and help future clinical trials with inclusion and outcome measurements by discounting ineligible patients and making studies more easily and accurately comparable.
Specific Aims of the Project: The aim of this study is to run our newly developed AI model on endoscopic images or videos from all clinical trials that have been conducted on patients with ulcerative colitis. The aim is to evaluate the accuracy of the AI model compared to centrally read images and assess whether such a tool can be used as a replacement for central reading.
Study Design: Methodological research
What is the purpose of the analysis being proposed? Please select all that apply.: Confirm or validate previously conducted research on treatment effectiveness Research on clinical trial methods Other
Software Used: Python
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study:
Data source: NCT00487539, NCT00488631, NCT00488774, NCT01863771, NCT01988961, NCT00036439, NCT00096655, NCT00336492, NCT01551290, NCT02407236
Inclusion criteria: patients with endoscopic images/videos
Exclusion criteria: none
Primary and Secondary Outcome Measure(s) and how they will be categorized/defined for your study:
The area under the receiver operating characteristic (AUROC) >0.90 & accuracy >0.85 will be deemed satisfactory. AUROC < 0.90 but >0.80 & accuracy 0.75 will be deemed moderate. Everything below will be deemed unsatisfactory.
Sensitivity and specificity will be reported as secondary evaluation metrics. We will also investigate the false positives and negatives of the AI model to evaluate its pitfalls using machine learning techniques.
Main Predictor/Independent Variable and how it will be categorized/defined for your study: Mayo endoscopic subscore (MES) will be the main variable in which the AI model shall correctly classify. The MES is a categorical score from 0-3.
Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study: Metadata/demographical and treatment data from before the assessment of the endoscopic disease severity will be required to get insight into images that are either false positively or negatively classified by the AI model.
Statistical Analysis Plan: We will apply the following metrics/analysis: AUROC, accuracy, sensitivity, specificity, Cohens Kappa, and adjusted Cohens Kappa. Statistical regression models and machine learning models will be used to assess metadata/demographical and treatment data in which could have resulted in wrongly classified images, and get an insight into pitfalls our AI model might have.
Narrative Summary: Ulcerative colitis is a chronic disease that is often diagnosed in youth. A relapsing disease course characterises the disease despite treatment. Endoscopic evaluation is regarded as the gold standard for objectively measuring disease severity. However, it has a high observer variance; even with central reading, which is the gold standard in clinical trials. We have developed an artificial intelligence (AI) model that can classify endoscopic severity on par with experts in the field and aim to further validate it on images and videos from clinical trials scored by central reading. If successful, central reading can, in the future, be conducted by our AI model, saving time and resources.
Project Timeline: The first study report is expected 6 months from data access. The manuscript will be submitted the same time as the study report.
Dissemination Plan: Positive, negative and inconclusive results will be submitted for publication in a peer-reviewed international and/or Danish journal or published for public access if rejected. Results will be presented at national and international conferences. Bobby Lo will be the first author. The remaining authorship will be delegated according to the Vancouver guidelines.
 Samaan MA, Mosli MH, Sandborn WJ, Feagan BG, D?Haens GR, Dubcenco E, et al. A systematic review of the measurement of endoscopic healing in ulcerative colitis clinical trials: Recommendations and implications for future research. Inflamm Bowel Dis 2014;20:1465?71. https://doi.org/10.1097/MIB.0000000000000046.
 Fernandes SR, Pinto JSLD, Marques Da Costa P, Correia L. Disagreement among Gastroenterologists Using the Mayo and Rutgeerts Endoscopic Scores. Inflamm Bowel Dis 2018;24:254?60. https://doi.org/10.1093/ibd/izx066.
 Mohammed Vashist N, Samaan M, Mosli MH, Parker CE, Macdonald JK, Nelson SA, et al. Endoscopic scoring indices for evaluation of disease activity in ulcerative colitis. Cochrane Database Syst Rev 2018;2018. https://doi.org/10.1002/14651858.CD011450.pub2.
 Feagan BG, Sandborn WJ, D?Haens G, Pola S, McDonald JWD, Rutgeerts P, et al. The role of centralized reading of endoscopy in a randomized controlled trial of mesalamine for ulcerative colitis. Gastroenterology 2013;145. https://doi.org/10.1053/j.gastro.2013.03.025.
 Ket SN, Palmer R, Travis S. Endoscopic Disease Activity in Inflammatory Bowel Disease 2015. https://doi.org/10.1007/s11894-015-0470-0.
Lo, Bobby MD1,2; Liu, ZhuoYuan MSc2,3; Bendtsen, Flemming MD, DMSc1,2; Igel, Christian Dr. Habil3; Vind, Ida MD, PhD1,2; Burisch, Johan MD, PhD1,2. High Accuracy in Classifying Endoscopic Severity in Ulcerative Colitis Using Convolutional Neural Network. The American Journal of Gastroenterology: October 2022 – Volume 117 – Issue 10 – p 1648-1654 doi: 10.14309/ajg.0000000000001904