Research Proposal
Background: Finding effective treatment methods for mental disorders is crucial to enhance quality of life. Precision Medicine, a method with momentous potential, involves developing personalized plans for each patient for effective treatments. Additionally, many disorders share common symptoms, making it necessary to explore symptom clusters both within and across diagnosis groups.
Objective: To build a Machine Learning (ML) model that can predict the effectiveness of common treatments across mental disorders using individual patient data; to categorize patients into symptom-clustering groups and discover similarities between disorders.
Study Design: We will train our model on clinical data to predict treatment efficacy for each drug and cross-validate it to assess how the model generalizes to an independent dataset. Using another model, we will analyze datasets across conditions to identify symptom-clustering groups.
Participants: Patients with psychotic, mood, anxiety, or disruptive behaviour disorders, epilepsy, autistic disorder, or ADHD.
Main Outcome Measure: For our predictive model, the outcome measure will be the sensitivity, specificity and predictive power relative to each specific drug; in our symptom-clustering analysis, we will test whether clusters are stable, and how treatment responsiveness differs among clusters.
Statistical Analysis: The relationship between clinical data and treatment outcomes will be examined using ML regression analysis. Clusters of similar symptom profiles will be identified with pattern recognition techniques.
Mental disorders significantly harm a person's quality of life because they can lead to detrimental physiological and psychological states. Therefore, finding efficient treatments is paramount to improving mental health. An approach with significant potential is Precision Medicine (Lu, Fizbein & Opfer, 1987), which involves creating individualized medical plans for each patient with pertinent drugs identified at an early stage. Currently, identifying the most appropriate treatment for a patient often involves a costly process of trial and error (costly in terms of time, money and health); individualized treatment plans offer a potential solution that would greatly improve this process. Recent research, such as the study by Chekroud et al (2016), shows that statistical models constructed from clinical data can enable the prediction of a patient’s drug response. In this project, we will use clinical data to build a Machine Learning model to predict the effectiveness (e.g. improvement of symptoms as measured with clinical assessments, time until relapse or incidence of adverse events) of various treatments for individual patients, which could provide a promising method for future personalized treatment plans.
Many mental disorders share similar symptoms. For example, Major Depressive Disorder and Generalized Anxiety Disorder both involve restlessness and a lack of concentration (Zbozinek et al, 2012). It is important to analyze symptom clusters that may have previously been thought of as belonging to distinct disorders in order to develop wide-ranging treatments. A paper by Chekroud et al (2017), demonstrated that the researchers were able to cluster empirically defined symptoms into groups with different responsiveness to treatments, both within and across antidepressant medications. In our project, we will create a ML model to categorize patients into symptom-clustering groups and both within and across disorders. Our investigation into similarities between disorders will contribute to discovering potential connections between mental illnesses.
This project will materially enhance our scientific knowledge of treatment efficacy and different outcomes across the treatment and placebo groups for specific medications (such as Risperidone, Paliperidone, or Topiramate). It will also provide more information about symptom clusters across the disorders that these medications are prescribed for. In terms of this project's relevance to public health, it will contribute to the growing efficiency, personalization, and precision of treatment applications and the potential use of treatments for multiple disorders. We recognize that this project is ambitious, nevertheless, collecting data about several mental disorders can allow us to evaluate the use of ML techniques for finding novel clusters, which could potentially alter their categorization in the future. Therefore, this project acts as a proof-of-concept pilot study.
The first aim of this project is to construct least absolute shrinkage and selection operator (LASSO) regression models from individual patient clinical data. These models will provide identification of patients who will or will not respond positively based on clinical measures (such as YMRS in Bipolar Disorder (Young et al, 1978) and PANSS in Schizophrenia (Kay, Fizbein & Opfer, 1987)) to specific treatments for multiple mental disorders as compared to placebo.
The second aim of this project is to build a k-nearest neighbour (KNN) ML model that clusters symptoms across all mental disorders which will be visualised using t-Distributed Stochastic Neighbour Embedding (t-SNE) in order to identify possible novel associations across clinical categories.
We hypothesize that our models will reliably predict treatment outcomes and that in investigating symptom-clustering groups we will discover novel similarities across clinical classifications.
Data Source: All available digital data (clinical, biochemical, cognitive, sociodemographic, etc.) from phase 3 and 4 randomized and/or open-label datasets for Schizophrenia, Schizoaffective Disorder, Psychosis, Bipolar Disorder, Major Depressive Disorder, Attention Deficit Hyperactivity Disorder, Disruptive Behaviour Disorders, Anxiety Disorders, Autistic Spectrum Disorder, and Epilepsy (as a reference).
Exclusion Criteria: In order to develop an ML model with strong predictive power, the model must be trained on a large dataset; the more variance captured in the clinical data, the more accurate our models will be at predicting the treatment effectiveness of unseen patients. We will, for this reason, include all patients in our study.
We will measure the sensitivity, specificity and predictive power of our ML models in predicting treatment effectiveness, relative to each specific drug through a cross-validation method (described below in the Statistical Analysis section). Our measure of treatment effectiveness will depend on the primary endpoints of the trials, with examples including:
• Time until remission (days)
• Change from baseline (using a clinically relevant measure such as PANSS (Kay, Fizbein & Opfer, 1987), or YMRS (Young et al, 1978))
• Survival (yes/no)
• High vs. low quality of life scores (such as the Short Form-36 (Ware & Sherbourne, 1992) or WHOQOL-BREF (Skevington, Lofty, & O’Connel, 2004))
For each of our ML models, our main independent variable will be treatment allocation and will be defined as a binary dummy variable.
Our predictor variables of interest are all digitally archived information, which includes patient profile characteristics such as variables of the demographic, clinical, cognitive, genetic, lab-test, and free-text survey information, as well as the characteristics of the trials, such as when and where the trial was performed, the number of subjects and the intervention used. The more moderating variables available in our data, the better our model will be able to make personalized predictions. The following list just provides a few potential example predictor variables that could be included in our analysis:
• Age (years)
• Sex (male/female/intersex)
• Race (Caucasian, African American, etc.)
• BMI (continuous)
• Smoker (yes/no)
• Time since diagnosis (years)
• Previous treatments (yes/no, name, dose)
• Additional Diseases / Comorbidities (yes/no, name)
• Measures of psychopathology (for example, YMRS)
• Relapse occurrence/time to relapse (yes/no, days)
All digitally archived variables.
• A descriptive analysis of all demographic, clinical, and pharmaceutical characteristics of participants. ANOVA and chi-square tests will be conducted to determine whether the distribution of continuous and categorical factors, respectively, are distributed equally among patients. Results will be displayed as the median and interquartile range (IQR) for continuous variables and as number and percentage frequency for categorical variables.
Aim One:
• All ML models will be developed and appraised with k-fold cross-validation (Pedregosa et al, 2011), partitioning the entirety of the relevant constructed dataset into k disjoint subsets, with the model trained on k-1 of the subsets and the model’s predictive power tested on the remaining subset.
• The least absolute shrinkage and selection operator (LASSO) regression analysis method will be performed to determine the relationship between patient profile and treatment outcome. LASSO regression can obtain the subset of predictors that minimizes prediction error for a quantitative response variable, which, for our project, will be the measure of the treatment outcome relative to each disorder (e.g. time until remission) (Santosa & Symes, 1986).
Aim Two:
• A descriptive analysis of all demographic, clinical, and pharmaceutical characteristics of participants will be undertaken, with ANOVA and chi-square tests conducted to determine whether the distribution of continuous and categorical factors, respectively, are distributed equally among patients. Results will be displayed as the median and interquartile range (IQR) for continuous variables and as number and percentage frequency for categorical variables.
• k-nearest neighbours algorithm (KNN) will be used to cluster patient profiles across all conditions of interest to discover previously unidentified symptom-clustering across mental disorders. KNN is a non-parametric pattern recognition method that can assign each patient profile to a particular cluster or group based on similarities across all patients and mental disorders (Altman, 1992).
• t-Distributed Stochastic Neighbour Embedding (t-SNE), an ML algorithm for dimensionality reduction, will be used to visualize the symptom-clustering groups in a low-dimensional space (van der Maaten & Hinton, 2008).
We will use Machine Learning (ML)to predict the effectiveness of common drugs used in the treatment of several mental disorders such as Schizophrenia and Major Depressive Disorder. Additionally, we will examine similarities between patient profiles to identify symptom clusters across disorders. This project has the potential to materially enhance our scientific knowledge of treatment accuracy and the categorization of mental disorders. Regarding public health, prospective identification of treatment accuracy and applicability can lead to optimal (precise, efficient, and individually-tailored) treatment plans for each patient, enhancing wellbeing.
Project start date: August 2019
Initial Analysis completion date: May 2020
Manuscript Drafted: June 2020
Manuscript submitted for publication: July 2020
Report back to YODA: August 2020
We anticipate the generation of at least two manuscripts from this project on our models' ability to predict treatment outcomes and recluster patient profiles. The target audience would be physicians as well as psychiatry and pharmacology researchers. Potentially suitable journals for these manuscripts include Neuropsychopharmacology, Journal of Psychiatric Research, The Canadian Journal of Psychiatry, JAMA Psychiatry, Journal of Machine Learning Research and Artificial Intelligence in Medicine.
Altman, N. (1992). An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician. 46(3), pp. 175 – 185. doi:10.1080/00031305.1992.10475879. hdl:1813/31637
Chekroud, A., Gueorguieva, R., Krumholz, H., Trivedi, M., Krystal, J., & McCarthy, G. (2017). Reevaluating the efficacy and predictability of antidepressant treatments: A system Clustering Approach. JAMA Psychiatry 74(4), pp. 370-378. doi:10.1001/jamapsychiatry.2017.0025
Chekroud, A., Zotti, R., Shehzad, Z., Gueorguieva, R., Johnson, M., Trivedi, M., . . . Corlett, P. (2016). Cross-trial prediction of treatment outcome in depression: a machine learning approach. Lancet Psychiatry 3, pp. 243-250. doi: https://doi.org/10.1016/S2215-0366(15)00471-X
Kay, S., Fizbein, A., & Opfer, L. (1987). The Positive and Negative Syndrome Scale (PANSS) for Schizophrenia. Schizophrenia Bulletin, 13(2), 261-276. Retrieved from http://dx.doi.org/10.1093/schbul/13.2.261
Lu, Y., Goldstein, D., Angrist, M., & Cavalleri, G. (2014). Personalized Medicine and Human Genetic Diversity. Cold Spring Harbor Perspectives in Medicine, 4(9). doi:10.1101/cshperspect.a008581
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, O., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. JMLR, 12, pp. 2825-2830. Retrieved from http://jmlr.csail.mit.edu/papers/v12/pedregosa11a.html
Santosa, F., & Symes, W. (1986). Linear inversion of band-limited reflection seismograms. SIAM Journal on Scientific and Statistical Computing. 7(4), pp. 1307–1330. doi:10.1137/0907087
Skevington, S., Lofty, M., & O’Connell, K. (2004). The World Health Organization’s WHOQOL-BREF quality of life assessment: Psychometric properties and results of the international field trial. Quality of Life Research, 13, pp. 299-310. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.463.2578&rep=re...
van der Maaten, L., & Hinton, G. (2008). Visualizing Data Using t-SNE. Journal of Machine Learning Research. 9, pp. 2579–2605. Retrieved from http://jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf
Ware, J., & Sherbourne, C. (1992). The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Medical Care, 30(6), pp. 473-483. Retrieved from https://pdfs.semanticscholar.org/06cb/0076e310136d0ca8b56cc8585ec2bf43e0...
Young, R., Biggs, J., Ziegler, V., & Meyer, D. (1978). A rating scale for mania: reliability, validity and sensitivity. British Journal of Psychiatry. 133(5), pp. 429-35. doi:10.1192/bjp.133.5.429
Zbozinek, T., Rose, R., Wolitzky-Taylor, K., Sherbourne, C., Sullivan, G., Stein, M., Roy-Burne, P., & Craske, M. (2012). Diagnostic overlap of generalized anxiety disorder in a primary care sample. Depression & Anxiety, 29(12), 1065-1071. doi:1002/da.22026