General Information
Conflict of Interest
Request Clinical Trials
Associated Trial(s):- NCT00091715 - A Randomized, Double-blind, Placebo-controlled, Multicenter Study to Assess the Efficacy, Safety, and Tolerability of Bosentan in Patients With Mildly Symptomatic Pulmonary Arterial Hypertension (PAH)
- NCT01323192 - A Double-blind, Placebo-controlled, Parallel-Group Study to Evaluate the Efficacy and Safety of JNS001 in Adults With Attention-Deficit/Hyperactivity Disorder at Doses of 18 mg, 36 mg, 54 mg, or 72 mg Per Day
- NCT00236665 - A Multicenter, Randomized, Double-Blind, Placebo-Controlled, Parallel Group Study of the Efficacy and Safety of Topiramate in the Treatment of Obese Patients With Mild to Moderate Essential Hypertension
Request Clinical Trials
Data Request Status
Status: OngoingResearch Proposal
Project Title: On the use of baseline covariates in test retest designs
Scientific Abstract:
Background: Many current clinical trials compare baseline and endline measures in their primary endpoints. When doing so, the baseline is always available as a covariate for covariate adjustment, and adjusting can decrease treatment effect estimator variance. Despite this, few studies with this design actually adjust. The statistical efficiency losses from failing to do so are substantial.
Objective: I aim to quantify, using three example studies, how large the variance losses from failing to adjust are. For each study, I will compare estimators with and without covariate adjustment. I will also look at how variances vary by sample size.
Study Design: I will re-analyze data from three clinical trials with endline-minus-baseline outcomes. I compare a raw difference in means comparison with an adjusted analogue, and measure how much lower variance the adjusted estimates are.
Participants: all original participants in the original trials will be used in the analysis.
Primary and Secondary Outcome Measures: The primary outcome will as closely as possible follow what was done in the original studies.
Statistical Analysis: A raw mean comparison of baseline/endline differences and an ANCOVA estimate of the same difference, using the baseline as a covariate.
Brief Project Background and Statement of Project Significance:
This project is motivated by the observation that 1) statistical theory is unequivocal about the power benefits from including covariates in adjustment when analyzing data from randomized trials, while 2) practitioners often fail to do so. In test-retest designs, the fact that practice seems to lag behind theory is especially puzzling: the baseline is almost by construction typically a good predictor variable to use. In this study, I aim to provide a back-of-the-envelope quantification of how substantial are the economic losses from not making the adjustments that theory recommends. I will do this by comparing standard errors across statistical analyses that do and do not adjust for the baseline. Specifically, I build off of prior work [1] which provides statistical theory showing the optimal way to adjust for a baseline covariate and how to conduct valid standard errors when making the adjustment.
[1] shows how to construct covariate-adjusted treatment effect estimators that are asymptotically normal, unbiased, and have correct standard errors. Building off of these insights, the economic losses for not making adjustments can be quantified through the following thought experiment: imagine that adjusting for the baseline reduces variance of the resulting estimator by (100 * X)%. In this case, a clinical trial that recruited N participants, but did not adjust would have the same standard errors as a trial that recruited N * (1-X) participants but did make the adjustment. Thus, the researcher who did not make the adjustment could have saved on the costs by recruiting X*N fewer patients, without sacrificing any statistical power.
Summing up over all trials, the savings from making adjustments for baselines when possible in the statistical analysis, relative to the status quo, is given by
[Total Spending on Clinical Trials ] * [fraction of studies that study baseline-endline differences] * [fraction of those studies that do not adjust] * X
The Congressional Budget Office in 2021 [2] estimates about $43B/year on phase 3 clinical trials, providing [Total Spending on Clinical Trials ]. My own data collection (which are unpublished, because they will be in the paper I plan to publish in part using the analysis from this request) from clinicaltrials.gov provide estimates [fraction of studies that study baseline-endline differences] = 0.2, and [fraction of those studies that do not adjust] = 0.8. Thus, the quantities in the brackets above are all obtained from other sources. All that remains is X, which can in turn be estimated as ((adjusted standard error) / (unadjusted standard error))^2, which is exactly what I plan to compute with the requested data.
Specific Aims of the Project: In this project, I will replicate treatment effect estimators in three studies, without and without adjusting for the baseline measure as a covariate. I will quantify how large the variance of the resulting treatment effect estimator is, depending on if the baseline is used or not.
Study Design: Methodological research
What is the purpose of the analysis being proposed? Please select all that apply.: Develop or refine statistical methods Research on clinical trial methods
Software Used: R, RStudio
Data Source and Inclusion/Exclusion Criteria to be used to define the patient sample for your study:
There are no exclusion criterion. In other words, no demographic information will be used to decide who does or does not end up in the sample. All individuals in the original dataset who were assigned treatment will be included for analysis.
The three trials I have chosen for this project were selected because they represent three examples of the test-retest structure, where the primary endpoint is a change in a measure. They were chosen to also correspond to different underlying conditions, to demonstrate the the basic statistical insight applies across contexts.
Primary and Secondary Outcome Measure(s) and how they will be categorized/defined for your study:
I will use the same primary endpoints as in the original publications. There are three studies, so I will describe each in detail sequentially.
The first trial is NCT00091715: Efficacy and Safety of Oral Bosentan in Pulmonary Arterial Hypertension Class II. In the publication corresponding to this trial, the main outcomes analyzed were pulmonary vascular resistance = (mean pulmonary arterial pressure - pulmonary capilary wedge pressure) * 80 / (cardiac output), as well as exercise capacity, as measured by 6-minute walking distance.
The second trial is NCT01323192: An Efficacy and Safety Study for JNS001 in Adults With Attention-Deficit Hyperactivity Disorder. The primary outcome reported in clincaltrials.gov is Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) Total Attention Deficit-Hyperactivity Disorder (ADHD) Symptoms Scores of Conners' Adult ADHD Rating Scale - Observer Screening Version (CAARS-O: SV), so I will take this as the primary outcome in my study as well.
The third trial is NCT00236665: A Study of Efficacy and Safety of Topiramate in the Treatment of Obese Patients With Mild to Moderate Essential Hypertension. The primary outcome measures reported in clinicaltrials.gov for this trial are body weight and sitting diastolic blood pressure, which once again, I will also take as my primary outcome.
Main Predictor/Independent Variable and how it will be categorized/defined for your study: The main predictor variable in this study is simply treatment assignment in the respective studies. In study 1, this is an indicator for whether the drug bosentan was taken (instead of a placebo in control). In study 2, this is an indicator for whether the drug JNS001 was taken. Finally, in study 3, the predictor variable is whether or not the drug topiramate was taken.
Other Variables of Interest that will be used in your analysis and how they will be categorized/defined for your study:
In each of the studies I plan on studying, the primary endpoint of the original study is a mean difference of the relevant measure(s), comparing baseline and endline.
Take study 2 as an example, which is easiest to describe, since it has a single outcome measure, a score on the diagnostic test: CAARS-O: SV. This score is measured twice, once at the start of the trial (baseline) and once at the end (endline). The primary _outcome_ is the difference, (endline - baseline). In addition to this difference, I will also separately use the baseline variable itself. The purpose here is to use the baseline in a covariate adjustment analysis to show how much doing so reduces variance.
In studies 1 and 3, I will do the analogous analysis, but with their respective outcome variables, as defined earlier.
Statistical Analysis Plan:
I will run two tests.
One is a simple two-sample t-test, comparing treatment and control. I will compute means of the outcome in treatment and control, compute standard errors of each respective mean, and compute the difference in means, as well as the standard error in this difference.
A second test is based on an analysis of covariance. The procedure can be described as follows. I will first run two OLS regressions, one using only treated observations and one using only control observations. The regression will be of the change, which is the primary endpoint, regressed on the baseline. Let beta_t, beta_c be the slope from these regressions. I will then perform a 2-sample t-test on the constructed quantity [change - (beta_t+beta_c)/2 * baseline], as I did with the raw outcomes. In the accompanying paper I am trying to write, I show 1) that this is also a valid estimate of the average treatment effect, 2) the standard errors from doing the 2-sample t-test are asymptotically correct, and 3) the variance of this procedure is lower. I aim to quantify how much lower.
Narrative Summary: Many clinical trials have a primary endpoint which can be described as the difference between an outcome measured at end-line compared to baseline. In all such settings, treatment effect estimators can have lower variances, provided that the baseline is adjusted for in a covariate adjustment. Using three example trials, I plan to demonstrate the degree of variance reduction typically available by doing so.
Project Timeline:
Start date - November 2025
Analysis completion date - January 2026
Manuscript draft - January 2026
Submitted for publication - February 2027
Dissemination Plan: I plan to submit the resulting analysis as part of a paper for the American Journal of Epidemiology.
Bibliography:
[1] John A List, Ian Muir, and Gregory Sun. Using machine learning for efficient flexible regression adjustment in economic experiments. Econometric Reviews, 44(1):2--40, 2024.
[2] Congressional Budget Office. Research and development in the pharmaceutical industry. https://www.cbo.gov/publication/57126#footnote-012, 2021