Frequency of multiple changes to prespecified primary outcomes of clinical trials completed between 2009 and 2017 in German university medical centers: A meta-research study.

Background: Clinical trial registries allow assessment of deviations of published trials from their protocol, which may indicate a considerable risk of bias. However, since entries in many registries can be updated at any time, deviations may go unnoticed. We aimed to assess the frequency of changes to primary outcomes in different historical versions of registry entries, and how often they would go unnoticed if only deviations between published trial reports and the most recent registry entry are assessed. Methods and findings: We analyzed the complete history of changes of registry entries in all 1746 randomized controlled trials completed at German university medical centers between 2009 and 2017, with published results up to 2022, that were registered in ClinicalTrials.gov or the German WHO primary registry (German Clinical Trials Register; DRKS). Data were retrieved on 24 January 2022. We assessed deviations between registry entries and publications in a random subsample of 292 trials. We determined changes of primary outcomes (1) between different versions of registry entries at key trial milestones, (2) between the latest registry entry version and the results publication, and (3) changes that occurred after trial start with no change between latest registry entry version and publication (so that assessing the full history of changes is required for detection of changes). We categorized changes as major if primary outcomes were added, dropped, changed to secondary outcomes, or secondary outcomes were turned into primary outcomes. We also assessed (4) the proportion of publications transparently reporting changes and (5) characteristics associated with changes. Of all 1746 trials, 23% (n = 393) had a primary outcome change between trial start and latest registry entry version, with 8% (n = 142) being major changes, that is, primary outcomes were added, dropped, changed to secondary outcomes, or secondary outcomes were turned into primary outcomes. Primary outcomes in publications were different from the latest registry entry version in 41% of trials (120 of the 292 sampled trials; 95% confidence interval (CI) [35%, 47%]), with major changes in 18% (54 of 292; 95% CI [14%, 23%]). Overall, 55% of trials (161 of 292; 95% CI [49%, 61%]) had primary outcome changes at any timepoint over the course of a trial, with 23% of trials (67 of 292; 95% CI [18%, 28%]) having major changes. Changes only within registry records, with no apparent discrepancy between latest registry entry version and publication, were observed in 14% of trials (41 of 292; 95% CI [10%, 19%]), with 4% (13 of 292; 95% CI [2%, 7%]) being major changes. One percent of trials with a change reported this in their publication (2 of 161 trials; 95% CI [0%, 4%]). An exploratory logistic regression analysis indicated that trials were less likely to have a discrepant registry entry if they were registered more recently (odds ratio (OR) 0.74; 95% CI [0.69, 0.80]; p<0.001), were not registered on ClinicalTrials.gov (OR 0.41; 95% CI [0.23, 0.70]; p = 0.002), or were not industry-sponsored (OR 0.29; 95% CI [0.21, 0.41]; p<0.001). Key limitations include some degree of subjectivity in the categorization of outcome changes and inclusion of a single geographic region. Conclusions: In this study, we observed that changes to primary outcomes occur in 55% of trials, with 23% trials having major changes. They are rarely transparently reported in the results publication and often not visible in the latest registry entry version. More transparency is needed, supported by deeper analysis of registry entries to make these changes more easily recognizable. Protocol registration: Open Science Framework (https://osf.io/t3qva; amendment in https://osf.io/qtd2b). In this meta-research study, Martin Holst and colleagues investigate changes to prespecified primary outcomes in the historic registries of clinical trials. Author summary: Why was this study done?: Clinical trial registries are a key tool to increase the trustworthiness of clinical trials. They allow assessment of how closely a published trial follows its original plan. However, registry entries can be updated at any time, which creates a trail of historical versions. If the latest registry entry version matches with the published trial report, important preceding changes might thus be unapparent to assessors at first glance. Our objective was to investigate how often primary outcomes are changed in the trial registry over the course of a trial, and how often outcome changes are unapparent if one compares only the latest registry entry version to the publication. What did the researchers do and find?: We assessed all 1746 randomized controlled trials completed at German university medical centers between 2009 and 2017, that have a results publication. We determined the frequency of outcome changes between different versions of a registry entry, as well as the latest registry entry and the results publication. We defined adding or dropping primary outcomes, changing them to secondary outcomes, or turning secondary outcomes into primary outcomes, as major changes. We found that approximately 55% of trials had primary outcome changes at any timepoint over the course of a trial; 23% of trials had major changes. We observed changes that can be easily identified by comparing the published results to the latest registry entry in 41% of trials. In 14% of trials, however, the changes would require an in-depth look within the historical versions of that trial's registry entry. Only 1% of trials with changes (2 trials) reported this in the corresponding publications. What do these findings mean?: Our analysis suggests that changes to primary outcomes of a clinical trial are common, are often major, and have a potential to go unnoticed. More transparency is needed, supported by deeper analysis of registry entries to reveal these outcome changes. [ABSTRACT FROM AUTHOR]