Effect Sizes Hypothesized and Observed in Contemporary Phase III Trials of Targeted and Immunological Therapies for Advanced Cancer

Background We sought to compare the effect sizes hypothesized in the trial design, observed in the trial results, and considered clinically meaningful by the American Society of Clinical Oncology (ASCO) 2014 recommendations, in phase III trials of targeted and immunological therapies. Methods We studied phase III, superiority trials of targeted and immunological therapies in advanced cancers published from 2005 to 2015. We recorded the characteristics, design parameters, and observed results for the primary endpoint of each trial. The effect sizes hypothesized in the trial design were compared with the ASCO 2014 recommendation that phase III trials be designed to detect overall survival (OS) benefits that are clinically meaningful (hazard ratio ≤0.8). Results All critical elements of the trial design (effect sizes hypothesized, estimated survival in the control group, power, and significance level) were identified in 165 of 213 included trials (77%). Of trials with a statistically significant result for the primary endpoint, 16 of 30 (53%) with a primary endpoint of OS and 20 of 53 (38%) with a primary endpoint of progression free survival (PFS) had an observed effect size less extreme than hypothesized; and 7 of 30 trials (23%) reported an observed effect size for OS that was statistically significant but not clinically meaningful (HR > 0.80) according to the ASCO 2014 recommendations. Conclusion Many trials were designed such that an observed benefit in OS or PFS that was not clinically meaningful would be statistically significant. Phase III trials should be designed to provide results that are statistically significant for observed effects that are clinically meaningful but not for observed results that are of dubious clinical importance.