Search

Results that are statistically significant are more likely to be reported by authors and more likely to be accepted by journals. These common biases warp the published evidence and undermine the ability of research to improve health by giving an incomplete body of evidence. We aimed to show the effect of the bias towards statistical significance on the evidence-base using published confidence intervals. We examined over 968,000 ratios and their confidence intervals in the abstracts of health and medical journals from Medline between 1976 and January 2019. We plotted the distributions of lower and upper confidence interval limits to visually show the strong bias for statistically significant results. There was a striking change in the distributions around 1, which is the statistically significant threshold for ratios. There was an excess of lower intervals just above 1, corresponding to a statistically significant increase in risk. There was a similar excess of upper intervals just below 1, corresponding to a statistically significant decrease in risk. These biases have not improved in recent years. The huge excesses of confidence intervals that are just above and below the statistically significant threshold are not statistically plausible. Large changes in research practice are needed to provide more results that better reflect the truth.