The use of statistics in medical research: a comparison of The New England Journal of Medicine and Nature Medicine

There is widespread evidence of the extensive use of statistical methods in medical research. Just the same, standards are generally low and a growing body of literature points to statistical errors in most medical journals. However, there is no comprehensive study contrasting the top medical journals of basic and clinical science for recent practice in their use of statistics. All original research articles in Volume 10, Numbers 1-6 of Nature Medicine (Nat Med) and Volume 350, Numbers 1-26 of The New England Journal of Medicine (NEJM) were screened for their statistical content. Types, frequencies, and complexity of applied statistical methods were systematically recorded. A 46-item checklist was used to evaluate statistical quality for a subgroup of papers. 94.5 percent (95% CI 87.6-98.2) of NEJM articles and 82.4 percent (95% CI 65.5-93.2) of Nat Med articles contained inferential statistics. NEJM papers were significantly more likely to use advanced statistical methods (p < 0.0001). Statistical errors were identified in a considerable proportion of articles, although not always serious in nature. Documentation of applied statistical methods was generally poor and insufficient, particularly in Nat Med. Compared to 1983, a vast increase in usage and complexity of statistical methods could be observed for NEJM papers. This does not necessarily hold true for Nat Med papers, as the results of the study indicate that basic science sticks with basic analysis. As statistical errors seem to remain common in medical literature, closer attention to statistical methodology should be seriously considered to raise standards. KEY WORDS: Complexity; Errors and shortcomings; Statistical methods in medical journals; Techniques.