Misleading medical literature: An observational study

OBJECTIVE Language that implies a conclusion not supported by the evidence is common in the medical literature. The hypothesis of the present study was that medical journal publications are more likely to use misleading language for the interpretation of a demonstrated null (i.e. chance or not statistically significant) effect than a demonstrated real (i.e. statistically significant) effect. METHODS This was an observational study of the medical literature with a systematic sampling method. Articles published in The Journal of the American Medical Association, The Lancet and The New England Journal of Medicine over the last two decades were eligible. The language used around the P-value was assessed for misleadingness (i.e. either suggesting an effect existed when a real effect did not exist or vice versa). RESULTS There were 228 unique manuscripts examined, containing 400 statements interpreting a P-value proximate to 0.05. The P-value was between 0.036 and 0.050 for 303 (75.8%) statements and between 0.050 and 0.064 for 97 (24.3%) statements. Forty-four (11%) of the statements were misleading. There were 40 (41.2%) false-positive sentences, implying statistical significance when the P-value was >0.05, and four (1.3%) false-negative sentences, implying no statistical significance when the P-value