1. Repetition and reproduction of preclinical medical studies: taking a leaf from the plant sciences with consideration of generalised systematic errors
- Author
-
Clark, Jeremy S. C., Salacka, Anna, Boron, Agnieszka, van de Wetering, Thierry, Podsiadlo, Konrad, Rydzewska, Kamila, Safranow, Krzysztof, Ciechanowski, Kazimierz, Domanski, Leszek, and Ciechanowicz, Andrzej
- Subjects
Quantitative Biology - Quantitative Methods - Abstract
Reproduction of pre-clinical results has a high failure rate. The fundamental methodology including replication ("protocol") for hypothesis testing/validation to a state allowing inference, varies within medical and plant sciences with little justification. Here, five protocols are distinguished which deal differently with systematic/random errors and vary considerably in result veracity. Aim: to compare prevalence of protocols (defined in text). Medical/plant science articles from 2017/2019 were surveyed: 713 random articles assessed for eligibility for counts: first (with p-values): 1) non-replicated; 2) global; 3) triple-result protocols; second: 4) replication-error protocol; 5) meta-analyses. Inclusion criteria: human/plant/fungal studies with categorical groups. Exclusion criteria: phased clinical trials, pilot studies, cases, reviews, technology, rare subjects, -omic studies. Abbreviated PICOS question: which protocol was evident for a main result with categorically distinct group difference(s) ? Electronic sources: Journal Citation Reports 2017/2019, Google. Triplication prevalence differed dramatically between sciences (both years p<10-16; cluster-adjusted chi-squared tests): From 320 studies (80/science/year): in 2017, 53 (66%, 95% confidence interval (C.I.) 56%:77%) and in 2019, 48 (60%, C.I. 49%:71%) plant studies had triple-result or triplicated global protocols, compared with, in both years, 4 (5%, C.I. 0.19%:9.8%) medical studies. Plant sciences had a higher prevalence of protocols more likely to counter generalised systematic errors (the most likely cause of false positives) and random error than non-replicated protocols, without suffering from serious flaws found with random-Institutes protocols. It is suggested that a triple-result (organised-reproduction) protocol, with Institute consortia, is likely to solve most problems connected with the replicability crisis., Comment: 40 pages, 1 figure, 3 tables. Supplemental files at https://github.com/Abiologist/Significance.git
- Published
- 2022