Search

Summary The reproducibility crisis is a multifaceted problem involving ingrained practices within the scientific community. Fortunately, some causes are addressed by the author's adherence to rigor and reproducibility criteria, implemented via checklists at various journals. We developed an automated tool (SciScore) that evaluates research articles based on their adherence to key rigor criteria, including NIH criteria and RRIDs, at an unprecedented scale. We show that despite steady improvements, less than half of the scoring criteria, such as blinding or power analysis, are routinely addressed by authors; digging deeper, we examined the influence of specific checklists on average scores. The average score for a journal in a given year was named the Rigor and Transparency Index (RTI), a new journal quality metric. We compared the RTI with the Journal Impact Factor and found there was no correlation. The RTI can potentially serve as a proxy for methodological quality., Graphical Abstract, Highlights • Evaluated 1,578,964 articles for rigor criteria using SciScore, an automated tool • Developed a journal quality metric and studied rigor criteria reporting trends • No correlation between the Rigor and Transparency Index and the Impact Factor • The Rigor and Transparency Index may serve as a proxy for methodological quality, Biological Sciences; Bioinformatics; Biological Sciences Research Methodologies; Methodology in Biological Sciences