1. Automated screening of COVID-19 preprints: can we help authors to improve transparency and reproducibility?
- Author
-
Halil Kilicoglu, Sarah K. McCann, Peter Grabitz, Alexandra Bannach-Brown, Robert Schulz, Cyril Labbé, Amanda Capes-Davis, Gerben ter Riet, Tracey L. Weissgerber, Shyam M. Saladi, Nico Riedel, René Bernard, Peter Eckmann, Guillaume Cabanac, Bertrand Favier, Jennifer A. Byrne, Anita Bandrowski, Cardiology, ACS - Diabetes & metabolism, APH - Aging & Later Life, APH - Personalized Medicine, Berlin Institute of Health (BIH), Charité - UniversitätsMedizin = Charité - University Hospital [Berlin], University of Illinois at Urbana-Champaign [Urbana], University of Illinois System, Systèmes d’Information - inGénierie et Modélisation Adaptables (SIGMA ), Laboratoire d'Informatique de Grenoble (LIG), Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes (UGA)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP ), Université Grenoble Alpes (UGA)-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes (UGA)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP ), Université Grenoble Alpes (UGA), University of California [San Diego] (UC San Diego), University of California, University of Amsterdam [Amsterdam] (UvA), The University of Sydney, Recherche d’Information et Synthèse d’Information (IRIT-IRIS), Institut de recherche en informatique de Toulouse (IRIT), Université Toulouse 1 Capitole (UT1), Université Fédérale Toulouse Midi-Pyrénées-Université Fédérale Toulouse Midi-Pyrénées-Université Toulouse - Jean Jaurès (UT2J)-Université Toulouse III - Paul Sabatier (UT3), Université Fédérale Toulouse Midi-Pyrénées-Centre National de la Recherche Scientifique (CNRS)-Institut National Polytechnique (Toulouse) (Toulouse INP), Université Fédérale Toulouse Midi-Pyrénées-Université Toulouse 1 Capitole (UT1), Université Fédérale Toulouse Midi-Pyrénées, Université Toulouse III - Paul Sabatier (UT3), CellBank Australia, Groupe de Recherche et d’Étude du Processus Inflammatoire (TIMC-GREPI), Translational Innovation in Medicine and Complexity / Recherche Translationnelle et Innovation en Médecine et Complexité - UMR 5525 (TIMC ), VetAgro Sup - Institut national d'enseignement supérieur et de recherche en alimentation, santé animale, sciences agronomiques et de l'environnement (VAS)-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes (UGA)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP ), Université Grenoble Alpes (UGA)-VetAgro Sup - Institut national d'enseignement supérieur et de recherche en alimentation, santé animale, sciences agronomiques et de l'environnement (VAS)-Centre National de la Recherche Scientifique (CNRS)-Université Grenoble Alpes (UGA)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP ), California Institute of Technology (CALTECH), Humboldt-Universität zu Berlin, Faculteit Gezondheid, Urban Vitality, Amsterdam University of Applied Sciences, and Cabanac, Guillaume
- Subjects
2019-20 coronavirus outbreak ,Coronavirus disease 2019 (COVID-19) ,Computer science ,Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) ,030204 cardiovascular system & hematology ,Article ,General Biochemistry, Genetics and Molecular Biology ,03 medical and health sciences ,Automation ,Medical research ,0302 clinical medicine ,[SDV.MHEP.MI]Life Sciences [q-bio]/Human health and pathology/Infectious diseases ,Humans ,030212 general & internal medicine ,ComputingMilieux_MISCELLANEOUS ,[INFO.INFO-BI] Computer Science [cs]/Bioinformatics [q-bio.QM] ,Research data ,Publishing ,Reproducibility ,Information retrieval ,COVID-19 ,General Medicine ,Bioethics ,Transparency (behavior) ,Databases, Bibliographic ,3. Good health ,[INFO.INFO-TT]Computer Science [cs]/Document and Text Processing ,Preprints as Topic ,[INFO.INFO-IR]Computer Science [cs]/Information Retrieval [cs.IR] ,[INFO.INFO-IR] Computer Science [cs]/Information Retrieval [cs.IR] ,[INFO.INFO-BI]Computer Science [cs]/Bioinformatics [q-bio.QM] ,[SDV.MHEP]Life Sciences [q-bio]/Human health and pathology - Abstract
International audience; The COVID-19 pandemic has thrust preprints into the spotlight, highlighting their advantages and disadvantages. The lack of peer review allows publication to occur with unprecedented speed, but this has raised concerns among biomedical scientists about the quality of the reported research. The study had three primary objectives in regard to COVID-19 preprints: 1. Test the feasibility of automatically and publicly evaluating preprints on a large scale; 2. Assess the prevalence of common quality problems in preprints; and 3. Compare the quality of preprints to published papers. While a substantial fraction (36.2%) of preprints addressed study limitations, the proportion that met other quality criteria was much lower, with only 20% addressing sex as a variable, ~14% sharing open code or data, 7.6% including non-colorblind safe images, and 7.3% showing misleading bar graphs. Both authors and non-authors interacted with the automated Tweets containing reports for preprints. This project shows that it is feasible to conduct large-scale automated screening of preprints for common quality criteria and provide feedback to study authors and readers before publication. These reports can publicly raise awareness about factors that affect study quality and reproducibility, while helping authors to present their research in a more transparent and reproducible manner.
- Published
- 2021
- Full Text
- View/download PDF