1. What’s in a Badge? A Computational Reproducibility Investigation of the Open Data Badge Policy in One Issue of Psychological Science
- Author
-
Sophia Crüwell, Deborah Apthorp, Bradley J. Baker, Lincoln Colling, Malte Elson, Sandra J. Geiger, Sebastian Lobentanzer, Jean Monéger, Alex Patterson, D. Samuel Schwarzkopf, Mirela Zaneva, Nicholas J. L. Brown, Crüwell, Sophia [0000-0003-4178-5820], Apthorp, Deborah [0000-0001-5785-024X], Baker, Bradley J [0000-0002-1697-4198], Colling, Lincoln [0000-0002-3572-7758], Elson, Malte [0000-0001-7806-9583], Geiger, Sandra J [0000-0002-3262-5609], Monéger, Jean [0000-0003-1178-1896], Patterson, Alex [0000-0002-7780-4192], Zaneva, Mirela [0000-0003-3569-931X], Brown, Nicholas JL [0000-0003-1579-0730], and Apollo - University of Cambridge Repository
- Subjects
Policy ,journal policy ,data sharing ,open badges ,open data ,Humans ,Reproducibility of Results ,Disclosure ,150 Psychology ,reproducibility ,General Psychology - Abstract
[Preprint; Manuscript accepted at Psychological Science] In April 2019, Psychological Science published its first issue in which all research articles received the Open Data badge. We used that issue to investigate the effectiveness of this badge, focusing on the adherence to its aim at Psychological Science: sharing both data and code to ensure reproducibility of results. Twelve researchers of varying experience levels attempted to reproduce the results of the empirical articles in the target issue (at least three researchers per article). We found that while all 14 articles provided at least some data and six provided analysis code, only one article was rated to be exactly reproducible, and three essentially reproducible with minor deviations. We suggest that researchers should be encouraged to adhere to the higher standard in force at Psychological Science. Moreover, a check of reproducibility during peer review may be preferable to the ‘disclosure method’ of awarding badges.
- Published
- 2023