1. Dataset decay and the problem of sequential analyses on open datasets
- Author
-
William Hedley Thompson, Jessey Wright, Patrick G Bissett, and Russell A Poldrack
- Subjects
open data ,sequential testing ,multiple comparisons ,multiple comparison correction ,meta-research ,Medicine ,Science ,Biology (General) ,QH301-705.5 - Abstract
Open data allows researchers to explore pre-existing datasets in new ways. However, if many researchers reuse the same dataset, multiple statistical testing may increase false positives. Here we demonstrate that sequential hypothesis testing on the same dataset by multiple researchers can inflate error rates. We go on to discuss a number of correction procedures that can reduce the number of false positives, and the challenges associated with these correction procedures.
- Published
- 2020
- Full Text
- View/download PDF