Search

Peer reviewed: True, The distribution of effect sizes may offer insights about the research done and reported in a scientific field. We have evaluated 12 412 manually collected correlation effect sizes (Sample 1) and 31 157 computer-extracted correlation effect sizes (Sample 2) published in journals focused on social or developmental psychology. Sample 1 consisted of 243 studies from six journals published in 2010 and 2019. Sample 2 consisted of 5012 papers published in 10 journals between 2010 and 2019. The 25th, 50th and 75th effect size percentiles were 0.08, 0.17 and 0.33, and 0.17, 0.31 and 0.52 in Samples 1 and 2, respectively. Sample 2 percentiles were probably larger because Sample 2 only included effect sizes from the text but not from tables. In text authors may have emphasized larger correlations. Large sample sizes were associated with smaller reported correlations. In Sample 1 about 70% of studies specified a directional hypothesis. In 2010 no papers had power calculations, while in 2019 14% of papers had power calculations. These data offer empirical insights into the distribution of reported correlations and may inform the interpretation of effect sizes. They also demonstrate the importance of computation of statistical power and highlight potential reporting bias.

Primary (year 4) and secondary (year 7 and 8) UK school students' scores in math and reading standardised assessments, as well as scores in 3 anxiety forms (math anxiety, test anxiety and general anxiety). This project was funded by the Nuffield Foundation.

Supplemental material, sj-docx-1-amp-10.1177_25152459211040837 for Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies From Psychology by Tom E. Hardwicke, Dénes Szűcs, Robert T. Thibault, Sophia Crüwell, Olmo R. van den Akker, Michèle B. Nuijten and John P. A. Ioannidis in Advances in Methods and Practices in Psychological Science

The attentional Spatial-Numerical Association of Response Codes (att-SNARC) effect (Fischer et al., 2003; Nature Neuroscience)—the finding that participants are quicker to detect left-side targets when the targets are preceded by small numbers and quicker to detect right-side targets when they are preceded by large numbers—has been used as evidence for embodied number representations and to allow for strong claims about the link between number and space (e.g., a mental number line). We 41 attempted to replicate Study 2 of Fischer et al. (2003) by collecting data from 1105 participants across seventeen labs. Across all 1105 participants and four ISI conditions, the proportion of times the direction of the observed effect was consistent with the original effect was 0.50. Further, the effects we observed both within and across labs were minuscule and incompatible with those observed in Fischer et al. (2003). Given this, we conclude that we have failed to replicate the effect reported by Fischer et al. (2003). In addition, our analysis of several participant-level moderators (finger counting preferences, reading/writing direction experience, handedness, and mathematics fluency and mathematics anxiety) revealed no substantial moderating effects. Our results demonstrate that the att-SNARC effect cannot be used as evidence to support the strong claims about the link between number and space discussed above.

We determined the relative importance of the so-called approximate number system (ANS), symbolic number comparison (SNC) and verbal and spatial short-term and working memory (WM) capacity for mathematics achievement in 1,254 Grade 2, 4 and 6 children. The large sample size assured high power and low false report probability and allowed us to determine effect sizes precisely. We used reading decoding as a control outcome measure to test whether findings were specific to mathematics. Bayesian analysis allowed us to provide support for both null and alternative hypotheses. We found very weak zero-order correlations between ANS measures and math achievement. These correlations were not specific to mathematics, became non-significant once intelligence was considered and ANS measures were not selected as predictors of math by regression models. In contrast, overall SNC accuracy and spatial WM measures were reliable and mostly specific predictors of math achievement. Verbal short-term and WM and SNC reaction time were predictors of both reading and math achievement. We conclude that ANS tasks are not suitable as measures of math development in school-age populations. In contrast, all other cognitive functions we studied are promising markers of mathematics development.

Supplemental material, CollingOpenPracticesDisclosure for Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003) by Lincoln J. Colling, Dénes Szűcs, Damiano De Marco, Krzysztof Cipora, Rolf Ulrich, Hans-Christoph Nuerk, Mojtaba Soltanlou, Donna Bryce, Sau-Chin Chen, Philipp Alexander Schroeder, Dion T. Henare, Christine K. Chrystall, Paul M. Corballis, Daniel Ansari, Celia Goffin, H. Moriah Sokolowski, Peter J. B. Hancock, Ailsa E. Millen, Stephen R. H. Langton, Kevin J. Holmes, Mark S. Saviano, Tia A. Tummino, Oliver Lindemann, Rolf A. Zwaan, Jiří Lukavský, Adéla Becková, Marek A. Vranka, Simone Cutini, Irene Cristina Mammarella, Claudio Mulatti, Raoul Bell, Axel Buchner, Laura Mieth, Jan Philipp Röer, Elise Klein, Stefan Huber, Korbinian Moeller, Brenda Ocampo, Juan Lupiáñez, Javier Ortiz-Tudela, Juanma de la Fuente, Julio Santiago, Marc Ouellet, Edward M. Hubbard, Elizabeth Y. Toomarian, Remo Job, Barbara Treccani and Blakeley B. McShane in Advances in Methods and Practices in Psychological Science