Author: "DREBER, ANNA" - Searchworks@Jio Institute Digital Library Search Results

Your search keyword '"DREBER, ANNA"' showing total 562 results

Start Over Author "DREBER, ANNA"

562 results on '"DREBER, ANNA"'

1. Promoting Reproducibility and Replicability in Political Science

Author: Brodeur, Abel, Esterling, Kevin, Ankel-Peters, Jörg, Bueno, Natália S, Desposato, Scott, Dreber, Anna, Genovese, Federica, Green, Donald P, Hepplewhite, Matthew, de la Guardia, Fernando Hoces, Johannesson, Magnus, Kotsadam, Andreas, Miguel, Edward, Velez, Yamil R, and Young, Lauren
Subjects: Political Science, Human Society, Reproducibility, replicability, political science, Political science
Abstract: This article reviews and summarizes current reproduction and replication practices in political science. We first provide definitions for reproducibility and replicability. We then review data availability policies for 28 leading political science journals and present the results from a survey of editors about their willingness to publish comments and replications. We discuss new initiatives that seek to promote and generate high-quality reproductions and replications. Finally, we make the case for standards and practices that may help increase data availability, reproducibility, and replicability in political science.
Published: 2024

2. Reproduction and replication at scale

Author: Brodeur, Abel, Dreber, Anna, Hoces de la Guardia, Fernando, and Miguel, Edward
Subjects: Biomedical and Clinical Sciences, Health Sciences, Psychology, Humans, Reproduction, Models, Biological, Biomedical and clinical sciences, Health sciences
Published: 2024

3. On the trajectory of discrimination: A meta-analysis and forecasting survey capturing 44 years of field experiments on gender and hiring decisions

Author: Schaerer, Michael, du Plessis, Christilene, Nguyen, My Hoang Bao, van Aert, Robbie CM, Tiokhin, Leo, Lakens, Daniël, Clemente, Elena Giulia, Pfeiffer, Thomas, Dreber, Anna, Johannesson, Magnus, Clark, Cory J, Collaboration, Gender Audits Forecasting, and Uhlmann, Eric Luis
Subjects: Commerce, Management, Tourism and Services, Psychology, Gender Equality, Psychology and Cognitive Sciences, Social Psychology, Commerce, management, tourism and services
Abstract: A preregistered meta-analysis, including 244 effect sizes from 85 field audits and 361,645 individual job applications, tested for gender bias in hiring practices in female-stereotypical and gender-balanced as well as male-stereotypical jobs from 1976 to 2020. A “red team” of independent experts was recruited to increase the rigor and robustness of our meta-analytic approach. A forecasting survey further examined whether laypeople (n = 499 nationally representative adults) and scientists (n = 312) could predict the results. Forecasters correctly anticipated reductions in discrimination against female candidates over time. However, both scientists and laypeople overestimated the continuation of bias against female candidates. Instead, selection bias in favor of male over female candidates was eliminated and, if anything, slightly reversed in sign starting in 2009 for mixed-gender and male-stereotypical jobs in our sample. Forecasters further failed to anticipate that discrimination against male candidates for stereotypically female jobs would remain stable across the decades.
Published: 2023

4. Replication games: how to make reproducibility research more systematic

Author: Brodeur, Abel, Dreber, Anna, Hoces de la Guardia, Fernando, and Miguel, Edward
Subjects: Economics, Applied Economics, Psychology, Reproducibility of Results, Research, Research Design, Research data, Research management, Scientific community, Sociology, General Science & Technology
Published: 2023

5. Computational Reproducibility in Finance: Evidence from 1,000 Tests.

Author: Pérignon, Christophe, Akmansoy, Olivier, Hurlin, Christophe, Dreber, Anna, Holzmeister, Felix, Huber, Jürgen, Johannesson, Magnus, Kirchler, Michael, Menkveld, Albert J, Razen, Michael, and Weitzel, Utz
Subjects: REPRODUCIBLE research, FINANCE, RESEARCH questions, RESEARCH personnel, DATA analysis, DATA analysis software
Abstract: We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers' code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and those exerting more effort. It is lower for more technical research questions, more complex code, and results lying in the tails of the distribution. Researchers exhibit overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss implementable reproducibility policies for academic journals. [ABSTRACT FROM AUTHOR]
Published: 2024
Full Text: View/download PDF

6. Can laypeople predict the replicability of social science studies without expert intervention: an exploratory study

Author: Wang, Juntao, Lei, Jonathan, Dreber, Anna, Gordon, Michael, Johannesson, Magnus, Pfeiffer, Thomas, and Chen, Yiling
Subjects: Computer Science - Human-Computer Interaction
Abstract: The low replication rate of published studies has long concerned the social science community, making understanding the replicability a critical problem. Several studies have shown that relevant research communities can make predictions about the replicability of individual studies with above-chance accuracy. Follow-up work further indicates that laypeople can also achieve above-chance accuracy in predicting replicability when experts interpret the studies into short descriptions that are more accessible for laypeople. The involvement of scarce expert resources may make these methods expensive from financial and time perspectives. In this work, we explored whether laypeople can predict the replicability of social science studies without expert intervention. We presented laypeople with raw materials truncated from published social science papers and elicited their answers to questions related to the paper. Our results suggested that laypeople were engaged in this technical task, providing reasonable and self-contained answers. The majority of them also demonstrated a good understanding of the material. However, the solicited information had limited predictive power on the actual replication outcomes. We further discuss several lessons we learned compared to the approach with expert intervention to inspire future works.
Published: 2022

7. Examining the generalizability of research findings from archival data.

Author: Delios, Andrew, Clemente, Elena, Wu, Tao, Tan, Hongbin, Wang, Yong, Gordon, Michael, Viganola, Domenico, Chen, Zhaowei, Dreber, Anna, Johannesson, Magnus, Pfeiffer, Thomas, and Uhlmann, Eric
Subjects: archival data, context sensitivity, generalizability, reproducibility, research reliability
Abstract: This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
Published: 2022

8. (Predicting) Replication Outcomes

Author: Dreber, Anna, speaker
Abstract: What share of results replicate in different literatures in some of the experimental social sciences? I will discuss several large replication projects in psychology and economics, where my coauthors and I have redone experiments published in high-impact journals with new and larger samples to see whether the main result replicates. The seminar will also cover our studies on forecasting prediction markets and surveys where researchers attempt to predict replication outcomes and outcomes of new hypothesis tests. I will also talk about recent work on multi-analyst projects and conceptual replications where we either ask researchers to test the same hypotheses on the same data or design different experiments testing the same hypothesis.
Published: 2023
Full Text: View/download PDF

9. Predicting replicability -- analysis of survey and prediction market data from large-scale forecasting projects

Author: Gordon, Michael, Viganola, Domenico, Dreber, Anna, Johannesson, Magnus, and Pfeiffer, Thomas
Subjects: Statistics - Applications
Abstract: The reproducibility of published research has become an important topic in science policy. A number of large-scale replication projects have been conducted to gauge the overall reproducibility in specific academic fields. Here, we present an analysis of data from four studies which sought to forecast the outcomes of replication projects in the social and behavioural sciences, using human experts who participated in prediction markets and answered surveys. Because the number of findings replicated and predicted in each individual study was small, pooling the data offers an opportunity to evaluate hypotheses regarding the performance of prediction markets and surveys at a higher power. In total, peer beliefs were elicited for the replication outcomes of 103 published findings. We find there is information within the scientific community about the replicability of scientific findings, and that both surveys and prediction markets can be used to elicit and aggregate this information. Our results show prediction markets can determine the outcomes of direct replications with 73% accuracy (n=103). Both the prediction market prices and the average survey responses are correlated with outcomes (0.581 and 0.564 respectively, both p < .001). We also found a significant relationship between p-values of the original findings and replication outcomes. The dataset is made available through the R package pooledmaRket and can be used to further study community beliefs towards replications outcomes as elicited in the surveys and prediction markets., Comment: 19 Pages
Published: 2021
Full Text: View/download PDF

10. Consensus-based guidance for conducting and reporting multi-analyst studies

Author: Aczel, Balazs, Szaszi, Barnabas, Nilsonne, Gustav, van den Akker, Olmo R, Albers, Casper J, van Assen, Marcel ALM, Bastiaansen, Jojanneke A, Benjamin, Daniel, Boehm, Udo, Botvinik-Nezer, Rotem, Bringmann, Laura F, Busch, Niko A, Caruyer, Emmanuel, Cataldo, Andrea M, Cowan, Nelson, Delios, Andrew, van Dongen, Noah NN, Donkin, Chris, van Doorn, Johnny B, Dreber, Anna, Dutilh, Gilles, Egan, Gary F, Gernsbacher, Morton Ann, Hoekstra, Rink, Hoffmann, Sabine, Holzmeister, Felix, Huber, Juergen, Johannesson, Magnus, Jonas, Kai J, Kindel, Alexander T, Kirchler, Michael, Kunkels, Yoram K, Lindsay, D Stephen, Mangin, Jean-Francois, Matzke, Dora, Munafò, Marcus R, Newell, Ben R, Nosek, Brian A, Poldrack, Russell A, van Ravenzwaaij, Don, Rieskamp, Jörg, Salganik, Matthew J, Sarafoglou, Alexandra, Schonberg, Tom, Schweinsberg, Martin, Shanks, David, Silberzahn, Raphael, Simons, Daniel J, Spellman, Barbara A, St-Jean, Samuel, Starns, Jeffrey J, Uhlmann, Eric Luis, Wicherts, Jelte, and Wagenmakers, Eric-Jan
Subjects: Biological Sciences, Biomedical and Clinical Sciences, Health Sciences, Consensus, Data Analysis, Datasets as Topic, Research, multi-analyst, metascience, statistical practice, science forum, expert consensus, analytical variability, None, medicine, neuroscience, none, Biochemistry and Cell Biology, Biological sciences, Biomedical and clinical sciences, Health sciences
Abstract: Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
Published: 2021

11. Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability

Author: Ebersole, Charles R, Mathur, Maya B, Baranski, Erica, Bart-Plange, Diane-Jo, Buttrick, Nicholas R, Chartier, Christopher R, Corker, Katherine S, Corley, Martin, Hartshorne, Joshua K, IJzerman, Hans, Lazarević, Ljiljana B, Rabagliati, Hugh, Ropovik, Ivan, Aczel, Balazs, Aeschbach, Lena F, Andrighetto, Luca, Arnal, Jack D, Arrow, Holly, Babincak, Peter, Bakos, Bence E, Baník, Gabriel, Baskin, Ernest, Belopavlović, Radomir, Bernstein, Michael H, Białek, Michał, Bloxsom, Nicholas G, Bodroža, Bojana, Bonfiglio, Diane BV, Boucher, Leanne, Brühlmann, Florian, Brumbaugh, Claudia C, Casini, Erica, Chen, Yiling, Chiorri, Carlo, Chopik, William J, Christ, Oliver, Ciunci, Antonia M, Claypool, Heather M, Coary, Sean, Čolić, Marija V, Collins, W Matthew, Curran, Paul G, Day, Chris R, Dering, Benjamin, Dreber, Anna, Edlund, John E, Falcão, Filipe, Fedor, Anna, Feinberg, Lily, Ferguson, Ian R, Ford, Máire, Frank, Michael C, Fryberger, Emily, Garinther, Alexander, Gawryluk, Katarzyna, Ashbaugh, Kayla, Giacomantonio, Mauro, Giessner, Steffen R, Grahe, Jon E, Guadagno, Rosanna E, Hałasa, Ewa, Hancock, Peter JB, Hilliard, Rias A, Hüffmeier, Joachim, Hughes, Sean, Idzikowska, Katarzyna, Inzlicht, Michael, Jern, Alan, Jiménez-Leal, William, Johannesson, Magnus, Joy-Gaba, Jennifer A, Kauff, Mathias, Kellier, Danielle J, Kessinger, Grecia, Kidwell, Mallory C, Kimbrough, Amanda M, King, Josiah PJ, Kolb, Vanessa S, Kołodziej, Sabina, Kovacs, Marton, Krasuska, Karolina, Kraus, Sue, Krueger, Lacy E, Kuchno, Katarzyna, Lage, Caio Ambrosio, Langford, Eleanor V, Levitan, Carmel A, de Lima, Tiago Jessé Souza, Lin, Hause, Lins, Samuel, Loy, Jia E, Manfredi, Dylan, Markiewicz, Łukasz, Menon, Madhavi, Mercier, Brett, Metzger, Mitchell, Meyet, Venus, Millen, Ailsa E, Miller, Jeremy K, and Montealegre, Andres
Subjects: Psychology, Cognitive and Computational Psychology, Applied and Developmental Psychology, Behavioral and Social Science, replication, reproducibility, metascience, peer review, Registered Reports, open data, preregistered, Applied and developmental psychology, Cognitive and computational psychology
Abstract: Replication studies in psychological science sometimes fail to reproduce prior findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically significant effect (p
Published: 2020

12. Variability in the analysis of a single neuroimaging dataset by many teams

Author: Botvinik-Nezer, Rotem, Holzmeister, Felix, Camerer, Colin F, Dreber, Anna, Huber, Juergen, Johannesson, Magnus, Kirchler, Michael, Iwanir, Roni, Mumford, Jeanette A, Adcock, R Alison, Avesani, Paolo, Baczkowski, Blazej M, Bajracharya, Aahana, Bakst, Leah, Ball, Sheryl, Barilari, Marco, Bault, Nadège, Beaton, Derek, Beitner, Julia, Benoit, Roland G, Berkers, Ruud MWJ, Bhanji, Jamil P, Biswal, Bharat B, Bobadilla-Suarez, Sebastian, Bortolini, Tiago, Bottenhorn, Katherine L, Bowring, Alexander, Braem, Senne, Brooks, Hayley R, Brudner, Emily G, Calderon, Cristian B, Camilleri, Julia A, Castrellon, Jaime J, Cecchetti, Luca, Cieslik, Edna C, Cole, Zachary J, Collignon, Olivier, Cox, Robert W, Cunningham, William A, Czoschke, Stefan, Dadi, Kamalaker, Davis, Charles P, Luca, Alberto De, Delgado, Mauricio R, Demetriou, Lysia, Dennison, Jeffrey B, Di, Xin, Dickie, Erin W, Dobryakova, Ekaterina, Donnat, Claire L, Dukart, Juergen, Duncan, Niall W, Durnez, Joke, Eed, Amr, Eickhoff, Simon B, Erhart, Andrew, Fontanesi, Laura, Fricke, G Matthew, Fu, Shiguang, Galván, Adriana, Gau, Remi, Genon, Sarah, Glatard, Tristan, Glerean, Enrico, Goeman, Jelle J, Golowin, Sergej AE, González-García, Carlos, Gorgolewski, Krzysztof J, Grady, Cheryl L, Green, Mikella A, Guassi Moreira, João F, Guest, Olivia, Hakimi, Shabnam, Hamilton, J Paul, Hancock, Roeland, Handjaras, Giacomo, Harry, Bronson B, Hawco, Colin, Herholz, Peer, Herman, Gabrielle, Heunis, Stephan, Hoffstaedter, Felix, Hogeveen, Jeremy, Holmes, Susan, Hu, Chuan-Peng, Huettel, Scott A, Hughes, Matthew E, Iacovella, Vittorio, Iordan, Alexandru D, Isager, Peder M, Isik, Ayse I, Jahn, Andrew, Johnson, Matthew R, Johnstone, Tom, Joseph, Michael JE, Juliano, Anthony C, Kable, Joseph W, Kassinopoulos, Michalis, Koba, Cemal, and Kong, Xiang-Zhen
Subjects: Generic health relevance, Brain, Data Analysis, Data Science, Datasets as Topic, Female, Functional Neuroimaging, Humans, Logistic Models, Magnetic Resonance Imaging, Male, Meta-Analysis as Topic, Models, Neurological, Reproducibility of Results, Research Personnel, Software, General Science & Technology
Abstract: Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2-5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.
Published: 2020

13. Crowdsourcing hypothesis tests: Making transparent how design choices shape research results.

Author: Landy, Justin F, Jia, Miaolei Liam, Ding, Isabel L, Viganola, Domenico, Tierney, Warren, Dreber, Anna, Johannesson, Magnus, Pfeiffer, Thomas, Ebersole, Charles R, Gronau, Quentin F, Ly, Alexander, van den Bergh, Don, Marsman, Maarten, Derks, Koen, Wagenmakers, Eric-Jan, Proctor, Andrew, Bartels, Daniel M, Bauman, Christopher W, Brady, William J, Cheung, Felix, Cimpian, Andrei, Dohle, Simone, Donnellan, M Brent, Hahn, Adam, Hall, Michael P, Jiménez-Leal, William, Johnson, David J, Lucas, Richard E, Monin, Benoît, Montealegre, Andres, Mullen, Elizabeth, Pang, Jun, Ray, Jennifer, Reinero, Diego A, Reynolds, Jesse, Sowden, Walter, Storage, Daniel, Su, Runkun, Tworek, Christina M, Van Bavel, Jay J, Walco, Daniel, Wills, Julian, Xu, Xiaobing, Yam, Kai Chi, Yang, Xiaoyu, Cunningham, William A, Schweinsberg, Martin, Urwitz, Molly, The Crowdsourcing Hypothesis Tests Collaboration, and Uhlmann, Eric L
Subjects: conceptual replications, crowdsourcing, forecasting, research robustness, scientific transparency, Crowdsourcing Hypothesis Tests Collaboration, Crowdsourcing, stimulus sampling, replication, Psychology, Cognitive Sciences, Marketing, Social Psychology
Abstract: To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N > 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = -0.37 to + 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Published: 2020

14. Cognitive reflection and 2D:4D: Evidence from a large population sample

Author: Neyse, Levent, Fossen, Frank M., Johannesson, Magnus, and Dreber, Anna
Published: 2023
Full Text: View/download PDF

15. Why do women ask for less?

Author: Dreber, Anna, Heikensten, Emma, and Säve-Söderbergh, Jenny
Published: 2022
Full Text: View/download PDF

16. Politicizing mask-wearing: predicting the success of behavioral interventions among republicans and democrats in the U.S.

Author: Dimant, Eugen, Clemente, Elena Giulia, Pieper, Dylan, Dreber, Anna, and Gelfand, Michele
Published: 2022
Full Text: View/download PDF

17. Redefine statistical significance.

Author: Benjamin, Daniel J, Berger, James O, Johannesson, Magnus, Nosek, Brian A, Wagenmakers, E-J, Berk, Richard, Bollen, Kenneth A, Brembs, Björn, Brown, Lawrence, Camerer, Colin, Cesarini, David, Chambers, Christopher D, Clyde, Merlise, Cook, Thomas D, De Boeck, Paul, Dienes, Zoltan, Dreber, Anna, Easwaran, Kenny, Efferson, Charles, Fehr, Ernst, Fidler, Fiona, Field, Andy P, Forster, Malcolm, George, Edward I, Gonzalez, Richard, Goodman, Steven, Green, Edwin, Green, Donald P, Greenwald, Anthony G, Hadfield, Jarrod D, Hedges, Larry V, Held, Leonhard, Hua Ho, Teck, Hoijtink, Herbert, Hruschka, Daniel J, Imai, Kosuke, Imbens, Guido, Ioannidis, John PA, Jeon, Minjeong, Jones, James Holland, Kirchler, Michael, Laibson, David, List, John, Little, Roderick, Lupia, Arthur, Machery, Edouard, Maxwell, Scott E, McCarthy, Michael, Moore, Don A, Morgan, Stephen L, Munafó, Marcus, Nakagawa, Shinichi, Nyhan, Brendan, Parker, Timothy H, Pericchi, Luis, Perugini, Marco, Rouder, Jeff, Rousseau, Judith, Savalei, Victoria, Schönbrodt, Felix D, Sellke, Thomas, Sinclair, Betsy, Tingley, Dustin, Van Zandt, Trisha, Vazire, Simine, Watts, Duncan J, Winship, Christopher, Wolpert, Robert L, Xie, Yu, Young, Cristobal, Zinman, Jonathan, and Johnson, Valen E
Subjects: Data Interpretation, Statistical, Models, Statistical, Reproducibility of Results, Scholarly Communication
Abstract: We propose to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005.
Published: 2018

18. Nonstandard Errors

Author: MENKVELD, ALBERT J., primary, DREBER, ANNA, additional, HOLZMEISTER, FELIX, additional, HUBER, JUERGEN, additional, JOHANNESSON, MAGNUS, additional, KIRCHLER, MICHAEL, additional, NEUSÜß, SEBASTIAN, additional, RAZEN, MICHAEL, additional, WEITZEL, UTZ, additional, ABAD‐DÍAZ, DAVID, additional, ABUDY, MENACHEM (MENI), additional, ADRIAN, TOBIAS, additional, AIT‐SAHALIA, YACINE, additional, AKMANSOY, OLIVIER, additional, ALCOCK, JAMIE T., additional, ALEXEEV, VITALI, additional, ALOOSH, ARASH, additional, AMATO, LIVIA, additional, AMAYA, DIEGO, additional, ANGEL, JAMES J., additional, AVETIKIAN, ALEJANDRO T., additional, BACH, AMADEUS, additional, BAIDOO, EDWIN, additional, BAKALLI, GAETAN, additional, BAO, LI, additional, BARBON, ANDREA, additional, BASHCHENKO, OKSANA, additional, BINDRA, PARAMPREET C., additional, BJØNNES, GEIR H., additional, BLACK, JEFFREY R., additional, BLACK, BERNARD S., additional, BOGOEV, DIMITAR, additional, CORREA, SANTIAGO BOHORQUEZ, additional, BONDARENKO, OLEG, additional, BOS, CHARLES S., additional, BOSCH‐ROSA, CIRIL, additional, BOURI, ELIE, additional, BROWNLEES, CHRISTIAN, additional, CALAMIA, ANNA, additional, CAO, VIET NGA, additional, CAPELLE‐BLANCARD, GUNTHER, additional, ROMERO, LAURA M. CAPERA, additional, CAPORIN, MASSIMILIANO, additional, CARRION, ALLEN, additional, CASKURLU, TOLGA, additional, CHAKRABARTY, BIDISHA, additional, CHEN, JIAN, additional, CHERNOV, MIKHAIL, additional, CHEUNG, WILLIAM, additional, CHINCARINI, LUDWIG B., additional, CHORDIA, TARUN, additional, CHOW, SHEUNG‐CHI, additional, CLAPHAM, BENJAMIN, additional, COLLIARD, JEAN‐EDOUARD, additional, COMERTON‐FORDE, CAROLE, additional, CURRAN, EDWARD, additional, DAO, THONG, additional, DARE, WALE, additional, DAVIES, RYAN J., additional, BLASIS, RICCARDO DE, additional, NARD, GIANLUCA F. DE, additional, DECLERCK, FANY, additional, DEEV, OLEG, additional, DEGRYSE, HANS, additional, DEKU, SOLOMON Y., additional, DESAGRE, CHRISTOPHE, additional, DIJK, MATHIJS A. VAN, additional, DIM, CHUKWUMA, additional, DIMPFL, THOMAS, additional, DONG, YUN JIANG, additional, DRUMMOND, PHILIP A., additional, DUDDA, TOM, additional, DUEVSKI, TEODOR, additional, DUMITRESCU, ARIADNA, additional, DYAKOV, TEODOR, additional, DYHRBERG, ANNE HAUBO, additional, DZIELIŃSKI, MICHAŁ, additional, EKSI, ASLI, additional, KALAK, IZIDIN EL, additional, ELLEN, SASKIA TER, additional, EUGSTER, NICOLAS, additional, EVANS, MARTIN D. D., additional, FARRELL, MICHAEL, additional, FELEZ‐VINAS, ESTER, additional, FERRARA, GERARDO, additional, FERROUHI, EL MEHDI, additional, FLORI, ANDREA, additional, FLUHARTY‐JAIDEE, JONATHAN T., additional, FOLEY, SEAN D. V., additional, FONG, KINGSLEY Y. L., additional, FOUCAULT, THIERRY, additional, FRANUS, TATIANA, additional, FRANZONI, FRANCESCO, additional, FRIJNS, BART, additional, FRÖMMEL, MICHAEL, additional, FU, SERVANNA M., additional, FÜLLBRUNN, SASCHA C., additional, GAN, BAOQING, additional, GAO, GE, additional, GEHRIG, THOMAS P., additional, GEMAYEL, ROLAND, additional, GERRITSEN, DIRK, additional, GIL‐BAZO, JAVIER, additional, GILDER, DUDLEY, additional, GLOSTEN, LAWRENCE R., additional, GOMEZ, THOMAS, additional, GORBENKO, ARSENY, additional, GRAMMIG, JOACHIM, additional, GRÉGOIRE, VINCENT, additional, GÜÇBILMEZ, UFUK, additional, HAGSTRÖMER, BJÖRN, additional, HAMBUCKERS, JULIEN, additional, HAPNES, ERIK, additional, HARRIS, JEFFREY H., additional, HARRIS, LAWRENCE, additional, HARTMANN, SIMON, additional, HASSE, JEAN‐BAPTISTE, additional, HAUTSCH, NIKOLAUS, additional, HE, XUE‐ZHONG (TONY), additional, HEATH, DAVIDSON, additional, HEDIGER, SIMON, additional, HENDERSHOTT, TERRENCE, additional, HIBBERT, ANN MARIE, additional, HJALMARSSON, ERIK, additional, HOELSCHER, SETH A., additional, HOFFMANN, PETER, additional, HOLDEN, CRAIG W., additional, HORENSTEIN, ALEX R., additional, HUANG, WENQIAN, additional, HUANG, DA, additional, HURLIN, CHRISTOPHE, additional, ILCZUK, KONRAD, additional, IVASHCHENKO, ALEXEY, additional, IYER, SUBRAMANIAN R., additional, JAHANSHAHLOO, HOSSEIN, additional, JALKH, NAJI, additional, JONES, CHARLES M., additional, JURKATIS, SIMON, additional, JYLHÄ, PETRI, additional, KAECK, ANDREAS T., additional, KAISER, GABRIEL, additional, KARAM, ARZÉ, additional, KARMAZIENE, EGLE, additional, KASSNER, BERNHARD, additional, KAUSTIA, MARKKU, additional, KAZAK, EKATERINA, additional, KEARNEY, FEARGHAL, additional, KERVEL, VINCENT VAN, additional, KHAN, SAAD A., additional, KHOMYN, MARTA K., additional, KLEIN, TONY, additional, KLEIN, OLGA, additional, KLOS, ALEXANDER, additional, KOETTER, MICHAEL, additional, KOLOKOLOV, ALEKSEY, additional, KORAJCZYK, ROBERT A., additional, KOZHAN, ROMAN, additional, KRAHNEN, JAN P., additional, KUHLE, PAUL, additional, KWAN, AMY, additional, LAJAUNIE, QUENTIN, additional, LAM, F. Y. ERIC C., additional, LAMBERT, MARIE, additional, LANGLOIS, HUGUES, additional, LAUSEN, JENS, additional, LAUTER, TOBIAS, additional, LEIPPOLD, MARKUS, additional, LEVIN, VLADIMIR, additional, LI, YIJIE, additional, LI, HUI, additional, LIEW, CHEE YOONG, additional, LINDNER, THOMAS, additional, LINTON, OLIVER, additional, LIU, JIACHENG, additional, LIU, ANQI, additional, LLORENTE, GUILLERMO, additional, LOF, MATTHIJS, additional, LOHR, ARIEL, additional, LONGSTAFF, FRANCIS, additional, LOPEZ‐LIRA, ALEJANDRO, additional, MANKAD, SHAWN, additional, MANO, NICOLA, additional, MARCHAL, ALEXIS, additional, MARTINEAU, CHARLES, additional, MAZZOLA, FRANCESCO, additional, MELOSO, DEBRAH, additional, MI, MICHAEL G., additional, MIHET, ROXANA, additional, MOHAN, VIJAY, additional, MOINAS, SOPHIE, additional, MOORE, DAVID, additional, MU, LIANGYI, additional, MURAVYEV, DMITRIY, additional, MURPHY, DERMOT, additional, NESZVEDA, GABOR, additional, NEUMEIER, CHRISTIAN, additional, NIELSSON, ULF, additional, NIMALENDRAN, MAHENDRARAJAH, additional, NOLTE, SVEN, additional, NORDEN, LARS L., additional, O'NEILL, PETER, additional, OBAID, KHALED, additional, ØDEGAARD, BERNT A., additional, ÖSTBERG, PER, additional, PAGNOTTA, EMILIANO, additional, PAINTER, MARCUS, additional, PALAN, STEFAN, additional, PALIT, IMON J., additional, PARK, ANDREAS, additional, PASCUAL, ROBERTO, additional, PASQUARIELLO, PAOLO, additional, PASTOR, LUBOS, additional, PATEL, VINAY, additional, PATTON, ANDREW J., additional, PEARSON, NEIL D., additional, PELIZZON, LORIANA, additional, PELLI, MICHELE, additional, PELSTER, MATTHIAS, additional, PÉRIGNON, CHRISTOPHE, additional, PFIFFER, CAMERON, additional, PHILIP, RICHARD, additional, PLÍHAL, TOMÁŠ, additional, PRAKASH, PUNEET, additional, PRESS, OLIVER‐ALEXANDER, additional, PRODROMOU, TINA, additional, PROKOPCZUK, MARCEL, additional, PUTNINS, TALIS, additional, QIAN, YA, additional, RAIZADA, GAURAV, additional, RAKOWSKI, DAVID, additional, RANALDO, ANGELO, additional, REGIS, LUCA, additional, REITZ, STEFAN, additional, RENAULT, THOMAS, additional, RENJIE, REX W., additional, RENO, ROBERTO, additional, RIDDIOUGH, STEVEN J., additional, RINNE, KALLE, additional, RINTAMÄKI, PAUL, additional, RIORDAN, RYAN, additional, RITTMANNSBERGER, THOMAS, additional, LONGARELA, IÑAKI RODRÍGUEZ, additional, ROESCH, DOMINIK, additional, ROGNONE, LAVINIA, additional, ROSEMAN, BRIAN, additional, ROŞU, IOANID, additional, ROY, SAURABH, additional, RUDOLF, NICOLAS, additional, RUSH, STEPHEN R., additional, RZAYEV, KHALADDIN, additional, RZEŹNIK, ALEKSANDRA A., additional, SANFORD, ANTHONY, additional, SANKARAN, HARIKUMAR, additional, SARKAR, ASANI, additional, SARNO, LUCIO, additional, SCAILLET, OLIVIER, additional, SCHARNOWSKI, STEFAN, additional, SCHENK‐HOPPÉ, KLAUS R., additional, SCHERTLER, ANDREA, additional, SCHNEIDER, MICHAEL, additional, SCHROEDER, FLORIAN, additional, SCHÜRHOFF, NORMAN, additional, SCHUSTER, PHILIPP, additional, SCHWARZ, MARCO A., additional, SEASHOLES, MARK S., additional, SEEGER, NORMAN J., additional, SHACHAR, OR, additional, SHKILKO, ANDRIY, additional, SHUI, JESSICA, additional, SIKIC, MARIO, additional, SIMION, GIORGIA, additional, SMALES, LEE A., additional, SÖDERLIND, PAUL, additional, SOJLI, ELVIRA, additional, SOKOLOV, KONSTANTIN, additional, SÖNKSEN, JANTJE, additional, SPOKEVICIUTE, LAIMA, additional, STEFANOVA, DENITSA, additional, SUBRAHMANYAM, MARTI G., additional, SZASZI, BARNABAS, additional, TALAVERA, OLEKSANDR, additional, TANG, YUEHUA, additional, TAYLOR, NICK, additional, THAM, WING WAH, additional, THEISSEN, ERIK, additional, THIMME, JULIAN, additional, TONKS, IAN, additional, TRAN, HAI, additional, TRAPIN, LUCA, additional, TROLLE, ANDERS B., additional, VADUVA, M. ANDREEA, additional, VALENTE, GIORGIO, additional, NESS, ROBERT A. VAN, additional, VASQUEZ, AURELIO, additional, VEROUSIS, THANOS, additional, VERWIJMEREN, PATRICK, additional, VILHELMSSON, ANDERS, additional, VILKOV, GRIGORY, additional, VLADIMIROV, VLADIMIR, additional, VOGEL, SEBASTIAN, additional, VOIGT, STEFAN, additional, WAGNER, WOLF, additional, WALTHER, THOMAS, additional, WEISS, PATRICK, additional, WEL, MICHEL VAN DER, additional, WERNER, INGRID M., additional, WESTERHOLM, P. JOAKIM, additional, WESTHEIDE, CHRISTIAN, additional, WIKA, HANS C., additional, WIPPLINGER, EVERT, additional, WOLF, MICHAEL, additional, WOLFF, CHRISTIAN C. P., additional, WOLK, LEONARD, additional, WONG, WING‐KEUNG, additional, WRAMPELMEYER, JAN, additional, WU, ZHEN‐XING, additional, XIA, SHUO, additional, XIU, DACHENG, additional, XU, KE, additional, XU, CAIHONG, additional, YADAV, PRADEEP K., additional, YAGÜE, JOSÉ, additional, YAN, CHENG, additional, YANG, ANTTI, additional, YOO, WOONGSUN, additional, YU, WENJIA, additional, YU, YIHE, additional, YU, SHIHAO, additional, YUESHEN, BART Z., additional, YUFEROVA, DARYA, additional, ZAMOJSKI, MARCIN, additional, ZAREEI, ABALFAZL, additional, ZEISBERGER, STEFAN M., additional, ZHANG, LU, additional, ZHANG, S. SARAH, additional, ZHANG, XIAOYU, additional, ZHAO, LU, additional, ZHONG, ZHUO, additional, ZHOU, Z. IVY, additional, ZHOU, CHEN, additional, ZHU, XINGYU S., additional, ZOICAN, MARIUS, additional, and ZWINKELS, REMCO, additional
Published: 2024
Full Text: View/download PDF

19. Selective reporting of placebo tests in top economics journals

Author: Dreber, Anna, primary, Johannesson, Magnus, additional, and Yang, Yifan, additional
Published: 2024
Full Text: View/download PDF

20. #EEGManyLabs: Investigating the replicability of influential EEG experiments

Author: Pavlov, Yuri G., Adamian, Nika, Appelhoff, Stefan, Arvaneh, Mahnaz, Benwell, Christopher S.Y., Beste, Christian, Bland, Amy R., Bradford, Daniel E., Bublatzky, Florian, Busch, Niko A., Clayson, Peter E., Cruse, Damian, Czeszumski, Artur, Dreber, Anna, Dumas, Guillaume, Ehinger, Benedikt, Ganis, Giorgio, He, Xun, Hinojosa, José A., Huber-Huber, Christoph, Inzlicht, Michael, Jack, Bradley N., Johannesson, Magnus, Jones, Rhiannon, Kalenkovich, Evgenii, Kaltwasser, Laura, Karimi-Rouzbahani, Hamid, Keil, Andreas, König, Peter, Kouara, Layla, Kulke, Louisa, Ladouceur, Cecile D., Langer, Nicolas, Liesefeld, Heinrich R., Luque, David, MacNamara, Annmarie, Mudrik, Liad, Muthuraman, Muthuraman, Neal, Lauren B., Nilsonne, Gustav, Niso, Guiomar, Ocklenburg, Sebastian, Oostenveld, Robert, Pernet, Cyril R., Pourtois, Gilles, Ruzzoli, Manuela, Sass, Sarah M., Schaefer, Alexandre, Senderecka, Magdalena, Snyder, Joel S., Tamnes, Christian K., Tognoli, Emmanuelle, van Vugt, Marieke K., Verona, Edelyn, Vloeberghs, Robin, Welke, Dominik, Wessel, Jan R., Zakharov, Ilya, and Mushtaq, Faisal
Published: 2021
Full Text: View/download PDF

21. Journal-based replication of experiments: An application to “Being Chosen to Lead”

Author: Drazen, Allan, Dreber, Anna, Ozbay, Erkut Y., and Snowberg, Erik
Published: 2021
Full Text: View/download PDF

22. Are women more generous than men? A meta-analysis

Author: Bilén, David, Dreber, Anna, and Johannesson, Magnus
Published: 2021
Full Text: View/download PDF

23. 2D:4D does not predict economic preferences: Evidence from a large, representative sample

Author: Neyse, Levent, Johannesson, Magnus, and Dreber, Anna
Published: 2021
Full Text: View/download PDF

24. A creative destruction approach to replication: Implicit work and sex morality across cultures

Author: Tierney, Warren, Hardy, Jay, III, Ebersole, Charles R., Viganola, Domenico, Clemente, Elena Giulia, Gordon, Michael, Hoogeveen, Suzanne, Haaf, Julia, Dreber, Anna, Johannesson, Magnus, Pfeiffer, Thomas, Huang, Jason L., Vaughn, Leigh Ann, DeMarree, Kenneth, Igou, Eric R., Chapman, Hanah, Gantman, Ana, Vanaman, Matthew, Wylie, Jordan, Storbeck, Justin, Andreychik, Michael R., McPhetres, Jon, and Uhlmann, Eric Luis
Published: 2021
Full Text: View/download PDF

25. Non-Standard Errors

Author: Menkveld, Albert J., Abudy, Menachem (Meni), Grammig, Joachim, Gregoire, Vincent, Hagströmer, Björn, Hambuckers, Julien, Hapnes, Erik, Harris, Jeffrey H., Harris, Lawrence, Hartmann, Simon, Hasse, Jean-Baptiste, Hautsch, Nikolaus, Adrian, Tobias, He, Xue-Zhong 'Tony', Heath, Davidson, Hediger, Simon, Hendershott, Terrence J., Hibbert, Ann Marie, Hjalmarsson, Erik, Hoelscher, Seth, Hoffmann, Peter, Holden, Craig W., Horenstein, Alex R., Ait-Sahalia, Yacine, Huang, Wenqian, Huang, Da, Hurlin, Christophe, Ivashchenko, Alexey, Iyer, Subramanian R., Jahanshahloo, Hossein, Jalkh, Naji, Jones, Charles M., Jurkatis, Simon, Jylha, Petri, Akmansoy, Olivier, Kaeck, Andreas, Kaiser, Gabriel, Karam, Arzé, Karmaziene, Egle, Kassner, Bernhard, Kaustia, Markku, Kazak, Ekaterina, Kearney, Fearghal, van Kervel, Vincent, Khan, Saad, Alcock, Jamie, Khomyn, Marta, Klein, Tony, Klein, Olga, Klos, Alexander, Koetter, Michael, Krahnen, Jan Pieter, Kolokolov, Aleksey, Korajczyk, Robert A., Kozhan, Roman, Kwan, Amy, Alexeev, Vitali, Lajaunie, Quentin, Lam, Full Yet Eric Campbell, Lambert, Marie, Langlois, Hugues, Lausen, Jens, Lauter, Tobias, Leippold, Markus, Levin, Vladimir, Li, Yijie, Li, (Michael) Hui, Aloosh, Arash, Liew, Chee Yoong, Lindner, Thomas, Linton, Oliver B., Liu, Jiacheng, Liu, Anqi, Llorente-Alvarez, Jesus-Guillermo, Lof, Matthijs, Lohr, Ariel, Longstaff, Francis A., Lopez-Lira, Alejandro, Amato, Livia, Mankad, Shawn, Mano, Nicola, Marchal, Alexis, Martineau, Charles, Mazzola, Francesco, Meloso, Debrah C, Mihet, Roxana, Mohan, Vijay, Moinas, Sophie, Moore, David, Amaya, Diego, Mu, Liangyi, Muravyev, Dmitriy, Murphy, Dermot, Neszveda, Gabor, Neumeier, Christian, Nielsson, Ulf, Nimalendran, Mahendrarajah, Nolte, Sven, Nordén, Lars L., O'Neill, Peter, Angel, James J., Obaid, Khaled, Ødegaard, Bernt Arne, Östberg, Per, Painter, Marcus, Palan, Stefan, Palit, Imon, Park, Andreas, Pascual Gascó, Roberto, Pasquariello, Paolo, Pastor, Lubos, Dreber, Anna, Bach, Amadeus, Patel, Vinay, Patton, Andrew J., Pearson, Neil D., Pelizzon, Loriana, Pelster, Matthias, Pérignon, Christophe, Pfiffer, Cameron, Philip, Richard, Plíhal, Tomáš, Prakash, Puneet, Baidoo, Edwin, Press, Oliver-Alexander, Prodromou, Tina, Putnins, Talis J., Raizada, Gaurav, Rakowski, David A., Ranaldo, Angelo, Regis, Luca, Reitz, Stefan, Renault, Thomas, Wang, Renjie, Bakalli, Gaetan, Renò, Roberto, Riddiough, Steven, Rinne, Kalle, Rintamäki, Paul, Riordan, Ryan, Rittmannsberger, Thomas, Rodríguez Longarela, Iñaki, Rösch, Dominik, Rognone, Lavinia, Roseman, Brian, Barbon, Andrea, Rosu, Ioanid, Roy, Saurabh, Rudolf, Nicolas, Rush, Stephen, Rzayev, Khaladdin, Rzeźnik, Aleksandra, Sanford, Anthony, Sankaran, Harikumar, Sarkar, Asani, Sarno, Lucio, Bashchenko, Oksana, Scaillet, Olivier, Scharnowski, Stefan, Schenk-Hoppé, Klaus Reiner, Schertler, Andrea, Schneider, Michael, Schroeder, Florian, Schürhoff, Norman, Schuster, Philipp, Schwarz, Marco A., Seasholes, Mark S., Bindra, Parampreet Christopher, Seeger, Norman, Shachar, Or, Shkilko, Andriy, Shui, Jessica, Sikic, Mario, Simion, Giorgia, Smales, Lee A., Söderlind, Paul, Sojli, Elvira, Sokolov, Konstantin, Bjonnes, Geir Hoidal, Spokeviciute, Laima, Stefanova, Denitsa, Subrahmanyam, Marti G., Neusüss, Sebastian, Szaszi, Barnabas, Talavera, Oleksandr, Tang, Yuehua, Taylor, Nicholas, Tham, Wing Wah, Theissen, Erik, Black, Jeffrey R., Thimme, Julian, Tonks, Ian, Tran, Hai, Trapin, Luca, Trolle, Anders B., Valente, Giorgio, Van Ness, Robert A., Vasquez, Aurelio, Verousis, Thanos, Verwijmeren, Patrick, Black, Bernard S., Vilhelmsson, Anders, Vilkov, Grigory, Vladimirov, Vladimir, Vogel, Sebastian, Voigt, Stefan, Wagner, Wolf, Walther, Thomas, Weiss, Patrick, van der Wel, Michel, Werner, Ingrid M., Bohorquez, Santiago, Westerholm, P. Joakim, Westheide, Christian, Wipplinger, Evert, Wolf, Michael, Wolff, Christian C. P., Wolk, Leonard, Wong, Wing-Keung, Wrampelmeyer, Jan, Xia, Shuo, Xiu, Dacheng, Holzmeister, Felix, Bondarenko, Oleg, Xu, Ke, Xu, Caihong, Yadav, Pradeep K., Yagüe, José, Yan, Cheng, Yang, Antti, Yoo, Woongsun, Yu, Wenjia, Yu, Shihao, Yueshen, Bart Zhou, Bos, Charles S., Yuferova, Darya, Zamojski, Marcin, Zareei, Abalfazl, Zeisberger, Stefan, Zhang, Lu, Zhang, Xiaoyu, Zhong, Zhuo, Zhou, Z. Ivy, Zhou, Chen, Zhu, Xingyu, Bosch-Rosa, Ciril, Zoican, Marius, Zwinkels, Remco C.J., Chen, Jian, Duevski, Teodor, Gao, Ge, Gemayel, Roland, Gilder, Dudley, Kuhle, Paul, Pagnotta, Emiliano, Pelli, Michele, Bouri, Elie, Sönksen, Jantje, Ilczuk, Konrad, Bogoev, Dimitar, Qian, Ya, Wika, Hans C., Yu, Yihe, Zhao, Lu, Mi, Michael, Bao, Li, Brownlees, Christian T., Vaduva, Andreea, Prokopczuk, Marcel, Avetikian, Alejandro, Wu, Zhen-Xing, Calamia, Anna, Cao, Viet Nga, Capelle-Blancard, Gunther, Capera, Laura, Caporin, Massimiliano, Huber, Juergen, Carrion, Allen, Caskurlu, Tolga, Chakrabarty, Bidisha, Chernov, Mikhail, Cheung, William Ming Yan, Chincarini, Ludwig B., Chordia, Tarun, Chow, Sheung Chi, Clapham, Benjamin, Colliard, Jean-Edouard, Johanneson, Magnus, Comerton-Forde, Carole, Curran, Edward, Dao, Thong, Dare, Wale, Davies, Ryan J., De Blasis, Riccardo, De Nard, Gianluca, Declerck, Fany, Deev, Oleg, Degryse, Hans, Kirchler, Michael, Deku, Solomon, Desagre, Christophe, Dim, Chukwuma, Dimpfl, Thomas, Dong, Yun Jiang, Drummond, Philip, Dudda, Tom, Dumitrescu, Ariadna, Dyakov, Teodor, Razen, Michael, Dyhrberg, Anne Haubo, Dzieliński, Michał, Eksi, Asli, El Kalak, Izidin, ter Ellen, Saskia, Eugster, Nicolas, Evans, Martin D.D., Farrell, Michael, Félez-Viñas, Ester, Ferrara, Gerardo, Weitzel, Utz, FERROUHI, El Mehdi, Flori, Andrea, Fluharty-Jaidee, Jonathan, Foley, Sean, Fong, Kingsley Y. L., Foucault, Thierry, Franus, Tatiana, Franzoni, Francesco A., Frijns, Bart, Frömmel, Michael, Abad, David, Fu, Servanna Mianjun, Füllbrunn, Sascha, Gan, Baoqing, Gehrig, Thomas, Gerritsen, Dirk, Gil-Bazo, Javier, Glosten, Lawrence R., Gomez, Thomas, Gorbenko, Arseny, Güçbilmez, Ufuk, Van Dijk, Matthijs A., Menkveld, Albert J., Abudy, Menachem (Meni), Grammig, Joachim, Gregoire, Vincent, Hagströmer, Björn, Hambuckers, Julien, Hapnes, Erik, Harris, Jeffrey H., Harris, Lawrence, Hartmann, Simon, Hasse, Jean-Baptiste, Hautsch, Nikolaus, Adrian, Tobias, He, Xue-Zhong 'Tony', Heath, Davidson, Hediger, Simon, Hendershott, Terrence J., Hibbert, Ann Marie, Hjalmarsson, Erik, Hoelscher, Seth, Hoffmann, Peter, Holden, Craig W., Horenstein, Alex R., Ait-Sahalia, Yacine, Huang, Wenqian, Huang, Da, Hurlin, Christophe, Ivashchenko, Alexey, Iyer, Subramanian R., Jahanshahloo, Hossein, Jalkh, Naji, Jones, Charles M., Jurkatis, Simon, Jylha, Petri, Akmansoy, Olivier, Kaeck, Andreas, Kaiser, Gabriel, Karam, Arzé, Karmaziene, Egle, Kassner, Bernhard, Kaustia, Markku, Kazak, Ekaterina, Kearney, Fearghal, van Kervel, Vincent, Khan, Saad, Alcock, Jamie, Khomyn, Marta, Klein, Tony, Klein, Olga, Klos, Alexander, Koetter, Michael, Krahnen, Jan Pieter, Kolokolov, Aleksey, Korajczyk, Robert A., Kozhan, Roman, Kwan, Amy, Alexeev, Vitali, Lajaunie, Quentin, Lam, Full Yet Eric Campbell, Lambert, Marie, Langlois, Hugues, Lausen, Jens, Lauter, Tobias, Leippold, Markus, Levin, Vladimir, Li, Yijie, Li, (Michael) Hui, Aloosh, Arash, Liew, Chee Yoong, Lindner, Thomas, Linton, Oliver B., Liu, Jiacheng, Liu, Anqi, Llorente-Alvarez, Jesus-Guillermo, Lof, Matthijs, Lohr, Ariel, Longstaff, Francis A., Lopez-Lira, Alejandro, Amato, Livia, Mankad, Shawn, Mano, Nicola, Marchal, Alexis, Martineau, Charles, Mazzola, Francesco, Meloso, Debrah C, Mihet, Roxana, Mohan, Vijay, Moinas, Sophie, Moore, David, Amaya, Diego, Mu, Liangyi, Muravyev, Dmitriy, Murphy, Dermot, Neszveda, Gabor, Neumeier, Christian, Nielsson, Ulf, Nimalendran, Mahendrarajah, Nolte, Sven, Nordén, Lars L., O'Neill, Peter, Angel, James J., Obaid, Khaled, Ødegaard, Bernt Arne, Östberg, Per, Painter, Marcus, Palan, Stefan, Palit, Imon, Park, Andreas, Pascual Gascó, Roberto, Pasquariello, Paolo, Pastor, Lubos, Dreber, Anna, Bach, Amadeus, Patel, Vinay, Patton, Andrew J., Pearson, Neil D., Pelizzon, Loriana, Pelster, Matthias, Pérignon, Christophe, Pfiffer, Cameron, Philip, Richard, Plíhal, Tomáš, Prakash, Puneet, Baidoo, Edwin, Press, Oliver-Alexander, Prodromou, Tina, Putnins, Talis J., Raizada, Gaurav, Rakowski, David A., Ranaldo, Angelo, Regis, Luca, Reitz, Stefan, Renault, Thomas, Wang, Renjie, Bakalli, Gaetan, Renò, Roberto, Riddiough, Steven, Rinne, Kalle, Rintamäki, Paul, Riordan, Ryan, Rittmannsberger, Thomas, Rodríguez Longarela, Iñaki, Rösch, Dominik, Rognone, Lavinia, Roseman, Brian, Barbon, Andrea, Rosu, Ioanid, Roy, Saurabh, Rudolf, Nicolas, Rush, Stephen, Rzayev, Khaladdin, Rzeźnik, Aleksandra, Sanford, Anthony, Sankaran, Harikumar, Sarkar, Asani, Sarno, Lucio, Bashchenko, Oksana, Scaillet, Olivier, Scharnowski, Stefan, Schenk-Hoppé, Klaus Reiner, Schertler, Andrea, Schneider, Michael, Schroeder, Florian, Schürhoff, Norman, Schuster, Philipp, Schwarz, Marco A., Seasholes, Mark S., Bindra, Parampreet Christopher, Seeger, Norman, Shachar, Or, Shkilko, Andriy, Shui, Jessica, Sikic, Mario, Simion, Giorgia, Smales, Lee A., Söderlind, Paul, Sojli, Elvira, Sokolov, Konstantin, Bjonnes, Geir Hoidal, Spokeviciute, Laima, Stefanova, Denitsa, Subrahmanyam, Marti G., Neusüss, Sebastian, Szaszi, Barnabas, Talavera, Oleksandr, Tang, Yuehua, Taylor, Nicholas, Tham, Wing Wah, Theissen, Erik, Black, Jeffrey R., Thimme, Julian, Tonks, Ian, Tran, Hai, Trapin, Luca, Trolle, Anders B., Valente, Giorgio, Van Ness, Robert A., Vasquez, Aurelio, Verousis, Thanos, Verwijmeren, Patrick, Black, Bernard S., Vilhelmsson, Anders, Vilkov, Grigory, Vladimirov, Vladimir, Vogel, Sebastian, Voigt, Stefan, Wagner, Wolf, Walther, Thomas, Weiss, Patrick, van der Wel, Michel, Werner, Ingrid M., Bohorquez, Santiago, Westerholm, P. Joakim, Westheide, Christian, Wipplinger, Evert, Wolf, Michael, Wolff, Christian C. P., Wolk, Leonard, Wong, Wing-Keung, Wrampelmeyer, Jan, Xia, Shuo, Xiu, Dacheng, Holzmeister, Felix, Bondarenko, Oleg, Xu, Ke, Xu, Caihong, Yadav, Pradeep K., Yagüe, José, Yan, Cheng, Yang, Antti, Yoo, Woongsun, Yu, Wenjia, Yu, Shihao, Yueshen, Bart Zhou, Bos, Charles S., Yuferova, Darya, Zamojski, Marcin, Zareei, Abalfazl, Zeisberger, Stefan, Zhang, Lu, Zhang, Xiaoyu, Zhong, Zhuo, Zhou, Z. Ivy, Zhou, Chen, Zhu, Xingyu, Bosch-Rosa, Ciril, Zoican, Marius, Zwinkels, Remco C.J., Chen, Jian, Duevski, Teodor, Gao, Ge, Gemayel, Roland, Gilder, Dudley, Kuhle, Paul, Pagnotta, Emiliano, Pelli, Michele, Bouri, Elie, Sönksen, Jantje, Ilczuk, Konrad, Bogoev, Dimitar, Qian, Ya, Wika, Hans C., Yu, Yihe, Zhao, Lu, Mi, Michael, Bao, Li, Brownlees, Christian T., Vaduva, Andreea, Prokopczuk, Marcel, Avetikian, Alejandro, Wu, Zhen-Xing, Calamia, Anna, Cao, Viet Nga, Capelle-Blancard, Gunther, Capera, Laura, Caporin, Massimiliano, Huber, Juergen, Carrion, Allen, Caskurlu, Tolga, Chakrabarty, Bidisha, Chernov, Mikhail, Cheung, William Ming Yan, Chincarini, Ludwig B., Chordia, Tarun, Chow, Sheung Chi, Clapham, Benjamin, Colliard, Jean-Edouard, Johanneson, Magnus, Comerton-Forde, Carole, Curran, Edward, Dao, Thong, Dare, Wale, Davies, Ryan J., De Blasis, Riccardo, De Nard, Gianluca, Declerck, Fany, Deev, Oleg, Degryse, Hans, Kirchler, Michael, Deku, Solomon, Desagre, Christophe, Dim, Chukwuma, Dimpfl, Thomas, Dong, Yun Jiang, Drummond, Philip, Dudda, Tom, Dumitrescu, Ariadna, Dyakov, Teodor, Razen, Michael, Dyhrberg, Anne Haubo, Dzieliński, Michał, Eksi, Asli, El Kalak, Izidin, ter Ellen, Saskia, Eugster, Nicolas, Evans, Martin D.D., Farrell, Michael, Félez-Viñas, Ester, Ferrara, Gerardo, Weitzel, Utz, FERROUHI, El Mehdi, Flori, Andrea, Fluharty-Jaidee, Jonathan, Foley, Sean, Fong, Kingsley Y. L., Foucault, Thierry, Franus, Tatiana, Franzoni, Francesco A., Frijns, Bart, Frömmel, Michael, Abad, David, Fu, Servanna Mianjun, Füllbrunn, Sascha, Gan, Baoqing, Gehrig, Thomas, Gerritsen, Dirk, Gil-Bazo, Javier, Glosten, Lawrence R., Gomez, Thomas, Gorbenko, Arseny, Güçbilmez, Ufuk, and Van Dijk, Matthijs A.
Abstract: In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
Published: 2024

26. Nonstandard Errors

Author: Universidad de Alicante. Departamento de Economía Financiera y Contabilidad, Menkveld, Albert J., Dreber, Anna, Holzmeister, Felix, Huber, Juergen, Johannesson, Magnus, Kirchler, Michael, Neusüß, Sebastian, Razen, Michael, Weitzel, Utz, Abad, David, Abudy, Menachem (Meni), Adrian, Tobias, Ait-Sahalia, Yacine, Akmansoy, Olivier, Alcock, Jamie T., Alexeev, Vitali, Aloosh, Arash, Amato, Livia, Amaya, Diego, Angel, James J., Avetikian, Alejandro T., Bach, Amadeus, Baidoo, Edwin, Bakalli, Gaetan, Bao, Li, Barbon, Andrea, Bashchenko, Oksana, Bindra, Parampreet C., Bjønnes, Geir H., Black, Jeffrey R., Black, Bernard S., Bogoev, Dimitar, Bohorquez Correa, Santiago, Bondarenko, Oleg, Bos, Charles S., Bosch-Rosa, Ciril, Bouri, Elie, Brownlees, Christian, Calamia, Anna, Cao, Viet Nga, Capelle-Blancard, Gunther, Capera Romero, Laura M., Caporin, Massimiliano, Carrion, Allen, Caskurlu, Tolga, Chakrabarty, Bidisha, Chen, Jian, Chernov, Mikhail, Cheung, William, Chincarini, Ludwig B., Chordia, Tarun, Chow, Sheung-Chi, Clapham, Benjamin, Colliard, Jean-Edouard, Comerton-Forde, Carole, Curran, Edward, Dao, Thong, Dare, Wale, Davies, Ryan J., De Blasis, Riccardo, De Nard, Gianluca F., Declerck, Fany, Deev, Oleg, Degryse, Hans, Deku, Solomon Y., Desagre, Christophe, Van Dijk, Mathijs A., Dim, Chukwuma, Dimpfl, Thomas, Dong, Yun Jiang, Drummond, Philip A., Dudda, Tom, Duevski, Teodor, Dumitrescu, Ariadna, Dyakov, Teodor, Dyhrberg, Anne Haubo, Dzieliński, Michał, Eksi, Asli, El Kalak, Izidin, Ellen, Saskia Ter, Eugster, Nicolas, Evans, Martin D. D., Farrell, Michael, Felez-Vinas, Ester, Ferrara, Gerardo, Ferrouhi, El Mehdi, Flori, Andrea, Fluharty-Jaidee, Jonathan T., Foley, Sean D. V., Fong, Kingsley Y. L., Foucault, Thierry, Franus, Tatiana, Franzoni, Francesco, Frijns, Bart, Frömmel, Michael, Fu, Servanna M., Füllbrunn, Sascha C., Gan, Baoqing, Gao, Ge, Gehrig, Thomas P., Gemayel, Roland, Gerritsen, Dirk, Gil-Bazo, Javier, Gilder, Dudley, Glosten, Lawrence R., Gomez, Thomas, Gorbenko, Arseny, Grammig, Joachim, Grégoire, Vincent, Güçbilmez, Ufuk, Hagströmer, Björn, Hambuckers, Julien, Hapnes, Erik, Harris, Jeffrey H., Harris, Lawrence, Hartmann, Simon, Hasse, Jean-Baptiste, Hautsch, Nikolaus, He, Xue-Zhong (Tony), Heath, Davidson, Hediger, Simon, Hendershott, Terrence, Hibbert, Ann Marie, Hjalmarsson, Erik, Hoelscher, Seth A., Hoffmann, Peter, Holden, Craig W., Horenstein, Alex R., Huang, Wenqian, Huang, Da, Hurlin, Christophe, Ilczuk, Konrad, Ivashchenko, Alexey, Iyer, Subramanian R., Jahanshahloo, Hossein, Jalkh, Naji, Jones, Charles M., Jurkatis, Simon, Jylhä, Petri, Kaeck, Andreas T., Kaiser, Gabriel, Karam, Arzé, Karmaziene, Egle, Kassner, Bernhard, Kaustia, Markku, Kazak, Ekaterina, Kearney, Fearghal, Van Kervel, Vincent, Khan, Saad A., Khomyn, Marta K., Klein, Tony, Klein, Olga, Klos, Alexander, Koetter, Michael, Kolokolov, Aleksey, Korajczyk, Robert A., Kozhan, Roman, Krahnen, Jan P., Kuhle, Paul, Kwan, Amy, Lajaunie, Quentin, Lam, F. Y. Eric C., Lambert, Marie, Langlois, Hugues, Lausen, Jens, Lauter, Tobias, Leippold, Markus, Levin, Vladimir, Li, Yijie, Li, Hui, Liew, Chee Yoong, Lindner, Thomas, Linton, Oliver, Liu, Jiacheng, Liu, Anqi, Llorente, Guillermo, Lof, Matthijs, Lohr, Ariel, Longstaff, Francis, Lopez-Lira, Alejandro, Mankad, Shawn, Mano, Nicola, Marchal, Alexis, Martineau, Charles, Mazzola, Francesco, Meloso, Debrah, Mi, Michael G., Mihet, Roxana, Mohan, Vijay, Moinas, Sophie, Moore, David, Mu, Liangyi, Muravyev, Dmitriy, Murphy, Dermot, Neszveda, Gabor, Neumeier, Christian, Nielsson, Ulf, Nimalendran, Mahendrarajah, Nolte, Sven, Norden, Lars L., O'neill, Peter, Obaid, Khaled, Ødegaard, Bernt A., Östberg, Per, Pagnotta, Emiliano, Painter, Marcus, Palan, Stefan, Palit, Imon J., Park, Andreas, Pascual, Roberto, Pasquariello, Paolo, Pastor, Lubos, Patel, Vinay, Patton, Andrew J., Pearson, Neil D., Pelizzon, Loriana, Pelli, Michele, Pelster, Matthias, Pérignon, Christophe, Pfiffer, Cameron, Philip, Richard, Plíhal, Tomáš, Prakash, Puneet, Press, Oliver-Alexander, Prodromou, Tina, Prokopczuk, Marcel, Putnins, Talis, Qian, Ya, Raizada, Gaurav, Rakowski, David, Ranaldo, Angelo, Regis, Luca, Reitz, Stefan, Renault, Thomas, Renjie, Rex W., Reno, Roberto, Riddiough, Steven J., Rinne, Kalle, Rintamäki, Paul, Riordan, Ryan, Rittmannsberger, Thomas, Rodríguez Longarela, Iñaki, Roesch, Dominik, Rognone, Lavinia, Roseman, Brian, Roşu, Ioanid, Roy, Saurabh, Rudolf, Nicolas, Rush, Stephen R., Rzayev, Khaladdin, Rzeźnik, Aleksandra A., Sanford, Anthony, Sankaran, Harikumar, Sarkar, Asani, Sarno, Lucio, Scaillet, Olivier, Scharnowski, Stefan, Schenk-Hoppé, Klaus R., Schertler, Andrea, Schneider, Michael, Schroeder, Florian, Schürhoff, Norman, Schuster, Philipp, Schwarz, Marco A., Seasholes, Mark S., Seeger, Norman J., Shachar, Or, Shkilko, Andriy, Shui, Jessica, Sikic, Mario, Simion, Giorgia, Smales, Lee A., Söderlind, Paul, Sojli, Elvira, Sokolov, Konstantin, Sönksen, Jantje, Spokeviciute, Laima, Stefanova, Denitsa, Subrahmanyam, Marti G., Szaszi, Barnabas, Talavera, Oleksandr, Tang, Yuehua, Taylor, Nick, Tham, Wing Wah, Theissen, Erik, Thimme, Julian, Tonks, Ian, Tran, Hai, Trapin, Luca, Trolle, Anders B., Vaduva, M. Andreea, Valente, Giorgio, Van Ness, Robert A., Vasquez, Aurelio, Verousis, Thanos, Verwijmeren, Patrick, Vilhelmsson, Anders, Vilkov, Grigory, Vladimirov, Vladimir, Vogel, Sebastian, Voigt, Stefan, Wagner, Wolf, Walther, Thomas, Weiss, Patrick, Van Der Wel, Michel, Werner, Ingrid M., Westerholm, P. Joakim, Westheide, Christian, Wika, Hans C., Wipplinger, Evert, Wolf, Michael, Wolff, Christian C. P., Wolk, Leonard, Wong, Wing-Keung, Wrampelmeyer, Jan, Wu, Zhen-Xing, Xia, Shuo, Xiu, Dacheng, Xu, Ke, Xu, Caihong, Yadav, Pradeep K., Yagüe, José, Yan, Cheng, Yang, Antti, Yoo, Woongsun, Yu, Wenjia, Yu, Yihe, Yu, Shihao, Yueshen, Bart Z., Yuferova, Darya, Zamojski, Marcin, Zareei, Abalfazl, Zeisberger, Stefan M., Zhang, Lu, Zhang, S. Sarah, Zhang, Xiaoyu, Zhao, Lu, Zhong, Zhuo, Zhou, Z. Ivy, Zhou, Chen, Zhu, Xingyu S., Zoican, Marius, Zwinkels, Remco, Universidad de Alicante. Departamento de Economía Financiera y Contabilidad, Menkveld, Albert J., Dreber, Anna, Holzmeister, Felix, Huber, Juergen, Johannesson, Magnus, Kirchler, Michael, Neusüß, Sebastian, Razen, Michael, Weitzel, Utz, Abad, David, Abudy, Menachem (Meni), Adrian, Tobias, Ait-Sahalia, Yacine, Akmansoy, Olivier, Alcock, Jamie T., Alexeev, Vitali, Aloosh, Arash, Amato, Livia, Amaya, Diego, Angel, James J., Avetikian, Alejandro T., Bach, Amadeus, Baidoo, Edwin, Bakalli, Gaetan, Bao, Li, Barbon, Andrea, Bashchenko, Oksana, Bindra, Parampreet C., Bjønnes, Geir H., Black, Jeffrey R., Black, Bernard S., Bogoev, Dimitar, Bohorquez Correa, Santiago, Bondarenko, Oleg, Bos, Charles S., Bosch-Rosa, Ciril, Bouri, Elie, Brownlees, Christian, Calamia, Anna, Cao, Viet Nga, Capelle-Blancard, Gunther, Capera Romero, Laura M., Caporin, Massimiliano, Carrion, Allen, Caskurlu, Tolga, Chakrabarty, Bidisha, Chen, Jian, Chernov, Mikhail, Cheung, William, Chincarini, Ludwig B., Chordia, Tarun, Chow, Sheung-Chi, Clapham, Benjamin, Colliard, Jean-Edouard, Comerton-Forde, Carole, Curran, Edward, Dao, Thong, Dare, Wale, Davies, Ryan J., De Blasis, Riccardo, De Nard, Gianluca F., Declerck, Fany, Deev, Oleg, Degryse, Hans, Deku, Solomon Y., Desagre, Christophe, Van Dijk, Mathijs A., Dim, Chukwuma, Dimpfl, Thomas, Dong, Yun Jiang, Drummond, Philip A., Dudda, Tom, Duevski, Teodor, Dumitrescu, Ariadna, Dyakov, Teodor, Dyhrberg, Anne Haubo, Dzieliński, Michał, Eksi, Asli, El Kalak, Izidin, Ellen, Saskia Ter, Eugster, Nicolas, Evans, Martin D. D., Farrell, Michael, Felez-Vinas, Ester, Ferrara, Gerardo, Ferrouhi, El Mehdi, Flori, Andrea, Fluharty-Jaidee, Jonathan T., Foley, Sean D. V., Fong, Kingsley Y. L., Foucault, Thierry, Franus, Tatiana, Franzoni, Francesco, Frijns, Bart, Frömmel, Michael, Fu, Servanna M., Füllbrunn, Sascha C., Gan, Baoqing, Gao, Ge, Gehrig, Thomas P., Gemayel, Roland, Gerritsen, Dirk, Gil-Bazo, Javier, Gilder, Dudley, Glosten, Lawrence R., Gomez, Thomas, Gorbenko, Arseny, Grammig, Joachim, Grégoire, Vincent, Güçbilmez, Ufuk, Hagströmer, Björn, Hambuckers, Julien, Hapnes, Erik, Harris, Jeffrey H., Harris, Lawrence, Hartmann, Simon, Hasse, Jean-Baptiste, Hautsch, Nikolaus, He, Xue-Zhong (Tony), Heath, Davidson, Hediger, Simon, Hendershott, Terrence, Hibbert, Ann Marie, Hjalmarsson, Erik, Hoelscher, Seth A., Hoffmann, Peter, Holden, Craig W., Horenstein, Alex R., Huang, Wenqian, Huang, Da, Hurlin, Christophe, Ilczuk, Konrad, Ivashchenko, Alexey, Iyer, Subramanian R., Jahanshahloo, Hossein, Jalkh, Naji, Jones, Charles M., Jurkatis, Simon, Jylhä, Petri, Kaeck, Andreas T., Kaiser, Gabriel, Karam, Arzé, Karmaziene, Egle, Kassner, Bernhard, Kaustia, Markku, Kazak, Ekaterina, Kearney, Fearghal, Van Kervel, Vincent, Khan, Saad A., Khomyn, Marta K., Klein, Tony, Klein, Olga, Klos, Alexander, Koetter, Michael, Kolokolov, Aleksey, Korajczyk, Robert A., Kozhan, Roman, Krahnen, Jan P., Kuhle, Paul, Kwan, Amy, Lajaunie, Quentin, Lam, F. Y. Eric C., Lambert, Marie, Langlois, Hugues, Lausen, Jens, Lauter, Tobias, Leippold, Markus, Levin, Vladimir, Li, Yijie, Li, Hui, Liew, Chee Yoong, Lindner, Thomas, Linton, Oliver, Liu, Jiacheng, Liu, Anqi, Llorente, Guillermo, Lof, Matthijs, Lohr, Ariel, Longstaff, Francis, Lopez-Lira, Alejandro, Mankad, Shawn, Mano, Nicola, Marchal, Alexis, Martineau, Charles, Mazzola, Francesco, Meloso, Debrah, Mi, Michael G., Mihet, Roxana, Mohan, Vijay, Moinas, Sophie, Moore, David, Mu, Liangyi, Muravyev, Dmitriy, Murphy, Dermot, Neszveda, Gabor, Neumeier, Christian, Nielsson, Ulf, Nimalendran, Mahendrarajah, Nolte, Sven, Norden, Lars L., O'neill, Peter, Obaid, Khaled, Ødegaard, Bernt A., Östberg, Per, Pagnotta, Emiliano, Painter, Marcus, Palan, Stefan, Palit, Imon J., Park, Andreas, Pascual, Roberto, Pasquariello, Paolo, Pastor, Lubos, Patel, Vinay, Patton, Andrew J., Pearson, Neil D., Pelizzon, Loriana, Pelli, Michele, Pelster, Matthias, Pérignon, Christophe, Pfiffer, Cameron, Philip, Richard, Plíhal, Tomáš, Prakash, Puneet, Press, Oliver-Alexander, Prodromou, Tina, Prokopczuk, Marcel, Putnins, Talis, Qian, Ya, Raizada, Gaurav, Rakowski, David, Ranaldo, Angelo, Regis, Luca, Reitz, Stefan, Renault, Thomas, Renjie, Rex W., Reno, Roberto, Riddiough, Steven J., Rinne, Kalle, Rintamäki, Paul, Riordan, Ryan, Rittmannsberger, Thomas, Rodríguez Longarela, Iñaki, Roesch, Dominik, Rognone, Lavinia, Roseman, Brian, Roşu, Ioanid, Roy, Saurabh, Rudolf, Nicolas, Rush, Stephen R., Rzayev, Khaladdin, Rzeźnik, Aleksandra A., Sanford, Anthony, Sankaran, Harikumar, Sarkar, Asani, Sarno, Lucio, Scaillet, Olivier, Scharnowski, Stefan, Schenk-Hoppé, Klaus R., Schertler, Andrea, Schneider, Michael, Schroeder, Florian, Schürhoff, Norman, Schuster, Philipp, Schwarz, Marco A., Seasholes, Mark S., Seeger, Norman J., Shachar, Or, Shkilko, Andriy, Shui, Jessica, Sikic, Mario, Simion, Giorgia, Smales, Lee A., Söderlind, Paul, Sojli, Elvira, Sokolov, Konstantin, Sönksen, Jantje, Spokeviciute, Laima, Stefanova, Denitsa, Subrahmanyam, Marti G., Szaszi, Barnabas, Talavera, Oleksandr, Tang, Yuehua, Taylor, Nick, Tham, Wing Wah, Theissen, Erik, Thimme, Julian, Tonks, Ian, Tran, Hai, Trapin, Luca, Trolle, Anders B., Vaduva, M. Andreea, Valente, Giorgio, Van Ness, Robert A., Vasquez, Aurelio, Verousis, Thanos, Verwijmeren, Patrick, Vilhelmsson, Anders, Vilkov, Grigory, Vladimirov, Vladimir, Vogel, Sebastian, Voigt, Stefan, Wagner, Wolf, Walther, Thomas, Weiss, Patrick, Van Der Wel, Michel, Werner, Ingrid M., Westerholm, P. Joakim, Westheide, Christian, Wika, Hans C., Wipplinger, Evert, Wolf, Michael, Wolff, Christian C. P., Wolk, Leonard, Wong, Wing-Keung, Wrampelmeyer, Jan, Wu, Zhen-Xing, Xia, Shuo, Xiu, Dacheng, Xu, Ke, Xu, Caihong, Yadav, Pradeep K., Yagüe, José, Yan, Cheng, Yang, Antti, Yoo, Woongsun, Yu, Wenjia, Yu, Yihe, Yu, Shihao, Yueshen, Bart Z., Yuferova, Darya, Zamojski, Marcin, Zareei, Abalfazl, Zeisberger, Stefan M., Zhang, Lu, Zhang, S. Sarah, Zhang, Xiaoyu, Zhao, Lu, Zhong, Zhuo, Zhou, Z. Ivy, Zhou, Chen, Zhu, Xingyu S., Zoican, Marius, and Zwinkels, Remco
Abstract: In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
Published: 2024

27. Non-Standard Errors

Author: Leerstoel Stigchel, Helmholtz Institute, Experimental Psychology (onderzoeksprogramma PF), Finance, UU LEG Research UUSE Multidisciplinary Economics, Sub General Pharmaceutics, Menkveld, Albert J., Abudy, Menachem (Meni), Grammig, Joachim, Gregoire, Vincent, Hagströmer, Björn, Hambuckers, Julien, Hapnes, Erik, Harris, Jeffrey H., Harris, Lawrence, Hartmann, Simon, Hasse, Jean-Baptiste, Hautsch, Nikolaus, Adrian, Tobias, He, Xue-Zhong 'Tony', Heath, Davidson, Hediger, Simon, Hendershott, Terrence J., Hibbert, Ann Marie, Hjalmarsson, Erik, Hoelscher, Seth, Hoffmann, Peter, Holden, Craig W., Horenstein, Alex R., Ait-Sahalia, Yacine, Huang, Wenqian, Huang, Da, Hurlin, Christophe, Ivashchenko, Alexey, Iyer, Subramanian R., Jahanshahloo, Hossein, Jalkh, Naji, Jones, Charles M., Jurkatis, Simon, Jylha, Petri, Akmansoy, Olivier, Kaeck, Andreas, Kaiser, Gabriel, Karam, Arzé, Karmaziene, Egle, Kassner, Bernhard, Kaustia, Markku, Kazak, Ekaterina, Kearney, Fearghal, van Kervel, Vincent, Khan, Saad, Alcock, Jamie, Khomyn, Marta, Klein, Tony, Klein, Olga, Klos, Alexander, Koetter, Michael, Krahnen, Jan Pieter, Kolokolov, Aleksey, Korajczyk, Robert A., Kozhan, Roman, Kwan, Amy, Alexeev, Vitali, Lajaunie, Quentin, Lam, Full Yet Eric Campbell, Lambert, Marie, Langlois, Hugues, Lausen, Jens, Lauter, Tobias, Leippold, Markus, Levin, Vladimir, Li, Yijie, Li, (Michael) Hui, Aloosh, Arash, Liew, Chee Yoong, Lindner, Thomas, Linton, Oliver B., Liu, Jiacheng, Liu, Anqi, Llorente-Alvarez, Jesus-Guillermo, Lof, Matthijs, Lohr, Ariel, Longstaff, Francis A., Lopez-Lira, Alejandro, Amato, Livia, Mankad, Shawn, Mano, Nicola, Marchal, Alexis, Martineau, Charles, Mazzola, Francesco, Meloso, Debrah C, Mihet, Roxana, Mohan, Vijay, Moinas, Sophie, Moore, David, Amaya, Diego, Mu, Liangyi, Muravyev, Dmitriy, Murphy, Dermot, Neszveda, Gabor, Neumeier, Christian, Nielsson, Ulf, Nimalendran, Mahendrarajah, Nolte, Sven, Nordén, Lars L., O'Neill, Peter, Angel, James J., Obaid, Khaled, Ødegaard, Bernt Arne, Östberg, Per, Painter, Marcus, Palan, Stefan, Palit, Imon, Park, Andreas, Pascual Gascó, Roberto, Pasquariello, Paolo, Pastor, Lubos, Dreber, Anna, Bach, Amadeus, Patel, Vinay, Patton, Andrew J., Pearson, Neil D., Pelizzon, Loriana, Pelster, Matthias, Pérignon, Christophe, Pfiffer, Cameron, Philip, Richard, Plíhal, Tomáš, Prakash, Puneet, Baidoo, Edwin, Press, Oliver-Alexander, Prodromou, Tina, Putnins, Talis J., Raizada, Gaurav, Rakowski, David A., Ranaldo, Angelo, Regis, Luca, Reitz, Stefan, Renault, Thomas, Wang, Renjie, Bakalli, Gaetan, Renò, Roberto, Riddiough, Steven, Rinne, Kalle, Rintamäki, Paul, Riordan, Ryan, Rittmannsberger, Thomas, Rodríguez Longarela, Iñaki, Rösch, Dominik, Rognone, Lavinia, Roseman, Brian, Barbon, Andrea, Rosu, Ioanid, Roy, Saurabh, Rudolf, Nicolas, Rush, Stephen, Rzayev, Khaladdin, Rzeźnik, Aleksandra, Sanford, Anthony, Sankaran, Harikumar, Sarkar, Asani, Sarno, Lucio, Bashchenko, Oksana, Scaillet, Olivier, Scharnowski, Stefan, Schenk-Hoppé, Klaus Reiner, Schertler, Andrea, Schneider, Michael, Schroeder, Florian, Schürhoff, Norman, Schuster, Philipp, Schwarz, Marco A., Seasholes, Mark S., Bindra, Parampreet Christopher, Seeger, Norman, Shachar, Or, Shkilko, Andriy, Shui, Jessica, Sikic, Mario, Simion, Giorgia, Smales, Lee A., Söderlind, Paul, Sojli, Elvira, Sokolov, Konstantin, Bjonnes, Geir Hoidal, Spokeviciute, Laima, Stefanova, Denitsa, Subrahmanyam, Marti G., Neusüss, Sebastian, Szaszi, Barnabas, Talavera, Oleksandr, Tang, Yuehua, Taylor, Nicholas, Tham, Wing Wah, Theissen, Erik, Black, Jeffrey R., Thimme, Julian, Tonks, Ian, Tran, Hai, Trapin, Luca, Trolle, Anders B., Valente, Giorgio, Van Ness, Robert A., Vasquez, Aurelio, Verousis, Thanos, Verwijmeren, Patrick, Black, Bernard S., Vilhelmsson, Anders, Vilkov, Grigory, Vladimirov, Vladimir, Vogel, Sebastian, Voigt, Stefan, Wagner, Wolf, Walther, Thomas, Weiss, Patrick, van der Wel, Michel, Werner, Ingrid M., Bohorquez, Santiago, Westerholm, P. Joakim, Westheide, Christian, Wipplinger, Evert, Wolf, Michael, Wolff, Christian C. P., Wolk, Leonard, Wong, Wing-Keung, Wrampelmeyer, Jan, Xia, Shuo, Xiu, Dacheng, Holzmeister, Felix, Bondarenko, Oleg, Xu, Ke, Xu, Caihong, Yadav, Pradeep K., Yagüe, José, Yan, Cheng, Yang, Antti, Yoo, Woongsun, Yu, Wenjia, Yu, Shihao, Yueshen, Bart Zhou, Bos, Charles S., Yuferova, Darya, Zamojski, Marcin, Zareei, Abalfazl, Zeisberger, Stefan, Zhang, Lu, Zhang, Xiaoyu, Zhong, Zhuo, Zhou, Z. Ivy, Zhou, Chen, Zhu, Xingyu, Bosch-Rosa, Ciril, Zoican, Marius, Zwinkels, Remco C.J., Chen, Jian, Duevski, Teodor, Gao, Ge, Gemayel, Roland, Gilder, Dudley, Kuhle, Paul, Pagnotta, Emiliano, Pelli, Michele, Bouri, Elie, Sönksen, Jantje, Ilczuk, Konrad, Bogoev, Dimitar, Qian, Ya, Wika, Hans C., Yu, Yihe, Zhao, Lu, Mi, Michael, Bao, Li, Brownlees, Christian T., Vaduva, Andreea, Prokopczuk, Marcel, Avetikian, Alejandro, Wu, Zhen-Xing, Calamia, Anna, Cao, Viet Nga, Capelle-Blancard, Gunther, Capera, Laura, Caporin, Massimiliano, Huber, Juergen, Carrion, Allen, Caskurlu, Tolga, Chakrabarty, Bidisha, Chernov, Mikhail, Cheung, William Ming Yan, Chincarini, Ludwig B., Chordia, Tarun, Chow, Sheung Chi, Clapham, Benjamin, Colliard, Jean-Edouard, Johanneson, Magnus, Comerton-Forde, Carole, Curran, Edward, Dao, Thong, Dare, Wale, Davies, Ryan J., De Blasis, Riccardo, De Nard, Gianluca, Declerck, Fany, Deev, Oleg, Degryse, Hans, Kirchler, Michael, Deku, Solomon, Desagre, Christophe, Dim, Chukwuma, Dimpfl, Thomas, Dong, Yun Jiang, Drummond, Philip, Dudda, Tom, Dumitrescu, Ariadna, Dyakov, Teodor, Razen, Michael, Dyhrberg, Anne Haubo, Dzieliński, Michał, Eksi, Asli, El Kalak, Izidin, ter Ellen, Saskia, Eugster, Nicolas, Evans, Martin D.D., Farrell, Michael, Félez-Viñas, Ester, Ferrara, Gerardo, Weitzel, Utz, FERROUHI, El Mehdi, Flori, Andrea, Fluharty-Jaidee, Jonathan, Foley, Sean, Fong, Kingsley Y. L., Foucault, Thierry, Franus, Tatiana, Franzoni, Francesco A., Frijns, Bart, Frömmel, Michael, Abad, David, Fu, Servanna Mianjun, Füllbrunn, Sascha, Gan, Baoqing, Gehrig, Thomas, Gerritsen, Dirk, Gil-Bazo, Javier, Glosten, Lawrence R., Gomez, Thomas, Gorbenko, Arseny, Güçbilmez, Ufuk, Van Dijk, Matthijs A., Leerstoel Stigchel, Helmholtz Institute, Experimental Psychology (onderzoeksprogramma PF), Finance, UU LEG Research UUSE Multidisciplinary Economics, Sub General Pharmaceutics, Menkveld, Albert J., Abudy, Menachem (Meni), Grammig, Joachim, Gregoire, Vincent, Hagströmer, Björn, Hambuckers, Julien, Hapnes, Erik, Harris, Jeffrey H., Harris, Lawrence, Hartmann, Simon, Hasse, Jean-Baptiste, Hautsch, Nikolaus, Adrian, Tobias, He, Xue-Zhong 'Tony', Heath, Davidson, Hediger, Simon, Hendershott, Terrence J., Hibbert, Ann Marie, Hjalmarsson, Erik, Hoelscher, Seth, Hoffmann, Peter, Holden, Craig W., Horenstein, Alex R., Ait-Sahalia, Yacine, Huang, Wenqian, Huang, Da, Hurlin, Christophe, Ivashchenko, Alexey, Iyer, Subramanian R., Jahanshahloo, Hossein, Jalkh, Naji, Jones, Charles M., Jurkatis, Simon, Jylha, Petri, Akmansoy, Olivier, Kaeck, Andreas, Kaiser, Gabriel, Karam, Arzé, Karmaziene, Egle, Kassner, Bernhard, Kaustia, Markku, Kazak, Ekaterina, Kearney, Fearghal, van Kervel, Vincent, Khan, Saad, Alcock, Jamie, Khomyn, Marta, Klein, Tony, Klein, Olga, Klos, Alexander, Koetter, Michael, Krahnen, Jan Pieter, Kolokolov, Aleksey, Korajczyk, Robert A., Kozhan, Roman, Kwan, Amy, Alexeev, Vitali, Lajaunie, Quentin, Lam, Full Yet Eric Campbell, Lambert, Marie, Langlois, Hugues, Lausen, Jens, Lauter, Tobias, Leippold, Markus, Levin, Vladimir, Li, Yijie, Li, (Michael) Hui, Aloosh, Arash, Liew, Chee Yoong, Lindner, Thomas, Linton, Oliver B., Liu, Jiacheng, Liu, Anqi, Llorente-Alvarez, Jesus-Guillermo, Lof, Matthijs, Lohr, Ariel, Longstaff, Francis A., Lopez-Lira, Alejandro, Amato, Livia, Mankad, Shawn, Mano, Nicola, Marchal, Alexis, Martineau, Charles, Mazzola, Francesco, Meloso, Debrah C, Mihet, Roxana, Mohan, Vijay, Moinas, Sophie, Moore, David, Amaya, Diego, Mu, Liangyi, Muravyev, Dmitriy, Murphy, Dermot, Neszveda, Gabor, Neumeier, Christian, Nielsson, Ulf, Nimalendran, Mahendrarajah, Nolte, Sven, Nordén, Lars L., O'Neill, Peter, Angel, James J., Obaid, Khaled, Ødegaard, Bernt Arne, Östberg, Per, Painter, Marcus, Palan, Stefan, Palit, Imon, Park, Andreas, Pascual Gascó, Roberto, Pasquariello, Paolo, Pastor, Lubos, Dreber, Anna, Bach, Amadeus, Patel, Vinay, Patton, Andrew J., Pearson, Neil D., Pelizzon, Loriana, Pelster, Matthias, Pérignon, Christophe, Pfiffer, Cameron, Philip, Richard, Plíhal, Tomáš, Prakash, Puneet, Baidoo, Edwin, Press, Oliver-Alexander, Prodromou, Tina, Putnins, Talis J., Raizada, Gaurav, Rakowski, David A., Ranaldo, Angelo, Regis, Luca, Reitz, Stefan, Renault, Thomas, Wang, Renjie, Bakalli, Gaetan, Renò, Roberto, Riddiough, Steven, Rinne, Kalle, Rintamäki, Paul, Riordan, Ryan, Rittmannsberger, Thomas, Rodríguez Longarela, Iñaki, Rösch, Dominik, Rognone, Lavinia, Roseman, Brian, Barbon, Andrea, Rosu, Ioanid, Roy, Saurabh, Rudolf, Nicolas, Rush, Stephen, Rzayev, Khaladdin, Rzeźnik, Aleksandra, Sanford, Anthony, Sankaran, Harikumar, Sarkar, Asani, Sarno, Lucio, Bashchenko, Oksana, Scaillet, Olivier, Scharnowski, Stefan, Schenk-Hoppé, Klaus Reiner, Schertler, Andrea, Schneider, Michael, Schroeder, Florian, Schürhoff, Norman, Schuster, Philipp, Schwarz, Marco A., Seasholes, Mark S., Bindra, Parampreet Christopher, Seeger, Norman, Shachar, Or, Shkilko, Andriy, Shui, Jessica, Sikic, Mario, Simion, Giorgia, Smales, Lee A., Söderlind, Paul, Sojli, Elvira, Sokolov, Konstantin, Bjonnes, Geir Hoidal, Spokeviciute, Laima, Stefanova, Denitsa, Subrahmanyam, Marti G., Neusüss, Sebastian, Szaszi, Barnabas, Talavera, Oleksandr, Tang, Yuehua, Taylor, Nicholas, Tham, Wing Wah, Theissen, Erik, Black, Jeffrey R., Thimme, Julian, Tonks, Ian, Tran, Hai, Trapin, Luca, Trolle, Anders B., Valente, Giorgio, Van Ness, Robert A., Vasquez, Aurelio, Verousis, Thanos, Verwijmeren, Patrick, Black, Bernard S., Vilhelmsson, Anders, Vilkov, Grigory, Vladimirov, Vladimir, Vogel, Sebastian, Voigt, Stefan, Wagner, Wolf, Walther, Thomas, Weiss, Patrick, van der Wel, Michel, Werner, Ingrid M., Bohorquez, Santiago, Westerholm, P. Joakim, Westheide, Christian, Wipplinger, Evert, Wolf, Michael, Wolff, Christian C. P., Wolk, Leonard, Wong, Wing-Keung, Wrampelmeyer, Jan, Xia, Shuo, Xiu, Dacheng, Holzmeister, Felix, Bondarenko, Oleg, Xu, Ke, Xu, Caihong, Yadav, Pradeep K., Yagüe, José, Yan, Cheng, Yang, Antti, Yoo, Woongsun, Yu, Wenjia, Yu, Shihao, Yueshen, Bart Zhou, Bos, Charles S., Yuferova, Darya, Zamojski, Marcin, Zareei, Abalfazl, Zeisberger, Stefan, Zhang, Lu, Zhang, Xiaoyu, Zhong, Zhuo, Zhou, Z. Ivy, Zhou, Chen, Zhu, Xingyu, Bosch-Rosa, Ciril, Zoican, Marius, Zwinkels, Remco C.J., Chen, Jian, Duevski, Teodor, Gao, Ge, Gemayel, Roland, Gilder, Dudley, Kuhle, Paul, Pagnotta, Emiliano, Pelli, Michele, Bouri, Elie, Sönksen, Jantje, Ilczuk, Konrad, Bogoev, Dimitar, Qian, Ya, Wika, Hans C., Yu, Yihe, Zhao, Lu, Mi, Michael, Bao, Li, Brownlees, Christian T., Vaduva, Andreea, Prokopczuk, Marcel, Avetikian, Alejandro, Wu, Zhen-Xing, Calamia, Anna, Cao, Viet Nga, Capelle-Blancard, Gunther, Capera, Laura, Caporin, Massimiliano, Huber, Juergen, Carrion, Allen, Caskurlu, Tolga, Chakrabarty, Bidisha, Chernov, Mikhail, Cheung, William Ming Yan, Chincarini, Ludwig B., Chordia, Tarun, Chow, Sheung Chi, Clapham, Benjamin, Colliard, Jean-Edouard, Johanneson, Magnus, Comerton-Forde, Carole, Curran, Edward, Dao, Thong, Dare, Wale, Davies, Ryan J., De Blasis, Riccardo, De Nard, Gianluca, Declerck, Fany, Deev, Oleg, Degryse, Hans, Kirchler, Michael, Deku, Solomon, Desagre, Christophe, Dim, Chukwuma, Dimpfl, Thomas, Dong, Yun Jiang, Drummond, Philip, Dudda, Tom, Dumitrescu, Ariadna, Dyakov, Teodor, Razen, Michael, Dyhrberg, Anne Haubo, Dzieliński, Michał, Eksi, Asli, El Kalak, Izidin, ter Ellen, Saskia, Eugster, Nicolas, Evans, Martin D.D., Farrell, Michael, Félez-Viñas, Ester, Ferrara, Gerardo, Weitzel, Utz, FERROUHI, El Mehdi, Flori, Andrea, Fluharty-Jaidee, Jonathan, Foley, Sean, Fong, Kingsley Y. L., Foucault, Thierry, Franus, Tatiana, Franzoni, Francesco A., Frijns, Bart, Frömmel, Michael, Abad, David, Fu, Servanna Mianjun, Füllbrunn, Sascha, Gan, Baoqing, Gehrig, Thomas, Gerritsen, Dirk, Gil-Bazo, Javier, Glosten, Lawrence R., Gomez, Thomas, Gorbenko, Arseny, Güçbilmez, Ufuk, and Van Dijk, Matthijs A.
Published: 2024

28. Heterogeneity in effect size estimates.

Author: Holzmeister, Felix, Johannesson, Magnus, Böhm, Robert, Dreber, Anna, Huber, Jürgen, and Kirchler, Michael
Subjects: PATH analysis (Statistics), ERROR rates, HETEROGENEITY, EXPERIMENTAL design, EMPIRICAL research
Abstract: A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty, limiting the generalizability of published scientific findings. We provide a framework for studying heterogeneity in the social sciences and divide heterogeneity into population, design, and analytical heterogeneity. Our framework suggests that after accounting for heterogeneity, the probability that the tested hypothesis is true for the average population, design, and analysis path can be much lower than implied by nominal error rates of statistically significant individual studies. We estimate each type's heterogeneity from 70 multilab replication studies, 11 prospective meta-analyses of studies employing different experimental designs, and 5 multianalyst studies. In our data, population heterogeneity tends to be relatively small, whereas design and analytical heterogeneity are large. Our results should, however, be interpreted cautiously due to the limited number of studies and the large uncertainty in the heterogeneity estimates. We discuss several ways to parse and account for heterogeneity in the context of different methodologies. [ABSTRACT FROM AUTHOR]
Published: 2024
Full Text: View/download PDF

29. A framework for evaluating reproducibility and replicability in economics.

Author: Dreber, Anna and Johannesson, Magnus
Subjects: *MEASUREMENT
Abstract: We propose a framework for evaluating reproducibility and replicability in economics. Reproducibility is defined as testing if the results of an original study can be reproduced using the same data and replicability is defined as testing if the results of an original study hold in new data. We further divide reproducibility and replicability studies into five types: computational reproducibility, recreate reproducibility, robustness reproducibility, direct replicability and conceptual replicability. In addition to this typology we propose indicators to measure the degree of reproducibility and replicability in both individual studies and for a group of studies. [ABSTRACT FROM AUTHOR]
Published: 2024
Full Text: View/download PDF

30. Creative destruction in science

Author: Tierney, Warren, Hardy, Jay H., III, Ebersole, Charles R., Leavitt, Keith, Viganola, Domenico, Clemente, Elena Giulia, Gordon, Michael, Dreber, Anna, Johannesson, Magnus, Pfeiffer, Thomas, and Uhlmann, Eric Luis
Published: 2020
Full Text: View/download PDF

31. Subjective Evidence Evaluation Survey For Multi-Analyst Studies

Author: Sarafoglou, Alexandra, primary, Hoogeveen, Suzanne, additional, van den Bergh, Don, additional, Aczel, Balazs, additional, Albers, Casper J, additional, Althoff, Tim, additional, Botvinik-Nezer, Rotem, additional, Busch, Niko, additional, Cataldo, Andrea Michael, additional, Devezer, Berna, additional, van Dongen, Noah N'Djaye Nikolai, additional, Dreber, Anna, additional, Fried, Eiko I, additional, Hoekstra, Rink, additional, Hoffmann, Sabine, additional, Holzmeister, Felix, additional, Huber, Juergen, additional, Huntington-Klein, Nick, additional, Ioannidis, John P.A., additional, Johannesson, Magnus, additional, Kirchler, Michael, additional, Loken, Eric, additional, Mangin, Jan-Francois, additional, Matzke, Dora, additional, Menkveld, Albert J., additional, Nilsonne, Gustav, additional, van Ravenzwaaij, Don, additional, Schweinsberg, Martin, additional, Schulz-Kümpel, Hannah, additional, Shanks, David, additional, Simons, Daniel J., additional, Spellman, Barbara A., additional, Stoevenbelt, Andrea Helena, additional, Szaszi, Barnabas, additional, Trübutschek, Darinka, additional, tuerlinckx, francis, additional, Uhlmann, Eric Luis, additional, vanpaemel, wolf, additional, Wicherts, Jelte M., additional, and Wagenmakers, Eric-Jan, additional
Published: 2024
Full Text: View/download PDF

32. Heterogeneity in effect size estimates: Empirical evidence and practical implications

Author: Holzmeister, Felix, primary, Johannesson, Magnus, additional, Böhm, Robert, additional, Dreber, Anna, additional, Huber, Juergen, additional, and Kirchler, Michael, additional
Published: 2024
Full Text: View/download PDF

33. Gender, risk preferences and willingness to compete in a random sample of the Swedish population✰

Author: Boschini, Anne, Dreber, Anna, von Essen, Emma, Muren, Astri, and Ranehill, Eva
Published: 2019
Full Text: View/download PDF

34. Exposure to half-dressed women and economic behavior

Author: Bonnier, Evelina, Dreber, Anna, Hederos, Karin, and Sandberg, Anna
Published: 2019
Full Text: View/download PDF

35. Predicting replication outcomes in the Many Labs 2 study

Author: Forsell, Eskil, Viganola, Domenico, Pfeiffer, Thomas, Almenberg, Johan, Wilson, Brad, Chen, Yiling, Nosek, Brian A., Johannesson, Magnus, and Dreber, Anna
Published: 2019
Full Text: View/download PDF

36. The digit ratio (2D:4D) and economic preferences: no robust associations in a sample of 330 women

Author: Parslow, Elle, Ranehill, Eva, Zethraeus, Niklas, Blomberg, Liselott, von Schoultz, Bo, Hirschberg, Angelica Lindén, Johannesson, Magnus, and Dreber, Anna
Published: 2019
Full Text: View/download PDF

37. Gender and altruism in a random sample

Author: Boschini, Anne, Dreber, Anna, von Essen, Emma, Muren, Astri, and Ranehill, Eva
Published: 2018
Full Text: View/download PDF

38. On the trajectory of discrimination: A meta-analysis and forecasting survey capturing 44 years of field experiments on gender and hiring decisions

Author: Schaerer, Michael, primary, du Plessis, Christilene, additional, Nguyen, My Hoang Bao, additional, van Aert, Robbie C.M., additional, Tiokhin, Leo, additional, Lakens, Daniël, additional, Giulia Clemente, Elena, additional, Pfeiffer, Thomas, additional, Dreber, Anna, additional, Johannesson, Magnus, additional, Clark, Cory J., additional, and Luis Uhlmann, Eric, additional
Published: 2023
Full Text: View/download PDF

39. Hormonal Contraceptives Do Not Impact Economic Preferences : Evidence from a Randomized Trial

Author: Ranehill, Eva, Zethraeus, Niklas, Blomberg, Liselott, von Schoultz, Bo, Hirschberg, Angelica Lindén, Johannesson, Magnus, and Dreber, Anna
Published: 2018

40. Competition and moral behavior:A meta-analysis of forty-five crowd-sourced experimental designs

Author: Huber, Christoph, Dreber, Anna, Huber, Jürgen, Johannesson, Magnus, Kirchler, Michael, Weitzel, Utz, Abellán, Miguel, Adayeva, Xeniya, Ay, Fehime Ceren, Barron, Kai, Berry, Zachariah, Bönte, Werner, Brütt, Katharina, Bulutay, Muhammed, Campos-Mercade, Pol, Cardella, Eric, Claassen, Maria Almudena, Cornelissen, Gert, Dawson, Ian G. J., Delnoij, Joyce, Demiral, Elif E., Dimant, Eugen, Doerflinger, Johannes Theodor, Dold, Malte, Emery, Cécile, Fiala, Lenka, Fiedler, Susann, Freddi, Eleonora, Fries, Tilman, Gasiorowska, Agata, Glogowsky, Ulrich, M. Gorny, Paul, Gretton, Jeremy David, Grohmann, Antonia, Hafenbrädl, Sebastian, Handgraaf, Michel, Hanoch, Yaniv, Hart, Einav, Hennig, Max, Hudja, Stanton, Hütter, Mandy, Hyndman, Kyle, Ioannidis, Konstantinos, Isler, Ozan, Jeworrek, Sabrina, Jolles, Daniel, Juanchich, Marie, KC, Raghabendra Pratap, Khadjavi, Menusch, Kugler, Tamar, Li, Shuwen, Lucas, Brian, Mak, Vincent, Mechtel, Mario, Merkle, Christoph, Meyers, Ethan Andrew, Mollerstrom, Johanna, Nesterov, Alexander, Neyse, Levent, Nieken, Petra, Nussberger, Anne-Marie, Palumbo, Helena, Peters, Kim, Pirrone, Angelo, Qin, Xiangdong, Rahal, Rima Maria, Rau, Holger, Rincke, Johannes, Ronzani, Piero, Roth, Yefim, Saral, Ali Seyhun, Schmitz, Jan, Schneider, Florian, Schram, Arthur, Schudy, Simeon, Schweitzer, Maurice E., Schwieren, Christiane, Scopelliti, Irene, Sirota, Miroslav, Sonnemans, Joep, Soraperra, Ivan, Spantig, Lisa, Steimanis, Ivo, Steinmetz, Janina, Suetens, Sigrid, Theodoropoulou, Andriana, Urbig, Diemo, Vorlaufer, Tobias, Waibel, Joschka, Woods, Daniel, Yakobi, Ofir, Yilmaz, Onurcan, Zaleskiewicz, Tomasz, Zeisberger, Stefan, Holzmeister, Felix, Finance, Ethics, Governance and Society, Tinbergen Institute, Spatial Economics, Amsterdam Sustainability Institute, School of Business and Economics, Huber, Christoph [0000-0001-5820-571X], Dreber, Anna [0000-0003-3989-9941], Huber, Jürgen [0000-0003-0073-0321], Johannesson, Magnus [0000-0001-8759-6393], Kirchler, Michael [0000-0002-5416-2545], Weitzel, Utz [0000-0003-0493-9333], Abellán, Miguel [0000-0002-0159-8505], Adayeva, Xeniya [0000-0003-3761-8763], Ay, Fehime Ceren [0000-0003-4444-2268], Barron, Kai [0000-0001-7259-4184], Berry, Zachariah [0000-0002-0827-6437], Brütt, Katharina [0000-0002-3003-0652], Claassen, Maria Almudena [0000-0001-7411-7330], Cornelissen, Gert [0000-0002-0300-4942], Dawson, Ian GJ [0000-0003-0555-9682], Delnoij, Joyce [0000-0002-0627-5318], Dimant, Eugen [0000-0002-5228-6966], Doerflinger, Johannes Theodor [0000-0002-0228-7725], Dold, Malte [0000-0002-1626-9878], Emery, Cécile [0000-0002-7272-1144], Fiala, Lenka [0000-0003-0216-7050], Fiedler, Susann [0000-0001-9337-2142], Freddi, Eleonora [0000-0002-1794-5188], Gasiorowska, Agata [0000-0002-3354-1095], Glogowsky, Ulrich [0000-0002-0781-1119], M Gorny, Paul [0000-0003-3737-2695], Gretton, Jeremy David [0000-0002-2417-4035], Hafenbrädl, Sebastian [0000-0002-5148-766X], Handgraaf, Michel [0000-0003-0809-5393], Hanoch, Yaniv [0000-0001-9453-4588], Hart, Einav [0000-0003-0641-2359], Hütter, Mandy [0000-0002-0952-3831], Hyndman, Kyle [0000-0003-3666-8734], Ioannidis, Konstantinos [0000-0003-2858-4688], Isler, Ozan [0000-0002-4638-2230], Jeworrek, Sabrina [0000-0001-7967-9383], Jolles, Daniel [0000-0003-1277-0793], Juanchich, Marie [0000-0003-0241-9529], Kc, Raghabendra Pratap [0000-0001-5627-5570], Khadjavi, Menusch [0000-0002-8152-3369], Kugler, Tamar [0000-0002-7211-9858], Li, Shuwen [0000-0002-8822-4185], Lucas, Brian [0000-0003-3569-3030], Mak, Vincent [0000-0002-4690-0819], Merkle, Christoph [0000-0002-5141-4639], Meyers, Ethan Andrew [0000-0001-6171-6780], Mollerstrom, Johanna [0000-0002-8097-2058], Nesterov, Alexander [0000-0002-9143-2938], Neyse, Levent [0000-0001-6005-579X], Nieken, Petra [0000-0001-6157-6749], Nussberger, Anne-Marie [0000-0002-1805-9399], Palumbo, Helena [0000-0003-1978-3386], Pirrone, Angelo [0000-0001-5984-7853], Qin, Xiangdong [0000-0002-4471-8730], Rahal, Rima Maria [0000-0002-1404-0471], Ronzani, Piero [0000-0002-8211-6028], Roth, Yefim [0000-0003-2732-0496], Saral, Ali Seyhun [0000-0002-5523-5355], Schmitz, Jan [0000-0002-5410-9322], Schneider, Florian [0000-0002-8406-2756], Schram, Arthur [0000-0002-1767-4998], Schudy, Simeon [0000-0002-4409-0576], Schweitzer, Maurice E [0000-0003-4795-4816], Schwieren, Christiane [0000-0002-5266-3725], Scopelliti, Irene [0000-0001-6712-5332], Sirota, Miroslav [0000-0003-2117-9532], Sonnemans, Joep [0000-0001-7545-7409], Soraperra, Ivan [0000-0002-4596-3885], Spantig, Lisa [0000-0003-0776-3863], Steimanis, Ivo [0000-0002-8550-4675], Steinmetz, Janina [0000-0003-3299-4858], Suetens, Sigrid [0000-0001-6168-6059], Theodoropoulou, Andriana [0000-0003-3100-543X], Vorlaufer, Tobias [0000-0002-1586-5715], Yakobi, Ofir [0000-0002-9253-7483], Yilmaz, Onurcan [0000-0002-6094-7162], Zaleskiewicz, Tomasz [0000-0002-4232-6151], Zeisberger, Stefan [0000-0002-8412-622X], Holzmeister, Felix [0000-0001-9606-0427], and Apollo - University of Cambridge Repository
Subjects: metascience, experimental design, Postdoc Directie - ENR, BF Psychology, Economics, WASS, moral behavior, Urban Economics, H Social Sciences, ddc:330, ddc:500, Institute for Management Research, competition, generalizability
Abstract: Proceedings of the National Academy of Sciences of the United States of America 120(23), e2215572120 (2023). doi:10.1073/pnas.2215572120, Published by National Acad. of Sciences, Washington, DC
Published: 2023

41. Is research in social psychology politically biased? Systematic empirical tests and a forecasting survey to address the controversy

Author: Eitan, Orly, Viganola, Domenico, Inbar, Yoel, Dreber, Anna, Johannesson, Magnus, Pfeiffer, Thomas, Thau, Stefan, and Uhlmann, Eric Luis
Published: 2018
Full Text: View/download PDF

42. Statistical Significance and the Replication Crisis in the Social Sciences

Author: Dreber, Anna and Johannesson, Magnus
Published: 2019
Full Text: View/download PDF

43. “I'm just a soul whose intentions are good”: The role of communication in noisy repeated games

Author: Arechar, Antonio A., Dreber, Anna, Fudenberg, Drew, and Rand, David G.
Published: 2017
Full Text: View/download PDF

44. Non-Standard Errors

Author: Menkveld, Albert J., Dreber, Anna, Declerck, Fany, and Moinas, Sophie
Subjects: B- ECONOMIE ET FINANCE
Abstract: In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is gener-ated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
Published: 2023

45. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

Author: Camerer, Colin F., Dreber, Anna, Holzmeister, Felix, Ho, Teck-Hua, Huber, Jürgen, Johannesson, Magnus, Kirchler, Michael, Nave, Gideon, Nosek, Brian A., Pfeiffer, Thomas, Altmejd, Adam, Buttrick, Nick, Chan, Taizan, Chen, Yiling, Forsell, Eskil, Gampa, Anup, Heikensten, Emma, Hummer, Lily, Imai, Taisuke, Isaksson, Siri, Manfredi, Dylan, Rose, Julia, Wagenmakers, Eric-Jan, and Wu, Hang
Published: 2018
Full Text: View/download PDF

46. Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs

Author: Huber, Christoph, primary, Dreber, Anna, additional, Huber, Jürgen, additional, Johannesson, Magnus, additional, Kirchler, Michael, additional, Weitzel, Utz, additional, Abellán, Miguel, additional, Adayeva, Xeniya, additional, Ay, Fehime Ceren, additional, Barron, Kai, additional, Berry, Zachariah, additional, Bönte, Werner, additional, Brütt, Katharina, additional, Bulutay, Muhammed, additional, Campos-Mercade, Pol, additional, Cardella, Eric, additional, Claassen, Maria Almudena, additional, Cornelissen, Gert, additional, Dawson, Ian G. J., additional, Delnoij, Joyce, additional, Demiral, Elif E., additional, Dimant, Eugen, additional, Doerflinger, Johannes Theodor, additional, Dold, Malte, additional, Emery, Cécile, additional, Fiala, Lenka, additional, Fiedler, Susann, additional, Freddi, Eleonora, additional, Fries, Tilman, additional, Gasiorowska, Agata, additional, Glogowsky, Ulrich, additional, M. Gorny, Paul, additional, Gretton, Jeremy David, additional, Grohmann, Antonia, additional, Hafenbrädl, Sebastian, additional, Handgraaf, Michel, additional, Hanoch, Yaniv, additional, Hart, Einav, additional, Hennig, Max, additional, Hudja, Stanton, additional, Hütter, Mandy, additional, Hyndman, Kyle, additional, Ioannidis, Konstantinos, additional, Isler, Ozan, additional, Jeworrek, Sabrina, additional, Jolles, Daniel, additional, Juanchich, Marie, additional, KC, Raghabendra Pratap, additional, Khadjavi, Menusch, additional, Kugler, Tamar, additional, Li, Shuwen, additional, Lucas, Brian, additional, Mak, Vincent, additional, Mechtel, Mario, additional, Merkle, Christoph, additional, Meyers, Ethan Andrew, additional, Mollerstrom, Johanna, additional, Nesterov, Alexander, additional, Neyse, Levent, additional, Nieken, Petra, additional, Nussberger, Anne-Marie, additional, Palumbo, Helena, additional, Peters, Kim, additional, Pirrone, Angelo, additional, Qin, Xiangdong, additional, Rahal, Rima Maria, additional, Rau, Holger, additional, Rincke, Johannes, additional, Ronzani, Piero, additional, Roth, Yefim, additional, Saral, Ali Seyhun, additional, Schmitz, Jan, additional, Schneider, Florian, additional, Schram, Arthur, additional, Schudy, Simeon, additional, Schweitzer, Maurice E., additional, Schwieren, Christiane, additional, Scopelliti, Irene, additional, Sirota, Miroslav, additional, Sonnemans, Joep, additional, Soraperra, Ivan, additional, Spantig, Lisa, additional, Steimanis, Ivo, additional, Steinmetz, Janina, additional, Suetens, Sigrid, additional, Theodoropoulou, Andriana, additional, Urbig, Diemo, additional, Vorlaufer, Tobias, additional, Waibel, Joschka, additional, Woods, Daniel, additional, Yakobi, Ofir, additional, Yilmaz, Onurcan, additional, Zaleskiewicz, Tomasz, additional, Zeisberger, Stefan, additional, and Holzmeister, Felix, additional
Published: 2023
Full Text: View/download PDF

47. Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs

Author: ManyDesigns, Huber, Christoph, Dreber, Anna, Huber, Jürgen, Delnoij, Joyce, Holzmeister, Felix, ManyDesigns, Huber, Christoph, Dreber, Anna, Huber, Jürgen, Delnoij, Joyce, and Holzmeister, Felix
Abstract: Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity-variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity-estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs-indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.
Published: 2023

48. Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs

Author: Finance, Economics and Institutions, UU LEG Research UUSE Multidisciplinary Economics, Leerstoel Aarts, ManyDesigns, Huber, Christoph, Dreber, Anna, Huber, Jürgen, Delnoij, Joyce, Holzmeister, Felix, Finance, Economics and Institutions, UU LEG Research UUSE Multidisciplinary Economics, Leerstoel Aarts, ManyDesigns, Huber, Christoph, Dreber, Anna, Huber, Jürgen, Delnoij, Joyce, and Holzmeister, Felix
Published: 2023

49. Competition and moral behavior : A meta-analysis of forty-five crowd-sourced experimental designs

Author: Huber, Christoph, Dreber, Anna, Huber, Jürgen, Johannesson, Magnus, Kirchler, Michael, Weitzel, Utz, Abellán, Miguel, Adayeva, Xeniya, Ay, Fehime Ceren, Barron, Kai, Berry, Zachariah, Bönte, Werner, Brütt, Katharina, Bulutay, Muhammed, Campos-Mercade, Pol, Cardella, Eric, Claassen, Maria Almudena, Cornelissen, Gert, Dawson, Ian G.J., Delnoij, Joyce, Demiral, Elif E., Dimant, Eugen, Doerflinger, Johannes Theodor, Dold, Malte, Emery, Cécile, Fiala, Lenka, Fiedler, Susann, Freddi, Eleonora, Fries, Tilman, Gasiorowska, Agata, Glogowsky, Ulrich, Gorny, Paul M., Gretton, Jeremy David, Grohmann, Antonia, Hafenbrädl, Sebastian, Handgraaf, Michel, Hanoch, Yaniv, Hart, Einav, Hennig, Max, Hudja, Stanton, Hütter, Mandy, Hyndman, Kyle, Ioannidis, Konstantinos, Isler, Ozan, Jeworrek, Sabrina, Jolles, Daniel, Juanchich, Marie, Pratap KC, Raghabendra, Khadjavi, Menusch, Kugler, Tamar, Li, Shuwen, Lucas, Brian, Mak, Vincent, Mechtel, Mario, Merkle, Christoph, Meyers, Ethan Andrew, Mollerstrom, Johanna, Nesterov, Alexander, Neyse, Levent, Nieken, Petra, Nussberger, Anne Marie, Palumbo, Helena, Peters, Kim, Pirrone, Angelo, Qin, Xiangdong, Rahal, Rima Maria, Rau, Holger, Rincke, Johannes, Ronzani, Piero, Roth, Yefim, Saral, Ali Seyhun, Schmitz, Jan, Schneider, Florian, Schram, Arthur, Schudy, Simeon, Schweitzer, Maurice E., Schwieren, Christiane, Scopelliti, Irene, Sirota, Miroslav, Sonnemans, Joep, Soraperra, Ivan, Spantig, Lisa, Steimanis, Ivo, Steinmetz, Janina, Suetens, Sigrid, Theodoropoulou, Andriana, Urbig, Diemo, Vorlaufer, Tobias, Waibel, Joschka, Woods, Daniel, Yakobi, Ofir, Yilmaz, Onurcan, Zaleskiewicz, Tomasz, Zeisberger, Stefan, Holzmeister, Felix, Huber, Christoph, Dreber, Anna, Huber, Jürgen, Johannesson, Magnus, Kirchler, Michael, Weitzel, Utz, Abellán, Miguel, Adayeva, Xeniya, Ay, Fehime Ceren, Barron, Kai, Berry, Zachariah, Bönte, Werner, Brütt, Katharina, Bulutay, Muhammed, Campos-Mercade, Pol, Cardella, Eric, Claassen, Maria Almudena, Cornelissen, Gert, Dawson, Ian G.J., Delnoij, Joyce, Demiral, Elif E., Dimant, Eugen, Doerflinger, Johannes Theodor, Dold, Malte, Emery, Cécile, Fiala, Lenka, Fiedler, Susann, Freddi, Eleonora, Fries, Tilman, Gasiorowska, Agata, Glogowsky, Ulrich, Gorny, Paul M., Gretton, Jeremy David, Grohmann, Antonia, Hafenbrädl, Sebastian, Handgraaf, Michel, Hanoch, Yaniv, Hart, Einav, Hennig, Max, Hudja, Stanton, Hütter, Mandy, Hyndman, Kyle, Ioannidis, Konstantinos, Isler, Ozan, Jeworrek, Sabrina, Jolles, Daniel, Juanchich, Marie, Pratap KC, Raghabendra, Khadjavi, Menusch, Kugler, Tamar, Li, Shuwen, Lucas, Brian, Mak, Vincent, Mechtel, Mario, Merkle, Christoph, Meyers, Ethan Andrew, Mollerstrom, Johanna, Nesterov, Alexander, Neyse, Levent, Nieken, Petra, Nussberger, Anne Marie, Palumbo, Helena, Peters, Kim, Pirrone, Angelo, Qin, Xiangdong, Rahal, Rima Maria, Rau, Holger, Rincke, Johannes, Ronzani, Piero, Roth, Yefim, Saral, Ali Seyhun, Schmitz, Jan, Schneider, Florian, Schram, Arthur, Schudy, Simeon, Schweitzer, Maurice E., Schwieren, Christiane, Scopelliti, Irene, Sirota, Miroslav, Sonnemans, Joep, Soraperra, Ivan, Spantig, Lisa, Steimanis, Ivo, Steinmetz, Janina, Suetens, Sigrid, Theodoropoulou, Andriana, Urbig, Diemo, Vorlaufer, Tobias, Waibel, Joschka, Woods, Daniel, Yakobi, Ofir, Yilmaz, Onurcan, Zaleskiewicz, Tomasz, Zeisberger, Stefan, and Holzmeister, Felix
Abstract: Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity-variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity-estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs-indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.
Published: 2023

50. Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs

Author: Huber, Christoph; https://orcid.org/0000-0001-5820-571X, Dreber, Anna; https://orcid.org/0000-0003-3989-9941, Huber, Jürgen; https://orcid.org/0000-0003-0073-0321, Johannesson, Magnus; https://orcid.org/0000-0001-8759-6393, Kirchler, Michael; https://orcid.org/0000-0002-5416-2545, Weitzel, Utz; https://orcid.org/0000-0003-0493-9333, Abellán, Miguel; https://orcid.org/0000-0002-0159-8505, Adayeva, Xeniya; https://orcid.org/0000-0003-3761-8763, Ay, Fehime Ceren; https://orcid.org/0000-0003-4444-2268, Barron, Kai; https://orcid.org/0000-0001-7259-4184, Berry, Zachariah; https://orcid.org/0000-0002-0827-6437, Bönte, Werner, Brütt, Katharina; https://orcid.org/0000-0002-3003-0652, Bulutay, Muhammed, Campos-Mercade, Pol, Cardella, Eric, Claassen, Maria Almudena; https://orcid.org/0000-0001-7411-7330, Cornelissen, Gert; https://orcid.org/0000-0002-0300-4942, Dawson, Ian G J; https://orcid.org/0000-0003-0555-9682, Delnoij, Joyce; https://orcid.org/0000-0002-0627-5318, Demiral, Elif E, Dimant, Eugen; https://orcid.org/0000-0002-5228-6966, Doerflinger, Johannes Theodor; https://orcid.org/0000-0002-0228-7725, Dold, Malte; https://orcid.org/0000-0002-1626-9878, Emery, Cécile; https://orcid.org/0000-0002-7272-1144, Fiala, Lenka; https://orcid.org/0000-0003-0216-7050, Fiedler, Susann; https://orcid.org/0000-0001-9337-2142, Freddi, Eleonora; https://orcid.org/0000-0002-1794-5188, Fries, Tilman, Zeisberger, Stefan; https://orcid.org/0000-0002-8412-622X, Huber, Christoph; https://orcid.org/0000-0001-5820-571X, Dreber, Anna; https://orcid.org/0000-0003-3989-9941, Huber, Jürgen; https://orcid.org/0000-0003-0073-0321, Johannesson, Magnus; https://orcid.org/0000-0001-8759-6393, Kirchler, Michael; https://orcid.org/0000-0002-5416-2545, Weitzel, Utz; https://orcid.org/0000-0003-0493-9333, Abellán, Miguel; https://orcid.org/0000-0002-0159-8505, Adayeva, Xeniya; https://orcid.org/0000-0003-3761-8763, Ay, Fehime Ceren; https://orcid.org/0000-0003-4444-2268, Barron, Kai; https://orcid.org/0000-0001-7259-4184, Berry, Zachariah; https://orcid.org/0000-0002-0827-6437, Bönte, Werner, Brütt, Katharina; https://orcid.org/0000-0002-3003-0652, Bulutay, Muhammed, Campos-Mercade, Pol, Cardella, Eric, Claassen, Maria Almudena; https://orcid.org/0000-0001-7411-7330, Cornelissen, Gert; https://orcid.org/0000-0002-0300-4942, Dawson, Ian G J; https://orcid.org/0000-0003-0555-9682, Delnoij, Joyce; https://orcid.org/0000-0002-0627-5318, Demiral, Elif E, Dimant, Eugen; https://orcid.org/0000-0002-5228-6966, Doerflinger, Johannes Theodor; https://orcid.org/0000-0002-0228-7725, Dold, Malte; https://orcid.org/0000-0002-1626-9878, Emery, Cécile; https://orcid.org/0000-0002-7272-1144, Fiala, Lenka; https://orcid.org/0000-0003-0216-7050, Fiedler, Susann; https://orcid.org/0000-0001-9337-2142, Freddi, Eleonora; https://orcid.org/0000-0002-1794-5188, Fries, Tilman, and Zeisberger, Stefan; https://orcid.org/0000-0002-8412-622X
Abstract: Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity—variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity—estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs—indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.
Published: 2023

Catalog

Books, media, physical & digital resources

See catalog results

Searchworks

Select search scope, currently: Articles Catalog books, media & more in Jio Institute collections Articles journal articles & other e-resources

Search

Search Constraints

Refine your results

Search Limiters

Topic

Publication Year Range

Language

Publication Type

Journal

Region

Database

Publisher

562 results on '"DREBER, ANNA"'

Search Results

Catalog

Select search scope, currently: Articles

Catalog

books, media & more in Jio Institute collections

Articles

journal articles & other e-resources