Search

Additional co-authors: Ivan Ropovik, Balazs Aczel, Lena F. Aeschbach, Luca Andrighetto, Jack D. Arnal, Holly Arrow, Peter Babincak, Bence E. Bakos, Gabriel Banik, Ernest Baskin, Radomir Belopavlovic, Michael H. Bernstein, Michal Bialek, Nicholas G. Bloxsom, Bojana Bodroža, Diane B. V. Bonfiglio, Leanne Boucher, Florian Bruhlmann, Claudia C. Brumbaugh, Erica Casini, Yiling Chen, Carlo Chiorri, William J. Chopik, Oliver Christ, Antonia M. Ciunci, Heather M. Claypool, Sean Coary, Marija V. Cˇolic, W. Matthew Collins, Paul G. Curran, Chris R. Day, Anna Dreber, John E. Edlund, Filipe Falcao, Anna Fedor, Lily Feinberg, Ian R. Ferguson, Maire Ford, Michael C. Frank, Emily Fryberger, Alexander Garinther, Katarzyna Gawryluk, Kayla Ashbaugh, Mauro Giacomantonio, Steffen R. Giessner, Jon E. Grahe, Rosanna E. Guadagno, Ewa Halasa, Rias A. Hilliard, Joachim Huffmeier, Sean Hughes, Katarzyna Idzikowska, Michael Inzlicht, Alan Jern, William Jimenez-Leal, Magnus Johannesson, Jennifer A. Joy-Gaba, Mathias Kauff, Danielle J. Kellier, Grecia Kessinger, Mallory C. Kidwell, Amanda M. Kimbrough, Josiah P. J. King, Vanessa S. Kolb, Sabina Kolodziej, Marton Kovacs, Karolina Krasuska, Sue Kraus, Lacy E. Krueger, Katarzyna Kuchno, Caio Ambrosio Lage, Eleanor V. Langford, Carmel A. Levitan, Tiago Jesse Souza de Lima, Hause Lin, Samuel Lins, Jia E. Loy, Dylan Manfredi, Łukasz Markiewicz, Madhavi Menon, Brett Mercier, Mitchell Metzger, Venus Meyet, Jeremy K. Miller, Andres Montealegre, Don A. Moore, Rafal Muda, Gideon Nave, Austin Lee Nichols, Sarah A. Novak, Christian Nunnally, Ana Orlic, Anna Palinkas, Angelo Panno, Kimberly P. Parks, Ivana Pedovic, Emilian Pekala, Matthew R. Penner, Sebastiaan Pessers, Boban Petrovic, Thomas Pfeiffer, Damian Pienkosz, Emanuele Preti, Danka Puric, Tiago Ramos, Jonathan Ravid, Timothy S. Razza, Katrin Rentzsch, Juliette Richetin, Sean C. Rife, Anna Dalla Rosa, Kaylis Hase Rudy, Janos Salamon, Blair Saunders, Przemyslaw Sawicki, Kathleen Schmidt, Kurt Schuepfer, Thomas Schultze, Stefan Schulz-Hardt, Astrid Schutz, Ani N. Shabazian, Rachel L. Shubella, Adam Siegel, Ruben Silva, Barbara Sioma, Lauren Skorb, Luana Elayne Cunha de Souza, Sara Steegen, L. A. R. Stein, R. Weylin Sternglanz, Darko Stojilovic, Daniel Storage, Gavin Brent Sullivan, Barnabas Szaszi, Peter Szecsi, Orsolya Szoke, Attila Szuts, Manuela Thomae, Natasha D. Tidwell, Carly Tocco, Ann-Kathrin Torka, Francis Tuerlinckx, Wolf Vanpaemel, Leigh Ann Vaughn, Michelangelo Vianello, Domenico Viganola, Maria Vlachou, Ryan J. Walker, Sophia C. Weissgerber, Aaron L. Wichman, Bradford J. Wiggins, Daniel Wolf, Michael J. Wood, David Zealley, Iris Žeželj, Mark Zrubka, and Brian A. Nosek

Risen and Gilovich (2008) found that subjects believed that “tempting fate” would be punished with ironic bad outcomes (a main effect), and that this effect was magnified when subjects were under cognitive load (an interaction). A previous replication study (Frank & Mathur, 2016) that used an online implementation of the protocol on Amazon Mechanical Turk failed to replicate both the main effect and the interaction. Before this replication was run, the authors of the original study expressed concern that the cognitive-load manipulation may be less effective when implemented online than when implemented in the lab and that subjects recruited online may also respond differently to the specific experimental scenario chosen for the replication. A later, large replication project, Many Labs 2 (Klein et al. 2018), replicated the main effect (though the effect size was smaller than in the original study), but the interaction was not assessed. Attempting to replicate the interaction while addressing the original authors’ concerns regarding the protocol for the first replication study, we developed a new protocol in collaboration with the original authors. We used four university sites ( N = 754) chosen for similarity to the site of the original study to conduct a high-powered, preregistered replication focused primarily on the interaction effect. Results from these sites did not support the interaction or the main effect and were comparable to results obtained at six additional universities that were less similar to the original site. Post hoc analyses did not provide strong evidence for statistical inconsistency between the original study’s estimates and our estimates; that is, the original study’s results would not have been extremely unlikely in the estimated distribution of population effects in our sites. We also collected data from a new Mechanical Turk sample under the first replication study’s protocol, and results were not meaningfully different from those obtained with the new protocol at universities similar to the original site. Secondary analyses failed to support proposed substantive mechanisms for the failure to replicate.

Replications in psychological science sometimes fail to reproduce prior findings. If replications use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replications from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) in which the original authors had expressed concerns about the replication designs before data collection and only one of which was “statistically significant” (p < .05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate (Gilbert et al., 2016). We revised the replication protocols and received formal peer review prior to conducting new replications. We administered the RP:P and Revised protocols in multiple laboratories (Median number of laboratories per original study = 6.5; Range 3 to 9; Median total sample = 1279.5; Range 276 to 3512) for high-powered tests of each original finding with both protocols. Overall, Revised protocols produced similar effect sizes as RP:P protocols following the preregistered analysis plan (Δr = .002 or .014, depending on analytic approach). The median effect size for Revised protocols (r = .05) was similar to RP:P protocols (r = .04) and the original RP:P replications (r = .11), and smaller than the original studies (r = .37). The cumulative evidence of original study and three replication attempts suggests that effect sizes for all 10 (median r = .07; range .00 to .15) are 78% smaller on average than original findings (median r = .37; range .19 to .50), with very precisely estimated effects.

When explaining other people's behavior, people generally find some explanations more satisfying than others. We propose that people judge behavior explanations based on two computational principles: simplicity and rational support-the extent to which an explanation makes the behavior "make sense" under the assumption that the person is a rational agent. Furthermore, we present a computational framework based on decision networks that can formalize both of these principles. We tested this account in a series of experiments in which subjects rated or generated explanations for other people's behavior. In Experiments 1 and 2, the explanations varied in what the other person liked and disliked. In Experiment 3, the explanations varied in what the other person knew or believed. Results from Experiments 1 and 2 supported the idea that people rely on both simplicity and rational support. However, Experiment 3 suggested that subjects rely only on rational support when judging explanations of people's behavior that vary in what someone knew.

When explaining other people's behavior, people generally find some explanations more satisfying than others. We propose that people judge behavior explanations based on two computational principles: simplicity and rational support -- the extent to which an explanation makes the behavior "make sense" under the assumption that the person is a rational agent. Furthermore, we present a computational framework based on decision networks that can formalize both of these principles. We tested this account in a series of experiments in which subjects rated or generated explanations for other people's behavior. In Experiments 1 and 2, the explanations varied in what the other person liked and disliked. In Experiment 3, the explanations varied in what the other person knew or believed. Results from Experiments 1 and 2 supported the idea that people rely on both simplicity and rational support. However, Experiment 3 suggested that subjects rely only on rational support when judging explanations of people's behavior that vary in what someone knew.

This article describes the ongoing efforts of a multidisciplinary group of faculty at an undergraduate institution to form a team and compete in the IBM AI XPRIZE competition. We describe the advantages and disadvantages of faculty participation in major engineering competitions over more traditional professional activities at undergraduate engineering institutions. Our discussion is focused on the benefits to three major groups: undergraduate students, faculty, and academic institutions. We use examples from our one year of experience in the competition and from the literature to illustrate these benefits. Already observed benefits from the competition include increased student engagement, development and introduction of a new minor in cognitive science, the purchase of a state-of-the-art robot and a deep learning server, enhanced multidisciplinary collaboration among faculty, and heightened awareness among administrators of the growing importance of artificial intelligence (AI) technologies. Results of a student survey regarding their involvement in with the team are presented.

The ability to predict and reason about other people’s choices is fundamental to social interaction. We propose that people reason about other people’s choices using mental models that are similar to decision networks. Decision networks are extensions of Bayesian networks that incorporate the idea that choices are made in order to achieve goals. In our first experiment, we explore how people predict the choices of others. Our remaining three experiments explore how people infer the goals and knowledge of others by observing the choices that they make. We show that decision networks account for our data better than alternative computational accounts that do not incorporate the notion of goal-directed choice or that do not rely on probabilistic inference.

[Commentary on Boyer & Petersen. (2018). Folk-economic beliefs: An evolutionary cognitive model. Behavioral and Brain Sciences.] Boyer and Petersen argue that a "rudimentary exchange psychology" is responsible for many of people’s folk-economic beliefs that are at odds with the consensus views of economists. However, they focus primarily on macroeconomic beliefs. I argue that the same rudimentary exchange psychology could be expected to produce fairly accurate microeconomic intuitions. Existing evidence supports this prediction.

Inductive inferences about objects, features, categories, and relations have been studied for many years, but there are few attempts to chart the range of inductive problems that humans are able to solve. We present a taxonomy of inductive problems that helps to clarify the relationships between familiar inductive problems such as generalization, categorization, and identification, and that introduces new inductive problems for psychological investigation. Our taxonomy is founded on the idea that semantic knowledge is organized into systems of objects, features, categories, and relations, and we attempt to characterize all of the inductive problems that can arise when these systems are partially observed. Recent studies have begun to address some of the new problems in our taxonomy, and future work should aim to develop unified theories of inductive reasoning that explain how people solve all of the problems in the taxonomy.

People are capable of imagining and generating new category exemplars and categories. This ability has not been addressed by previous models of categorization, most of which focus on classifying category exemplars rather than generating them. We develop a formal account of exemplar and category generation which proposes that category knowledge is represented by probability distributions over exemplars and categories, and that new exemplars and categories are generated by sampling from these distributions. This sampling account of generation is evaluated in two pairs of behavioral experiments. In the first pair of experiments, participants were asked to generate novel exemplars of a category. In the second pair of experiments, participants were asked to generate a novel category after observing exemplars from several related categories. The results suggest that generation is influenced by both structural and distributional properties of the observed categories, and we argue that our data are better explained by the sampling account than by several alternative approaches.

Belief polarization occurs when 2 people with opposing prior beliefs both strengthen their beliefs after observing the same data. Many authors have cited belief polarization as evidence of irrational behavior. We show, however, that some instances of polarization are consistent with a normative account of belief revision. Our analysis uses Bayesian networks to characterize different kinds of relationships between hypotheses and data, and distinguishes between cases in which normative reasoners with opposing beliefs should both strengthen their beliefs, cases in which both should weaken their beliefs, and cases in which one should strengthen and the other should weaken his or her belief. We apply our analysis to several previous studies of belief polarization and present a new experiment that suggests that people tend to update their beliefs in the directions predicted by our normative account.

Developing a full coreference system able to run all the way from raw text to semantic interpretation is a considerable engineering effort, yet there is very limited availability of off-the shelf tools for researchers whose interests are not in coreference, or for researchers who want to concentrate on a specific aspect of the problem. We present BART, a highly modular toolkit for developing coreference applications. In the Johns Hopkins workshop on using lexical and encyclopedic knowledge for entity disambiguation, the toolkit was used to extend a reimplementation of the Soon et al. (2001) proposal with a variety of additional syntactic and knowledge-based features, and experiment with alternative resolution processes, preprocessing tools, and classifiers.