Search

A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty, limiting the generalizability of published scientific findings. We provide a framework for studying heterogeneity in the social sciences and divide heterogeneity into population, design, and analytical heterogeneity. Our framework suggests that after accounting for heterogeneity, the probability that the tested hypothesis is true for the average population, design, and analysis path can be much lower than implied by nominal error rates of statistically significant individual studies. We estimate each type's heterogeneity from 70 multilab replication studies, 11 prospective meta-analyses of studies employing different experimental designs, and 5 multianalyst studies. In our data, population heterogeneity tends to be relatively small, whereas design and analytical heterogeneity are large. Our results should, however, be interpreted cautiously due to the limited number of studies and the large uncertainty in the heterogeneity estimates. We discuss several ways to parse and account for heterogeneity in the context of different methodologies., Competing Interests: Competing interests statement:The authors declare no competing interest.

Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study., Competing Interests: We declare we have no competing interests., (© 2024 The Authors.)

Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity-variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity-estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs-indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.

Analysis of research data entails many choices. As a result, a space of different analytical strategies is open to researchers. Different justifiable analyses may not give similar results. The method of multiple analysts is a way to study the analytical flexibility and behaviour of researchers under naturalistic conditions, as part of the field known as metascience. Analytical flexibility and risks of bias can be counteracted by open data sharing, pre-registration of analysis plans, and registration of clinical trials in trial registers. These measures are particularly important for retrospective studies where analytical flexibility can be greatest, although pre-registration is less useful in this context. Synthetic datasets can be an alternative to pre-registration when used to decide what analyses should be conducted on real datasets by independent parties. All these strategies help build trustworthiness in scientific reports, and improve the reliability of research findings.

Many publications on COVID-19 were released on preprint servers such as medRxiv and bioRxiv. It is unknown how reliable these preprints are, and which ones will eventually be published in scientific journals. In this study, we use crowdsourced human forecasts to predict publication outcomes and future citation counts for a sample of 400 preprints with high Altmetric score. Most of these preprints were published within 1 year of upload on a preprint server (70%), with a considerable fraction (45%) appearing in a high-impact journal with a journal impact factor of at least 10. On average, the preprints received 162 citations within the first year. We found that forecasters can predict if preprints will be published after 1 year and if the publishing journal has high impact. Forecasts are also informative with respect to Google Scholar citations within 1 year of upload on a preprint server. For both types of assessment, we found statistically significant positive correlations between forecasts and observed outcomes. While the forecasts can help to provide a preliminary assessment of preprints at a faster pace than traditional peer-review, it remains to be investigated if such an assessment is suited to identify methodological problems in preprints., (© 2022 The Authors.)

This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability-for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.

Scientists and policymakers seek to choose effective interventions that promote preventative health measures. We evaluated whether academics, behavioral science practitioners, and laypeople (N = 1034) were able to forecast the effectiveness of seven different messages compared to a baseline message for Republicans and Democrats separately. These messages were designed to nudge mask-wearing attitudes, intentions, and behaviors. When examining predictions across political parties, forecasters predicted larger effects than those observed for Democrats compared to Republicans and made more accurate predictions for Republicans compared to Democrats. These results are partly driven by a lack of nudge effects on Democrats, as reported in Gelfand et al. (J Exp Soc Psychol, 2021). Academics and practitioners made more accurate predictions compared to laypeople. Although forecasters' predictions were correlated with the nudge interventions, all groups overestimated the observed results. We discuss potential reasons for why the forecasts did not perform better and how more accurate forecasts of behavioral intervention outcomes could potentially provide insight that can help save resources and increase the efficacy of interventions., (© 2022. The Author(s).)

Replication-an important, uncommon, and misunderstood practice-is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research., Competing Interests: BA, BS, GN, Ov, CA, Mv, JB, DB, UB, RB, LB, NB, EC, AC, NC, AD, Nv, CD, Jv, AD, GD, GE, MG, RH, SH, FH, JH, MJ, KJ, AK, MK, YK, DL, JM, DM, MM, BN, RP, Dv, JR, MS, AS, TS, MS, DS, RS, DS, BS, SS, JS, EU, JW, EW No competing interests declared, BN Executive Director of the Center for Open Science, a non-profit technology and culture change organization with a mission to increase openness, integrity, and reproducibility of research, (© 2021, Aczel et al.)

There is growing awareness across the neuroscience community that the replicability of findings about the relationship between brain activity and cognitive phenomena can be improved by conducting studies with high statistical power that adhere to well-defined and standardised analysis pipelines. Inspired by recent efforts from the psychological sciences, and with the desire to examine some of the foundational findings using electroencephalography (EEG), we have launched #EEGManyLabs, a large-scale international collaborative replication effort. Since its discovery in the early 20th century, EEG has had a profound influence on our understanding of human cognition, but there is limited evidence on the replicability of some of the most highly cited discoveries. After a systematic search and selection process, we have identified 27 of the most influential and continually cited studies in the field. We plan to directly test the replicability of key findings from 20 of these studies in teams of at least three independent laboratories. The design and protocol of each replication effort will be submitted as a Registered Report and peer-reviewed prior to data collection. Prediction markets, open to all EEG researchers, will be used as a forecasting tool to examine which findings the community expects to replicate. This project will update our confidence in some of the most influential EEG findings and generate a large open access database that can be used to inform future research practices. Finally, through this international effort, we hope to create a cultural shift towards inclusive, high-powered multi-laboratory collaborations., (Copyright © 2021 The Authors. Published by Elsevier Ltd.. All rights reserved.)

There is evidence that prediction markets are useful tools to aggregate information on researchers' beliefs about scientific results including the outcome of replications. In this study, we use prediction markets to forecast the results of novel experimental designs that test established theories. We set up prediction markets for hypotheses tested in the Defense Advanced Research Projects Agency's (DARPA) Next Generation Social Science (NGS2) programme. Researchers were invited to bet on whether 22 hypotheses would be supported or not. We define support as a test result in the same direction as hypothesized, with a Bayes factor of at least 10 (i.e. a likelihood of the observed data being consistent with the tested hypothesis that is at least 10 times greater compared with the null hypothesis). In addition to betting on this binary outcome, we asked participants to bet on the expected effect size (in Cohen's d ) for each hypothesis. Our goal was to recruit at least 50 participants that signed up to participate in these markets. While this was the case, only 39 participants ended up actually trading. Participants also completed a survey on both the binary result and the effect size. We find that neither prediction markets nor surveys performed well in predicting outcomes for NGS2., (© 2021 The Authors.)

The reproducibility of published research has become an important topic in science policy. A number of large-scale replication projects have been conducted to gauge the overall reproducibility in specific academic fields. Here, we present an analysis of data from four studies which sought to forecast the outcomes of replication projects in the social and behavioural sciences, using human experts who participated in prediction markets and answered surveys. Because the number of findings replicated and predicted in each individual study was small, pooling the data offers an opportunity to evaluate hypotheses regarding the performance of prediction markets and surveys at a higher power. In total, peer beliefs were elicited for the replication outcomes of 103 published findings. We find there is information within the scientific community about the replicability of scientific findings, and that both surveys and prediction markets can be used to elicit and aggregate this information. Our results show prediction markets can determine the outcomes of direct replications with 73% accuracy (n = 103). Both the prediction market prices, and the average survey responses are correlated with outcomes (0.581 and 0.564 respectively, both p < .001). We also found a significant relationship between p-values of the original findings and replication outcomes. The dataset is made available through the R package "pooledmaRket" and can be used to further study community beliefs towards replications outcomes as elicited in the surveys and prediction markets., Competing Interests: The authors have declared that no competing interests exist.

The Defense Advanced Research Projects Agency (DARPA) programme 'Systematizing Confidence in Open Research and Evidence' (SCORE) aims to generate confidence scores for a large number of research claims from empirical studies in the social and behavioural sciences. The confidence scores will provide a quantitative assessment of how likely a claim will hold up in an independent replication. To create the scores, we follow earlier approaches and use prediction markets and surveys to forecast replication outcomes. Based on an initial set of forecasts for the overall replication rate in SCORE and its dependence on the academic discipline and the time of publication, we show that participants expect replication rates to increase over time. Moreover, they expect replication rates to differ between fields, with the highest replication rate in economics (average survey response 58%), and the lowest in psychology and in education (average survey response of 42% for both fields). These results reveal insights into the academic community's views of the replication crisis, including for research fields for which no large-scale replication studies have been undertaken yet., Competing Interests: We declare we have no competing interests., (© 2020 The Authors.)

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses 1 . The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset 2-5 . Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N > 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = -0.37 to + 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

We measure how accurately replication of experimental results can be predicted by black-box statistical models. With data from four large-scale replication projects in experimental psychology and economics, and techniques from machine learning, we train predictive models and study which variables drive predictable replication. The models predicts binary replication with a cross-validated accuracy rate of 70% (AUC of 0.77) and estimates of relative effect sizes with a Spearman ρ of 0.38. The accuracy level is similar to market-aggregated beliefs of peer scientists [1, 2]. The predictive power is validated in a pre-registered out of sample test of the outcome of [3], where 71% (AUC of 0.73) of replications are predicted correctly and effect size correlations amount to ρ = 0.25. Basic features such as the sample and effect sizes in original papers, and whether reported effects are single-variable main effects or two-variable interactions, are predictive of successful replication. The models presented in this paper are simple tools to produce cheap, prognostic replicability metrics. These models could be useful in institutionalizing the process of evaluation of new findings and guiding resources to those direct replications that are likely to be most informative., Competing Interests: The authors have declared that no competing interests exist.

There is an ongoing debate about the replicability of neuroimaging research. It was suggested that one of the main reasons for the high rate of false positive results is the many degrees of freedom researchers have during data analysis. In the Neuroimaging Analysis Replication and Prediction Study (NARPS), we aim to provide the first scientific evidence on the variability of results across analysis teams in neuroscience. We collected fMRI data from 108 participants during two versions of the mixed gambles task, which is often used to study decision-making under risk. For each participant, the dataset includes an anatomical (T1 weighted) scan and fMRI as well as behavioral data from four runs of the task. The dataset is shared through OpenNeuro and is formatted according to the Brain Imaging Data Structure (BIDS) standard. Data pre-processed with fMRIprep and quality control reports are also publicly shared. This dataset can be used to study decision-making under risk and to test replicability and interpretability of previous results in the field.

Many studies report on the association between 2D:4D, a putative marker for prenatal testosterone exposure, and economic preferences. However, most of these studies have limited sample sizes and test multiple hypotheses (without preregistration). In this study we mainly replicate the common specifications found in the literature for the association between the 2D:4D ratio and risk taking, the willingness to compete, and dictator game giving separately. In a sample of 330 women we find no robust associations between any of these economic preferences and 2D:4D. We find no evidence of a statistically significant relation for 16 of the 18 total regressions we run. The two regression specifications which are statistically significant have not previously been reported and the associations are not in the expected direction, and therefore they are unlikely to represent a real effect., (© The Author(s) 2019.)

Being able to replicate scientific findings is crucial for scientific progress 1-15 . We replicate 21 systematically selected experimental studies in the social sciences published in Nature and Science between 2010 and 2015 16-36 . The replications follow analysis plans reviewed by the original authors and pre-registered prior to the replications. The replications are high powered, with sample sizes on average about five times higher than in the original studies. We find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size. Replicability varies between 12 (57%) and 14 (67%) studies for complementary replicability indicators. Consistent with these results, the estimated true-positive rate is 67% in a Bayesian analysis. The relative effect size of true positives is estimated to be 71%, suggesting that both false positives and inflated effect sizes of true positives contribute to imperfect reproducibility. Furthermore, we find that peer beliefs of replicability are strongly related to replicability, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.

Objective: To determine whether there is a causal effect of oral contraceptive (OC) treatment on general well-being and depressed mood in healthy women., Design: Double-blind, randomized, and placebo-controlled trial., Setting: University hospital., Patient(s): Three hundred and forty healthy women aged 18-35 years randomized to treatment, of whom 332 completed the data collection at follow-up evaluation., Intervention(s): A combined OC (150 μg levonorgestrel and 30 μg ethinylestradiol) or placebo for 3 months of treatment., Main Outcome Measure(s): Primary outcome measures: global score of Psychological General Well-Being Index (PGWBI) and the Beck Depression Inventory (BDI); secondary outcome measures: six separate dimensions of the PGWBI., Result(s): The OC treatment statistically significantly decreased general well-being compared with placebo -4.12 (95% CI, -7.18 to -1.06). Furthermore, OC decreased the following PGWBI dimensions compared with placebo: positive well-being -3.90 (95% CI, -7.78 to -0.01), self-control -6.63 (95% CI, -11.20 to -2.06), and vitality -6.84 (95% CI, -10.80 to -2.88). The effect of OC on depressive symptoms and on the PGWBI dimension depressed mood were not statistically significant., Conclusion(s): This study demonstrates a statistically significant reduction in general well-being by a first-choice OC in comparison with placebo in healthy women. We found no statistically significant effects on depressive symptoms. A reduction in general well-being should be of clinical importance., (Copyright © 2017 American Society for Reproductive Medicine. Published by Elsevier Inc. All rights reserved.)

Individual willingness to enter competitive environments predicts career choices and labor market outcomes. Meanwhile, many people experience competitive contexts as stressful. We use two laboratory experiments to investigate whether factors related to stress can help explain individual differences in tournament entry. Experiment 1 studies whether stress responses (measured as salivary cortisol) to taking part in a mandatory tournament predict individual willingness to participate in a voluntary tournament. We find that competing increases stress levels. This cortisol response does not predict tournament entry for men but is positively and significantly correlated with choosing to enter the tournament for women. In Experiment 2, we exogenously induce physiological stress using the cold-pressor task. We find a positive causal effect of stress on tournament entry for women but no effect for men. Finally, we show that although the effect of stress on tournament entry differs between the genders, stress reactions cannot explain the well-documented gender difference in willingness to compete.

Context: There is a lack of knowledge about how oral contraceptives may affect sexual function., Objective: To determine whether there is a causal effect of oral contraceptives on sexuality. We hypothesized that a widely used pill impairs sexuality., Design: A double-blind, randomized, placebo-controlled trial. Enrollment began in February 2012 and was completed in August 2015., Setting: Karolinska University Hospital, Stockholm, Sweden., Participants: A total of 340 healthy women, aged 18-35 years, were randomized to treatment, and 332 completed the study., Interventions: A combined oral contraceptive (150 μg levonorgestrel and 30 μg ethinylestradiol) or placebo for 3 months of treatment., Main Outcome Measures: The primary outcome was the aggregate score on the Profile of Female Sexual Function (PFSF). Secondary outcomes were the seven domains of the PFSF, the Sexual Activity Log, and the Personal Distress Scale., Results: Overall sexual function was similar in women in the oral contraceptive and placebo groups. The PFSF domains desire (-4.4; 95% confidence interval [CI], -8.49 to -0.38; P = .032), arousal (-5.1; 95% CI, -9.63 to -0.48; P = .030), and pleasure (-5.1; 95% CI, -9.97 to -0.32; P = .036) were significantly reduced in comparison to placebo, whereas orgasm, concern, responsiveness, and self-image were similar between groups. The mean frequency of satisfying sexual episodes and personal distress were also similar between groups., Conclusions: This study shows no negative impact of a levonorgestrel-containing oral contraceptive on overall sexual function, although three of seven sexual function domains were adversely affected.

The replicability of some scientific findings has recently been called into question. To contribute data about replicability in economics, we replicated 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014. All of these replications followed predefined analysis plans that were made publicly available beforehand, and they all have a statistical power of at least 90% to detect the original effect size at the 5% significance level. We found a significant effect in the same direction as in the original study for 11 replications (61%); on average, the replicated effect size is 66% of the original. The replicability rate varies between 67% and 78% for four additional replicability indicators, including a prediction market measure of peer beliefs., (Copyright © 2016, American Association for the Advancement of Science.)

Concerns about a lack of reproducibility of statistically significant results have recently been raised in many fields, and it has been argued that this lack comes at substantial economic costs. We here report the results from prediction markets set up to quantify the reproducibility of 44 studies published in prominent psychology journals and replicated in the Reproducibility Project: Psychology. The prediction markets predict the outcomes of the replications well and outperform a survey of market participants' individual forecasts. This shows that prediction markets are a promising tool for assessing the reproducibility of published scientific results. The prediction markets also allow us to estimate probabilities for the hypotheses being true at different testing stages, which provides valuable information regarding the temporal dynamics of scientific discovery. We find that the hypotheses being tested in psychology typically have low prior probabilities of being true (median, 9%) and that a "statistically significant" finding needs to be confirmed in a well-powered replication to have a high probability of being true. We argue that prediction markets could be used to obtain speedy information about reproducibility at low cost and could potentially even be used to determine which studies to replicate to optimally allocate limited resources into replications.

The 2014 Research Excellence Framework (REF2014) was conducted to assess the quality of research carried out at higher education institutions in the UK over a 6 year period. However, the process was criticized for being expensive and bureaucratic, and it was argued that similar information could be obtained more simply from various existing metrics. We were interested in whether a prediction market on the outcome of REF2014 for 33 chemistry departments in the UK would provide information similar to that obtained during the REF2014 process. Prediction markets have become increasingly popular as a means of capturing what is colloquially known as the 'wisdom of crowds', and enable individuals to trade 'bets' on whether a specific outcome will occur or not. These have been shown to be successful at predicting various outcomes in a number of domains (e.g. sport, entertainment and politics), but have rarely been tested against outcomes based on expert judgements such as those that formed the basis of REF2014.

In this article we compare cooperation among Colombian and Swedish children aged 9-12. We illustrate the dynamics of the prisoner's dilemma in a new task that is easily understood by children and performed during a physical education class. We find no robust evidence of a difference in cooperation between Colombia and Sweden overall. However, Colombian girls cooperate less than Swedish girls. We also find indications that girls in Colombia are less cooperative than boys. Finally, there is also a tendency for children to be more cooperative with boys than with girls on average.

While baseline testosterone has recently been implicated in risk-taking in men, less is known about the effects of changing levels of testosterone on financial risk. Here we attempt to influence testosterone in men by having them win or lose money in a chance-based competition against another male opponent. We employ two treatments where we vary the amount of money at stake so that we can directly compare winners to losers who earn the same amount, thereby abstracting from income effects. We find that men who experience a greater increase in bioactive testosterone take on more risk, an association that remains when controlling for whether the participant won the competition. In fact, whether subjects won the competition did not predict future risk. These results suggest that testosterone change, and thus individual differences in testosterone reactivity, rather than the act of winning or losing, influence financial risk-taking., (Copyright © 2013. Published by Elsevier Ltd.)

Guala argues that there is a mismatch between most laboratory experiments on costly punishment and behavior in the field. In the lab, experimental designs typically suppress retaliation. The same is true for most theoretical models of the co-evolution of costly punishment and cooperation, which a priori exclude the possibility of defectors punishing cooperators.

The 7 repeat (7R) allele of the dopamine receptor D4 gene has been associated with attention deficit hyperactivity disorder and risk taking. On the cross-population scale, 7R allele frequencies have been shown to be higher in populations with more of a history of long-term migrations. It has also been shown that the 7R allele is associated with individuals having multiple ancestries. Here, we conduct a replication of this latter finding with two independent samples. Measures of subjects' ancestry are used to examine past reproductive bonds. The individuals' history of interracial/ancestral dating and their feelings about this are also assessed. Tentative support for an association between multiple ancestries and the 7R allele was found. These results are dependent upon the method of questioning subjects about their ancestries, with only finer-scale measures of ancestry being associated with 7R. Interracial dating and feelings about interracial pairing were not related to the presence of the 7R allele. This study provides continued support for a role for the 7R allele in migration and/or mate choice patterns. However, replications and extensions of this study are needed and the way ancestry/race is assessed must be carefully considered.

To test the relationship between androgen exposure, dopaminergic reward and sensation-seeking, we compared variation in salivary testosterone (T), 2D:4D digit ratio, facial masculinity, Zuckerman's sensation-seeking scale (SSS) and the D4 dopamine receptor (DRD4) genes from 98 young men, between the ages of 18 and 23 years. In univariate analyses, both salivary T and facial masculinity were significantly correlated with the SSS boredom susceptibility subscale, while the presence of the 7-repeat allele (7R+) in the dopamine receptor D4 gene was associated with the SSS thrill and adventure-seeking and overall sensation-seeking. Neither left nor right 2D:4D digit ratio was associated with any sensation-seeking scale. In multivariate models, salivary T and facial masculinity were significant predictors of SSS boredom susceptibility, while 7R+ was a significant predictor of SSS thrill and adventure-seeking. For overall SSS, both 7R+ and salivary T were significant predictors. There was no significant interaction of 7R+ and androgen exposure for SSS or any of the SSS subscales. These results add to earlier reports of an association between T and sensation-seeking. In addition, our results also indicate that genetic variation in DRD4 is independently associated with SSS sensation-seeking., ((c) 2010 Elsevier B.V. All rights reserved.)

The public goods game is the classic laboratory paradigm for studying collective action problems. Each participant chooses how much to contribute to a common pool that returns benefits to all participants equally. The ideal outcome occurs if everybody contributes the maximum amount, but the self-interested strategy is not to contribute anything. Most previous studies have found punishment to be more effective than reward for maintaining cooperation in public goods games. The typical design of these studies, however, represses future consequences for today's actions. In an experimental setting, we compare public goods games followed by punishment, reward, or both in the setting of truly repeated games, in which player identities persist from round to round. We show that reward is as effective as punishment for maintaining public cooperation and leads to higher total earnings. Moreover, when both options are available, reward leads to increased contributions and payoff, whereas punishment has no effect on contributions and leads to lower payoff. We conclude that reward outperforms punishment in repeated public goods games and that human cooperation in such repeated settings is best supported by positive interactions with others.

How do people respond to others' accidental behaviors? Reward and punishment for an accident might depend on the actor's intentions, or instead on the unintended outcomes she brings about. Yet, existing paradigms in experimental economics do not include the possibility of accidental monetary allocations. We explore the balance of outcomes and intentions in a two-player economic game where monetary allocations are made with a "trembling hand": that is, intentions and outcomes are sometimes mismatched. Player 1 allocates $10 between herself and Player 2 by rolling one of three dice. One die has a high probability of a selfish outcome, another has a high probability of a fair outcome, and the third has a high probability of a generous outcome. Based on Player 1's choice of die, Player 2 can infer her intentions. However, any of the three die can yield any of the three possible outcomes. Player 2 is given the opportunity to respond to Player 1's allocation by adding to or subtracting from Player 1's payoff. We find that Player 2's responses are influenced substantially by the accidental outcome of Player 1's roll of the die. Comparison to control conditions suggests that in contexts where the allocation is at least partially under the control of Player 1, Player 2 will punish Player 1 accountable for unintentional negative outcomes. In addition, Player 2's responses are influenced by Player 1's intention. However, Player 2 tends to modulate his responses substantially more for selfish intentions than for generous intentions. This novel economic game provides new insight into the psychological mechanisms underlying social preferences for fairness and retribution.

People often favor members of their own group, while discriminating against members of other groups. Such in-group favoritism has been shown to play an important role in human cooperation. However, in the face of changing conflicts and shifting alliances, it is essential for group identities to be flexible. Using the dictator game from behavioral economics, we demonstrate the remodeling of group identities among supporters of Democratic presidential candidates Barack Obama and Hillary Clinton. After Clinton's concession in June 2008, Democrats were more generous toward supporters of their own preferred candidate than to supporters of the other Democratic candidate. The bias observed in June persisted into August, and disappeared only in early September after the Democratic National Convention. We also observe a strong gender effect, with bias both appearing and subsiding among men only. This experimental study illustrates a dynamic change in bias, tracking the realignment of real world conflict lines and public efforts to reconstitute group identity. The change in salient group identity we describe here likely contributed to the victory of Barack Obama in the 2008 presidential election.

Background: In a recent controversial essay, published by JPA Ioannidis in PLoS Medicine, it has been argued that in some research fields, most of the published findings are false. Based on theoretical reasoning it can be shown that small effect sizes, error-prone tests, low priors of the tested hypotheses and biases in the evaluation and publication of research findings increase the fraction of false positives. These findings raise concerns about the reliability of research. However, they are based on a very simple scenario of scientific research, where single tests are used to evaluate independent hypotheses., Methodology/principal Findings: In this study, we present computer simulations and experimental approaches for analyzing more realistic scenarios. In these scenarios, research tasks are solved sequentially, i.e. subsequent tests can be chosen depending on previous results. We investigate simple sequential testing and scenarios where only a selected subset of results can be published and used for future rounds of test choice. Results from computer simulations indicate that for the tasks analyzed in this study, the fraction of false among the positive findings declines over several rounds of testing if the most informative tests are performed. Our experiments show that human subjects frequently perform the most informative tests, leading to a decline of false positives as expected from the simulations., Conclusions/significance: For the research tasks studied here, findings tend to become more reliable over time. We also find that the performance in those experimental settings where not all performed tests could be published turned out to be surprisingly inefficient. Our results may help optimize existing procedures used in the practice of scientific research and provide guidance for the development of novel forms of scholarly communication.

A key aspect of human behaviour is cooperation. We tend to help others even if costs are involved. We are more likely to help when the costs are small and the benefits for the other person significant. Cooperation leads to a tension between what is best for the individual and what is best for the group. A group does better if everyone cooperates, but each individual is tempted to defect. Recently there has been much interest in exploring the effect of costly punishment on human cooperation. Costly punishment means paying a cost for another individual to incur a cost. It has been suggested that costly punishment promotes cooperation even in non-repeated games and without any possibility of reputation effects. But most of our interactions are repeated and reputation is always at stake. Thus, if costly punishment is important in promoting cooperation, it must do so in a repeated setting. We have performed experiments in which, in each round of a repeated game, people choose between cooperation, defection and costly punishment. In control experiments, people could only cooperate or defect. Here we show that the option of costly punishment increases the amount of cooperation but not the average payoff of the group. Furthermore, there is a strong negative correlation between total payoff and use of costly punishment. Those people who gain the highest total payoff tend not to use costly punishment: winners don't punish. This suggests that costly punishment behaviour is maladaptive in cooperation games and might have evolved for other reasons.