Search

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian evidence synthesis, Bayesian variable selection and model averaging, and Bayesian evaluation of cognitive models. We elaborate what each application entails, give illustrative examples, and provide an overview of key references and software with links to other applications. The article is concluded with a discussion of the opportunities and pitfalls of Bayes factor applications and a sketch of corresponding future research lines. The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The Bayes factor provides a method for quantifying the relative evidence for two competing hypotheses that are both instantiated by specific statistical models with prior distributions on the parameters. This general approach can be used to address many specific, theoretically relevant research questions. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: whether a randomized experiment has an effect or not (point null hypothesis), whether an effect is inside or outside a range of negligible effect sizes (interval hypothesis), whether a set of means follows a specific order (informative hypothesis), whether a set of studies jointly corroborate a theoretical claim (evidence synthesis), which variables are most relevant for prediction (variable selection), and which model provides the best account of latent processes (cognitive modeling). We elaborate what each application entails, give illustrative examples with reproducible files for the software R and JASP, and provide an overview of key references and software with links to other applications. We concluded with a discussion of the opportunities and pitfalls of the Bayes factor. [ABSTRACT FROM AUTHOR]

Auditors often have prior information about the auditee before starting the substantive testing phase. We show that applying Bayesian statistics in substantive testing allows for integration of this information into the statistical analysis through the prior distribution. For example, an auditor might have performed an audit last year, they might have information on certain controls in place, or they might have performed analytical procedures in an earlier stage of the audit. Incorporating this information directly in the statistical procedure enables auditors to tailor their sampling plan to the auditee, thereby increasing audit transparency and efficiency. However, defining a suitable prior distribution can be difficult because what constitutes a suitable prior depends on the specifics of the audit and the auditee. To help the auditor in constructing a prior distribution we introduce five methodologies, consider their pros and cons, and give examples of how to apply them in practice. [ABSTRACT FROM AUTHOR]

We outline a network method to synthesize a literature overview from search results obtained by multiple team members. Several network statistics are used to create a single representativeness ranking. We illustrate the method with the dispersed literature on a common misinterpretation of analysis of covariance (ANCOVA). The network method yields a top ten list of the most relevant articles that students and researchers can take as a point of departure for a more detailed study on this topic. The proposed methodology is implemented in Shiny, an open-source R package. [ABSTRACT FROM AUTHOR]

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer five original research questions related to moral judgments, negotiations, and implicit cognition. Participants from 2 separate large samples (total N > 15,000) were then randomly assigned to complete 1 version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: Materials from different teams rendered statistically significant effects in opposite directions for 4 of 5 hypotheses, with the narrowest range in estimates being d = −0.37 to + 0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for 2 hypotheses and a lack of support for 3 hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, whereas considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim. Public Significance Statement: Research in the social sciences often has implications for public policies as well as individual decisions—for good reason, the robustness of research findings is therefore of widespread interest both inside and outside academia. Yet, even findings that directly replicate—emerging again when the same methodology is repeated—may not always prove conceptually robust to different methodological approaches. The present initiative suggests that crowdsourcing study designs using many research teams can help reveal the conceptual robustness of the effects, better informing the public about the state of the empirical evidence. [ABSTRACT FROM AUTHOR]