Search

Academics are increasingly calling for and asked to, work in interdisciplinary teams to address pressing social-ecological challenges. However, there are significant barriers to pursuing interdisciplinary collaborations within current university structures. Taking the first two years of our Centre for Unusual Collaborations (CUCo) as a case study of setting up a space for exploration and experimentation, we discuss how unusual interdisciplinary collaborations had unexpected effects, beyond the potential for societal impact. Most surprisingly, we found the CUCo model offered a welcome opportunity to break away from the productivism and competition that is common in academia while stimulating exploration of our own disciplines. This often led to an expansion of ideas and deepened understanding in ways that sparked joy, curiosity, creativity and meaning. We discuss how academic culture currently hampers collaboration: key roadblocks are identified, specifically rewards and recognition, the lack of spaces for trust-building, and competence and skills that are not geared towards collaboration. We present lessons learned in overcoming roadblocks to stimulate research across disciplinary lines and explain how unusual interdisciplinary collaborations provide opportunities for opening and deepening research lines, and how they can be fun and meaningful. We argue that, at a moment when academia faces growing rates of burnout and stress, such collaborations are fundamental.

Academics are increasingly calling for and asked to, work in interdisciplinary teams to address pressing social-ecological challenges. However, there are significant barriers to pursuing interdisciplinary collaborations within current university structures. Taking the first two years of our Centre for Unusual Collaborations (CUCo) as a case study of setting up a space for exploration and experimentation, we discuss how unusual interdisciplinary collaborations had unexpected effects, beyond the potential for societal impact. Most surprisingly, we found the CUCo model offered a welcome opportunity to break away from the productivism and competition that is common in academia while stimulating exploration of our own disciplines. This often led to an expansion of ideas and deepened understanding in ways that sparked joy, curiosity, creativity and meaning. We discuss how academic culture currently hampers collaboration: key roadblocks are identified, specifically rewards and recognition, the lack of spaces for trust-building, and competence and skills that are not geared towards collaboration. We present lessons learned in overcoming roadblocks to stimulate research across disciplinary lines and explain how unusual interdisciplinary collaborations provide opportunities for opening and deepening research lines, and how they can be fun and meaningful. We argue that, at a moment when academia faces growing rates of burnout and stress, such collaborations are fundamental.

A preregistered meta-analysis, including 244 effect sizes from 85 field audits and 361,645 individual job applications, tested for gender bias in hiring practices in female-stereotypical and gender-balanced as well as male-stereotypical jobs from 1976 to 2020. A “red team” of independent experts was recruited to increase the rigor and robustness of our meta-analytic approach. A forecasting survey further examined whether laypeople (n = 499 nationally representative adults) and scientists (n = 312) could predict the results. Forecasters correctly anticipated reductions in discrimination against female candidates over time. However, both scientists and laypeople overestimated the continuation of bias against female candidates. Instead, selection bias in favor of male over female candidates was eliminated and, if anything, slightly reversed in sign starting in 2009 for mixed-gender and male-stereotypical jobs in our sample. Forecasters further failed to anticipate that discrimination against male candidates for stereotypically female jobs would remain stable across the decades.

Robust scientific knowledge is contingent upon replication of original findings. However, replicating researchers are constrained by resources, and will almost always have to choose one replication effort to focus on from a set of potential candidates. To select a candidate efficiently in these cases, we need methods for deciding which out of all candidates considered would be the most useful to replicate, given some overall goal researchers wish to achieve. In this article we assume that the overall goal researchers wish to achieve is to maximize the utility gained by conducting the replication study. We then propose a general rule for study selection in replication research based on the replication value of the set of claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by conducting a replication of the claim, and is a function of (a) the value of being certain about the claim, and (b) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value. Our goal in this article is not to define concrete guidelines for study selection, but to provide the necessary theoretical foundations on which such concrete guidelines could be built. Translational Abstract Replication—redoing a study using the same procedures—is an important part of checking the robustness of claims in the psychological literature. The practice of replicating original studies has been woefully devalued for many years, but this is now changing. Recent calls for improving the quality of research in psychology has generated a surge of interest in funding, conducting, and publishing replication studies. Because many studies have never been

A preregistered meta-analysis, including 244 effect sizes from 85 field audits and 361,645 individual job applications, tested for gender bias in hiring practices in female-stereotypical and gender-balanced as well as male-stereotypical jobs from 1976 to 2020. A “red team” of independent experts was recruited to increase the rigor and robustness of our meta-analytic approach. A forecasting survey further examined whether laypeople (n = 499 nationally representative adults) and scientists (n = 312) could predict the results. Forecasters correctly anticipated reductions in discrimination against female candidates over time. However, both scientists and laypeople overestimated the continuation of bias against female candidates. Instead, selection bias in favor of male over female candidates was eliminated and, if anything, slightly reversed in sign starting in 2009 for mixed-gender and male-stereotypical jobs in our sample. Forecasters further failed to anticipate that discrimination against male candidates for stereotypically female jobs would remain stable across the decades.

Item does not contain fulltext, To help move researchers away from heuristically dismissing "small" effects as unimportant, recent articles have revisited arguments to defend why seemingly small effect sizes in psychological science matter. One argument is based on the idea that an observed effect size may increase in impact when generalized to a new context because of processes of accumulation over time or application to large populations. However, the field is now in danger of heuristically accepting all effects as potentially important. We aim to encourage researchers to think thoroughly about the various mechanisms that may both amplify and counteract the importance of an observed effect size. Researchers should draw on the multiple amplifying and counteracting mechanisms that are likely to simultaneously apply to the effect when that effect is being generalized to a new and likely more dynamic context. In this way, researchers should aim to transparently provide verifiable lines of reasoning to justify their claims about an effect's importance or unimportance. This transparency can help move psychological science toward a more rigorous assessment of when psychological findings matter for the contexts that researchers want to generalize to.

Contains fulltext : 271043.pdf (Publisher’s version ) (Open Access), In the January 2022 issue of Perspectives, Götz et al. argued that small effects are "the indispensable foundation for a cumulative psychological science." They supported their argument by claiming that (a) psychology, like genetics, consists of complex phenomena explained by additive small effects; (b) psychological-research culture rewards large effects, which means small effects are being ignored; and (c) small effects become meaningful at scale and over time. We rebut these claims with three objections: First, the analogy between genetics and psychology is misleading; second, p values are the main currency for publication in psychology, meaning that any biases in the literature are (currently) caused by pressure to publish statistically significant results and not large effects; and third, claims regarding small effects as important and consequential must be supported by empirical evidence or, at least, a falsifiable line of reasoning. If accepted uncritically, we believe the arguments of Götz et al. could be used as a blanket justification for the importance of any and all "small" effects, thereby undermining best practices in effect-size interpretation. We end with guidance on evaluating effect sizes in relative, not absolute, terms.

31 mei 2022, Item does not contain fulltext, Psychological theories of habit posit that when a strong habit is formed through behavioral repetition, it can trigger behavior automatically in the same environment. Given the reciprocal relationship between habit and behavior, changing lifestyle behaviors is largely a task of breaking old habits and creating new and healthy ones. Thus, representing users’ habit strengths can be very useful for behavior change support systems, for example, to predict behavior or to decide when an intervention reaches its intended effect. However, habit strength is not directly observable and existing self-report measures are taxing for users. In this paper, building on recent computational models of habit formation, we propose a method to enable intelligent systems to compute habit strength based on observable behavior. The hypothesized advantage of using computed habit strength for behavior prediction was tested using data from two intervention studies on dental behavior change (N=36 and N=75), where we instructed participants to brush their teeth twice a day for three weeks and monitored their behaviors using accelerometers. The results showed that for the task of predicting future brushing behavior, the theory-based model that computed habit strength achieved an accuracy of 68.6% (Study 1) and 76.1% (Study 2), which outperformed the model that relied on self-reported behavioral determinants but showed no advantage over models that relied on past behavior. We discuss the implications of our results for research on behavior change support systems and habit formation.

20 september 2022, Contains fulltext : 271043.pdf (Publisher’s version ) (Open Access), In the January 2022 issue of Perspectives, Götz et al. argued that small effects are "the indispensable foundation for a cumulative psychological science." They supported their argument by claiming that (a) psychology, like genetics, consists of complex phenomena explained by additive small effects; (b) psychological-research culture rewards large effects, which means small effects are being ignored; and (c) small effects become meaningful at scale and over time. We rebut these claims with three objections: First, the analogy between genetics and psychology is misleading; second, p values are the main currency for publication in psychology, meaning that any biases in the literature are (currently) caused by pressure to publish statistically significant results and not large effects; and third, claims regarding small effects as important and consequential must be supported by empirical evidence or, at least, a falsifiable line of reasoning. If accepted uncritically, we believe the arguments of Götz et al. could be used as a blanket justification for the importance of any and all "small" effects, thereby undermining best practices in effect-size interpretation. We end with guidance on evaluating effect sizes in relative, not absolute, terms.

Academic journals provide a key quality-control mechanism in science. Yet, information asymmetries and conflicts of interests incentivize scientists to deceive journals about the quality of their research. How can honesty be ensured, despite incentives for deception? Here, we address this question by applying the theory of honest signaling to the publication process. Our models demonstrate that several mechanisms can ensure honest journal submission, including differential benefits, differential costs, and costs to resubmitting rejected papers. Without submission costs, scientists benefit from submitting all papers to high-ranking journals, unless papers can only be submitted a limited number of times. Counterintuitively, our analysis implies that inefficiencies in academic publishing (e.g., arbitrary formatting requirements, long review times) can serve a function by disincentivizing scientists from submitting low-quality work to high-ranking journals. Our models provide simple, powerful tools for understanding how to promote honest paper submission in academic publishing.

Researchers often conclude an effect is absent when a null-hypothesis significance test yields a nonsignificant p value. However, it is neither logically nor statistically correct to conclude an effect is absent when a hypothesis test is not significant. We present two methods to evaluate the presence or absence of effects: Equivalence testing (based on frequentist statistics) and Bayes factors (based on Bayesian statistics). In four examples from the gerontology literature, we illustrate different ways to specify alternative models that can be used to reject the presence of a meaningful or predicted effect in hypothesis tests. We provide detailed explanations of how to calculate, report, and interpret Bayes factors and equivalence tests. We also discuss how to design informative studies that can provide support for a null model or for the absence of a meaningful effect. The conceptual differences between Bayes factors and equivalence tests are discussed, and we also note when and why they might lead to similar or different inferences in practice. It is important that researchers are able to falsify predictions or can quantify the support for predicted null effects. Bayes factors and equivalence tests provide useful statistical tools to improve inferences about null effects.

[No abstract available], Funding Agencies|Flinders University; RU, Radboud Universiteit; NIU, Northern Illinois University; University of Nottingham; Universität zu Köln; Charité – Universitätsmedizin Berlin; SLU, Saint Louis University; TU/e, Technische Universiteit Eindhoven; University of Leeds; IUPUI, Indiana University-Purdue University Indianapolis; UiT, Universitetet i Tromsø; EUR, Erasmus Universiteit Rotterdam; UvT, Universiteit van Tilburg; University of Oxford; UNIL, Université de Lausanne; SEBM, Society for Experimental Biology and Medicine; NYU, New York University; UiT, Universitetet i Tromsø; RWTH Aachen University; KU Leuven; Universität Bielefeld; University of Louisiana at Lafayette; UA, University of Arkansas; Genentech Foundation for Biomedical Sciences; University of Bristol; UV, University of Virginia; City of Mobile; Université de Fribourg; Universiteit Gent; Yale University; VT, Virginia Polytechnic Institute and State University; DIBS, Duke Institute for Brain Sciences, Duke University; UCB UK; University of Leeds; BIBS, Brown Institute for Brain Science; UU, Universiteit Utrecht; Keele University; HRD, Division of Human Resource Development; WVU, West Virginia University; King’s College London; NTU, Nottingham Trent University; Rijksuniversiteit Groningen; KACST, King Abdulaziz City for Science and Technology; University of Edinburgh; UCONN, University of Connecticut; University of Kent; University of Cambridge; UNLV, University of Nevada, Las Vegas; Universiteit Leiden; UBC, University of British Columbia; HGS, Human Genome Sciences; AV CR, Akademie Ved Ceské Republiky; IUPUI, Indiana University-Purdue University Indianapolis; UK, University of Kentucky; University of Texas at Austin; DRS, Dahlem Research School, Freie Universität Berlin; Bangor University; RAND Corporation; University of Glasgow; University of Chester; CONICET, Consejo Nacional de Investigaciones Científicas y Técnicas; Massey University; UC, University of California; NIH, National Institutes

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance (p <.05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion (p <.0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely highpowered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than.20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above.10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and

Although researchers have repeatedly shown that the meaning of the same concept can vary across different contexts, it has proven difficult to predict when people will assign which meaning to a concept, and which associations will be activated by a concept. Building on the affective theory of meaning (Osgood, Suci, & Tannenbaum, 1957) and the polarity correspondence principle (Proctor & Cho, 2006), we propose the dimension-specificity hypothesis with the aim to understand and predict the context-dependency of cross-modal associations. We present three sets of experiments in which we use the dimension-specificity hypothesis to predict the cross-modal associations that should emerge between aggression-related concepts and saturation and brightness. The dimension-specificity hypothesis predicts that cross-modal associations emerge depending upon which affective dimension of meaning (i.e., the evaluation, activity, or potency dimension) is most salient in a specific context. The salience of dimensions of meaning is assumed to depend upon the relative conceptual distances between bipolar opposed concept pairs (e.g., good vs. bad). The dimension-specificity hypothesis proposes that plus and minus polarities will be attributed to the bipolar concepts, and associations between concrete and affective abstract concepts that share plus or minus polarities will become activated. Our data support the emergence of dimension-specific polarity attributions. We discuss the potential of dimension-specific polarity attributions to understand and predict how the context influences the emergence of cross-modal associations.

Introduction: Attitudes and habit strength are two important determinants of behavior, and from a dual-processing perspective they represent two distinct pathways of behavioral control. Since attitudes are based on beliefs, they influence behavior through intentions and decisions. In contrast, when defined as mental associations between cues and behaviors, habits are said to automatically trigger behavior. Despite their theoretical importance, studies that examine the joint influence of attitudes and habit strength on behavior are rare and two important questions warrant more research. Firstly, will attitudes cease to influence behavior when a habit is formed? This is plausible based on dual-process theories, but has not been empirically tested. Secondly, how does repeatedly performing a behavior influence attitudes and habit strength? Examining the reciprocal relationship is important for a mechanistic understanding of behavior change. We examine these two questions in the context of dental behavior. Method: Forty young adults participated in a 4-week field study in which they were persuaded to brushing teeth twice rather than once a day. In the first two weeks, a mobile app was used to remind them of brushing their teeth. Toothbrushing behavior was tracked by attaching movement sensors to participants’ toothbrushes in the first three weeks. Attitudes and habit strength were self-reported in five short surveys (at the baseline and also weekly). Multilevel models were built to test the hypotheses. Results: Participants performed the targeted behavior 46% of the time, and 14 participants maintained the new habit till the end. Overall, weekly behavior frequency of targeted brushing was significantly influenced by attitude (B = 0.08, p = .001), but not by habit strength (B = 0.03, p = .14). There was no evidence for the moderating role of habit strength on the attitude-behavior link, as indicated by the absence of an interaction effect (B = 0.003, p = .81). When weekly

The debate about whether replication studies should become mainstream is essentially driven by disagreements about their costs and benefits and the best ways to allocate limited resources. Determining when replications are worthwhile requires quantifying their expected utility. We argue that a formalized framework for such evaluations can be useful for both individual decision-making and collective discussions about replication.

Psychologists must be able to test both for the presence of an effect and for the absence of an effect. In addition to testing against zero, researchers can use the Two One-Sided Tests (TOST) procedure to test for equivalence and reject the presence of a smallest effect size of interest (SESOI). TOST can be used to determine if an observed effect is surprisingly small, given that a true effect at least as large as the SESOI exists. We explain a range of approaches to determine the SESOI in psychological science, and provide detailed examples of how equivalence tests should be performed and reported. Equivalence tests are an important extension of statistical tools psychologists currently use, and enable researchers to falsify predictions about the presence, and declare the absence, of meaningful effects.

Although context effects have repeatedly been demonstrated, it remains difficult to predict how context features influence the associative meaning of concepts. In a recent series of Experiments (see Part 1, Schietecat, Lakens, IJsselsteijn, & de Kort, 2018), we proposed and tested the dimension-specificity hypothesis for understanding and predicting context-dependent cross-modal associations between saturation, brightness, and aggression. In the current manuscript, Part 2, we aim to further test the dimension-specificity hypothesis by predicting the context-dependency of the meaning of the color red observed in the literature. The results of a series of five experiments revealed that the associations between red and valence could be predicted based on the activated dimensions of meaning (i.e., the evaluation or activity dimension) through the opposing concepts in the task. In the context of green, red was associated with negativity, whereas in context of blue, red was mostly associated with activity. Therefore, red was related to both aggression (a negative concept) and enthusiasm (a positive concept), depending on the context.

Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or “too good to be true”) that a set of studies yields exclusively significant results. Here, we use likelihood ratios to explain when sets of studies that contain a mix of significant and nonsignificant results are likely to be true or “too true to be bad.” As we show, mixed results are not only likely to be observed in lines of research but also, when observed, often provide evidence for the alternative hypothesis, given reasonable levels of statistical power and an adequately controlled low Type 1 error rate. Researchers should feel comfortable submitting such lines of research with an internal meta-analysis for publication. A better understanding of probabilities, accompanied by more realistic expectations of what real sets of studies look like, might be an important step in mitigating publication bias in the scientific literature.

When comparing two independent groups, psychology researchers commonly use Student's t-Tests. Assumptions of normality and homogeneity of variance underlie this test. More often than not, when these conditions are not met, Student's t-Test can be severely biased and lead to invalid statistical inferences. Moreover, we argue that the assumption of equal variances will seldom hold in psychological research, and choosing between Student's t-Test and Welch's t-Test based on the outcomes of a test of the equality of variances often fails to provide an appropriate answer. We show that the Welch's t-Test provides a better control of Type 1 error rates when the assumption of homogeneity of variance is not met, and it loses little robustness compared to Student's t-Test when the assumptions are met. We argue that Welch's t-Test should be used as a default strategy.

Processing action-language affects the planning and execution of motor acts, which suggests that the motor system might be involved in action-language understanding. However, this claim is hotly debated. For the first time, we compared the processing of action-verbs in children with Developmental Coordination Disorder (DCD), a disease that specifically affects the motor system, with children with a typical development (TD). We administered two versions of a go/no-go task in which verbs expressing either hand, foot or abstract actions were presented. We found that only when the semantic content of a verb has to be retrieved, TD children showed an increase in reaction times if the verb involved the same effector used to give the response. In contrast, DCD patients did not show any difference between verb categories irrespective of the task. These findings suggest that the pathological functioning of the motor system in individuals with DCD also affects language processing.

Scientists should be able to provide support for the absence of a meaningful effect. Currently, researchers often incorrectly conclude an effect is absent based a nonsignificant result. A widely recommended approach within a frequentist framework is to test for equivalence. In equivalence tests, such as the two one-sided tests (TOST) procedure discussed in this article, an upper and lower equivalence bound is specified based on the smallest effect size of interest. The TOST procedure can be used to statistically reject the presence of effects large enough to be considered worthwhile. This practical primer with accompanying spreadsheet and R package enables psychologists to easily perform equivalence tests (and power analyses) by setting equivalence bounds based on standardized effect sizes and provides recommendations to prespecify equivalence bounds. Extending your statistical tool kit with equivalence tests is an easy way to improve your statistical and theoretical inferences.

Ubicomp/ISWC'16 Adjunct, 12 september 2016, Contains fulltext : 159687.pdf (publisher's version ) (Open Access), Mobile computing systems hold the promise of becoming a cost-effective solution for supporting behavior change towards more healthy lifestyles. We present here an approach where the system implements a formal model of habit formation based on psychology theories, anticipates the behaviors and cognitive states of the users, and picks interventions based on model predictions. First, we discuss the motivation and system requirements for the approach. Next, we propose in detail an underlying computational model of habit formation which constitutes the key component of the system. Finally, future work and challenges will be discussed, focusing on the empirical validation of the model and the mapping between the model and intervention techniques

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle.

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle.

Discussion The present article highlights the need to improve the reproducibility of meta-analyses to facilitate the identification of errors, allow researchers to examine the impact of subjective choices such as inclusion criteria, and update the meta-analysis after several years. Reproducibility can be improved by applying standardized reporting guidelines and sharing all meta-analytic data underlying the meta-analysis, including quotes from articles to specify how effect sizes were calculated. Pre-registration of the research protocol (which can be peer-reviewed using novel ‘registered report’ formats) can be used to distinguish a-priori analysis plans from data-driven choices, and reduce the amount of criticism after the results are known. Summary The recommendations put forward in this article aim to improve the reproducibility of meta-analyses. In addition, they have the benefit of “future-proofing” meta-analyses by allowing the shared data to be re-analyzed as new theoretical viewpoints emerge or as novel statistical techniques are developed. Adoption of these practices will lead to increased credibility of meta-analytic conclusions, and facilitate cumulative scientific knowledge.