Search

Missing data are ubiquitous in psychological research. They may come about as an unwanted result of coding or computer error, participants' non-response or absence, or missing values may be intentional, as in planned missing designs. We discuss the effects of missing data on χ²-based goodness-of-fit indices in Structural Equation Modeling (SEM), specifically on the Root Mean Squared Error of Approximation (RMSEA). We use simulations to show that naive implementations of the RMSEA have a downward bias in the presence of missing data and, thus, overestimate model goodness-of-fit. Unfortunately, many state-of-the-art software packages report the biased form of RMSEA. As a consequence, the scientific community may have been accepting a much larger fraction of models with non-acceptable model fit. We propose a bias-correction for the RMSEA based on information-theoretic considerations that take into account the expected misfit of a person with fully observed data. The corrected RMSEA is asymptotically independent of the proportion of missing data for misspecified models. Importantly, results of the corrected RMSEA computation are identical to naive RMSEA if there are no missing data.

Ordinal variables, while extremely common in Psychology, are almost exclusively analysed with statistical models that falsely assume them to be metric. This practice can lead to distorted effect size estimates, inflated error rates, and other problems. We argue for the application of ordinal models that make appropriate assumptions about the variables under study. In this tutorial article, we first explain the three major ordinal model classes; the cumulative, sequential and adjacent category models. We then show how to fit ordinal models in a fully Bayesian framework with the R package brms, using data sets on stem cell opinions and marriage time courses. Two appendices provide detailed mathematical derivations of the models, and a third practical example that connects ordinal models to Signal Detection Theory with confidence rating data. Ordinal models provide better theoretical interpretation and numerical inference from ordinal data, and we recommend their widespread adoption in Psychology.

[Correction Notice: An Erratum for this article was reported in Vol 115(6) of Journal of Personality and Social Psychology (see record 2018-63189-002). In the article, the original supplemental material has been revised to include a clarifying note to the Tests of model fit over larger sample (Samples C-G) section and post-peer review analyses added to the Post-peer review Factor Analytic Tests section. All versions of this article have been corrected.] Philosophers and behavioral scientists refer to wisdom as unbiased reasoning that guides one toward a balance of interests and promotes a good life. However, major instruments developed to test wisdom appear biased, and it is unclear whether they capture balance-related tendencies. We examined whether shifting from global, de-contextualized reports to state-level reports about concrete situations provides a less biased method to assess wise reasoning (e.g., intellectual humility, recognition of uncertainty and change, consideration of the broader context at hand and perspectives of others, integration of these perspectives or compromise), which may be aligned with the notion of balancing interests. Results of a large-scale psychometric investigation (N = 4,463) revealed that the novel Situated WIse Reasoning Scale (SWIS) is reliable and appears independent of psychological biases (attribution bias, bias blind spot, self-deception, and impression management), whereas global wisdom reports are subject to such biases. Moreover, SWIS scores were positively related to indices of living well (e.g., adaptive emotion regulation, mindfulness), and balancing of cooperative and self-protective interests, goals (influence-vs.-adjustment), and causal inferences about conflict (attribution to the self-vs.-other party). In contrast, global wisdom reports were unrelated or negatively related to balance-related measures. Notably, people showed modest within-person consistency in wise reasoning across situations or over time, suggesting that a single-shot measurement may be insufficient for whole understanding of trait-level wisdom. We discuss theoretical and practical implications for research on wisdom, judgment and decision making, well-being, and prosociality. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

The prevalence of obesity has driven searches for cognitive or behavioural economic factors related to Body Mass Index (BMI). One compelling candidate is delay discounting: those who prefer smaller sooner rewards over larger but later rewards are hypothesised to have higher BMI. The findings in the literature are mixed however, with meta analyses suggesting only a very small correlation between discounting and BMI. Here we present novel empirical data (N=381) and Bayesian analyses which suggest no such relationship between discounting of either monetary or weight loss rewards and BMI. We also find evidence against our novel proposal that discounting moderates the rate of BMI gain over time. We also present our data in the context of a random effects Bayesian meta-analytical result which does suggest the presence of a small correlation overall. The strength of the correlation is so weak (2.25% shared variance) that its practical significance may be minor to non existent. However because we found decisive evidence for unaccounted for study-level variance, due to study heterogeneity, we argue that we should treat such meta-analytic correlations with extreme caution. While the relationship between discounting and health outcomes such as BMI remain theoretically appealing, our empirical and meta-analytic results suggest we should be cautious in inferring a correlational, let alone a causal, role for discounting processes in driving BMI or moderating BMI gain with age.

This article introduces the NetworkToolbox package for R. Network analysis offers an intuitive perspective on complex phenomena via models depicted by nodes (variables) and edges (correlations). The ability of networks to model complexity has made them the standard approach for modeling the intricate interactions in the brain. Similarly, networks have become an increasingly attractive model for studying the complexity of psychological and psychopathological phenomena. NetworkToolbox aims to provide psychological researchers with state-of-the-art methods and measures for estimating and analyzing brain, cognitive, and psychometric data. In this article, I introduce NetworkToolbox and provide a manual for applying the package’s functions to psychometric (personality) data.

Psychological data often violate the normality assumptions made by commonly used statistical methods. These violations are addressed in a variety of ways such as transformations or assuming the employed method is robust to violations. Here we argue that data transformations are unnecessary at best and severely misleading at worst. An alternative approach is to use a Bayesian model that permits skewness and other perturbations to classical assumptions (e.g., heteroskedasticity). Through simulation, we demonstrate that a Bayesian skew-normal model has optimal frequentist properties (i.e., "type 1" error, "power", unbiasedness) compared to normal-assumptive models with or without transformation. Furthermore, the Bayesian skew-normal model has greater predictive utility, as indicated by posterior predictive checking and approximate leave-one-out cross-validation. After an applied example, we discuss practical implications of our findings for psychological science in general, and specifically how Bayesian modeling can improve reproducibility in psychology.

Previous studies found that an automatic meaning-composition process affects the processing of morphologically complex words, and related this operation to conceptual combination. However, research on embodied cognition demonstrates that concepts are more than just lexical meanings, rather being also grounded in perceptual experience. Therefore, perception-based information should also be involved in mental operations on concepts, such as conceptual combination. Consequently, we should expect to find perceptual effects in the processing of morphologically complex words. In order to investigate this hypothesis, we present the first fully-implemented and data-driven model of perception-based (more specifically, vision-based) conceptual combination, and use the predictions of such a model to investigate processing times for compound words in four large-scale behavioral experiments employing three paradigms (naming, lexical decision, and timed sensibility judgments). We observe facilitatory effects of vision-based compositionality in all three paradigms, over and above a strong language-based (lexical and semantic) baseline, thus demonstrating for the first time perceptually grounded effects at the sub-lexical level. This suggests that perceptually-grounded information is not only utilized according to specific task demands but rather automatically activated when available.

Here, we take a computational approach to understand the mechanisms underlying face perception biases in depression. Thirty participants diagnosed with major depressive disorder and 30 healthy control participants took part in three studies involving recognition of identity and emotion in faces. We used signal detection theory to determine whether any perceptual biases exist in depression aside from decisional biases. We found lower sensitivity to happiness in general, and lower sensitivity to both happiness and sadness with ambiguous stimuli. Our use of highly-controlled face stimuli ensures that such asymmetry is truly perceptual in nature, rather than the result of studying expressions with inherently different discriminability. We found no systematic effect of depression on the perceptual interactions between face expression and identity. We also found that decisional strategies used in our task were different for people with depression and controls, but in a way that was highly specific to the stimulus set presented. We show through simulation that the observed perceptual effects, as well as other biases found in the literature, can be explained by a computational model in which channels encoding positive expressions are selectively suppressed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Mixed-effects models are becoming common in psychological science. Although they have many desirable features, there is still untapped potential. It is customary to view homogeneous variance as an assumption to satisfy. We argue to move beyond that perspective, and to view modeling within-person variance as an opportunity to gain a richer understanding of psychological processes. The technique to do so is based on the mixed-effects location scale model that can simultaneously estimate mixed-effects submodels to both the mean (location) and within-person variance (scale). We develop a framework that goes beyond assessing the submodels in isolation of one another and introduce a novel Bayesian hypothesis test for mean—variance correlations in the distribution of random effects. We first present a motivating example, which makes clear how the model can characterize mean—variance relations. We then apply the method to reaction times (RTs) gathered from 2 cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean—variance relations. This stands in contrast to the dominant view of within-person variance (i.e., “noise”). The results also point toward paradoxical within-person, as opposed to between-person, effects: several people had slower and less variable incongruent responses. This contradicts the typical pattern, wherein larger means tend to be associated with more variability. We conclude with future directions, spanning from methodological to theoretical inquires, that can be answered with the presented methodology.Impact StatementTranslational Abstract: Mixed-effects models are becoming common in psychological science. Although researchers typically focus on explaining the dependent variable, characterizing individual differences in within-person variance can provide valuable insights into behavioral consistency. The technique to do so is based on an extension to the mixed-effect model that can simultaneously predict both the mean and within-person variance. We introduce Bayesian methodology to test the covariance among individual difference parameters that capture mean and within-person variance relations. This is motivated by theoretical models that predict larger means will be associated with more variability. Our approach allows for confirmatory hypothesis testing of this proposed relation in the distribution of random effects. Hence, researchers can begin to test the interplay between an experimental effect on the mean and within-person variability in their repeated measurements. We apply our method to RTs gathered from two cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean-variance relations. We also uncovered strikingly large correlations between the experimental effect on the mean and within-person variability. Substantively this novel finding suggests that cognitive inhibition has important and highly interrelated signatures on both RT and RT consistency. The method is implemented in the user-friendly R package hypMuVar.

A wide variety of effect size indices have been proposed for quantifying the magnitude of treatment effects in single-case designs. Commonly used measures include parametric indices such as the standardized mean difference as well as nonoverlap measures such as the percentage of nonoverlapping data, improvement rate difference, and nonoverlap of all pairs. Currently, little is known about the properties of these indices when applied to behavioral data collected by systematic direct observation, even though systematic direct observation is the most common method for outcome measurement in single-case research. This study uses Monte Carlo simulation to investigate the properties of several widely used single-case effect size measures when applied to systematic direct observation data. Results indicate that the magnitude of the nonoverlap measures and of the standardized mean difference can be strongly influenced by procedural details of the study's design, which is a significant limitation to using these indices as effect sizes for meta-analysis of single-case designs. A less widely used parametric index, the log response ratio, has the advantage of being insensitive to sample size and observation session length, although its magnitude is influenced by the use of partial interval recording. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

State trace analyses assess the latent dimensionality of a cognitive process by asking whether the means of two dependent variables conform to a monotonic function across a set of conditions. Using an assumption of independence between the measures, recently proposed statistical tests address bivariate measurement error, allowing both frequentist and Bayesian analyses of monotonicity (e.g., Davis-Stober, Morey, Gretton, & Heathcote, 2016; Kalish, Dunn, Burdakov, & Sysoev, 2016). However, statistical inference can be biased by unacknowledged dependencies between measures, particularly when the data are insufficient to overwhelm an incorrect prior assumption of independence. To address this limitation, we developed a hierarchical Bayesian model that explicitly models the separate roles of subject, item, and trial-level dependencies between two measures. Assessment of monotonicity is then performed by fitting separate models that do or do not allow a non-monotonic relation between the condition effects (i.e., same versus different rank orders). The Widely Applicable Information Criterion (WAIC) and Pseudo Bayesian Model Averaging – both cross validation measures of model fit – are used for model comparison, providing an inferential conclusion regarding the dimensionality of the latent psychological space. We validated this new state trace analysis technique using model recovery simulation studies, which assumed different ground truths regarding monotonicity and the direction/magnitude of the subject- and trial-level dependence. We also provide an example application of this new technique to a visual object learning study that compared performance on a visual retrieval task (forced choice part recognition) versus a verbal retrieval task (cued recall).

Personality and demographics are important variables in social sciences, while in NLP they can aid in interpretability and removal of societal biases. However, datasets with both personality and demographic labels are scarce. To address this, we present PANDORA, the first large-scale dataset of Reddit comments labeled with three personality models (including the well-established Big 5 model) and demographics (age, gender, and location) for more than 10k users. We showcase the usefulness of this dataset on three experiments, where we leverage the more readily available data from other personality models to predict the Big 5 traits, analyze gender classification biases arising from psycho-demographic variables, and carry out a confirmatory and exploratory analysis based on psychological theories. Finally, we present benchmark prediction models for all personality and demographic variables., Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, NAACL 2021, https://www.aclweb.org/anthology/2021.socialnlp-1.12

Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.

A key assumption of models of human cognition is that there is variability in information processing. Evidence accumulation models (EAMs) commonly assume 2 broad variabilities in information processing: within-trial variability, which is thought to reflect moment-to-moment fluctuations in perceptual processes, and between-trial variability, which is thought to reflect variability in slower-changing processes like attention, or systematic variability between the stimuli on different trials. Recently, Ratcliff, Voskuilen, and McKoon (2018) claimed to "provide direct evidence that external noise is, in fact, required to explain the data from five simple two-choice decision tasks" (p. 33), suggesting that at least some portion of the between-trial variability in information processing is due to "noise." However, we argue that Ratcliff et al. (2018) failed to distinguish between 2 different potential sources of between-trial variability: random (i.e., "external noise") and systematic (e.g., item effects). Contrary to the claims of Ratcliff et al. (2018), we show that "external noise" is not required to explain their findings, as the same trends of data can be produced when only item effects are present. Furthermore, we contend that the concept of "noise" within cognitive models merely serves as a convenience parameter for sources of variability that we know exist but are unable to account for. Therefore, we question the usefulness of experiments aimed at testing the general existence of "random" variability and instead suggest that future research should attempt to replace the random variability terms within cognitive models with actual explanations of the process. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Human decisions often deviate from economic rationality and are influenced by cognitive biases. One such bias is the memory bias according to which people prefer choice options they have a better memory of—even when the options’ utilities are comparatively low. Although this phenomenon is well supported empirically, its cognitive foundation remains elusive. Here we test two conceivable computational accounts of the memory bias against each other. On the one hand, a single-process account explains the memory bias by assuming a single biased evidence-accumulation process in favor of remembered options. On the contrary, a dual-process account posits that some decisions are driven by a purely memory-driven process and others by a utility-maximizing one. We show that both accounts are indistinguishable based on choices alone as they make similar predictions with respect to the memory bias. However, they make qualitatively different predictions about response times. We tested the qualitative and quantitative predictions of both accounts on behavioral data from a memory-based decision-making task. Our results show that a single-process account provides a better account of the data, both qualitatively and quantitatively. In addition to deepening our understanding of memory-based decision-making, our study provides an example of how to rigorously compare single- versus dual-process models using empirical data and hierarchical Bayesian parameter estimation methods.

Multiple learning systems allow individuals to flexibly respond to opportunities and challenges present in the environment. An evolutionarily conserved “Pavlovian” learning mechanism couples valence and action, promoting a tendency to approach cues associated with reward and to inhibit action in the face of anticipated punishment. Although this default response system may be adaptive, these hard-wired reactions can hinder the ability to learn flexible “instrumental” actions in pursuit of a goal. Such constraints on behavioral flexibility have been studied extensively in adults. However, the extent to which these valence-specific response tendencies bias instrumental learning across development remains poorly characterized. Here, we show that while Pavlovian response biases constrain flexible action learning in children and adults, these biases are attenuated in adolescents. This adolescent-specific reduction in Pavlovian bias may promote unbiased exploration of approach and avoidance responses, facilitating the discovery of rewarding behavior in the many novel contexts that adolescents encounter.

Moral intuitions are a central motivator in human behavior. Recent work highlights the importance of moral intuitions for understanding a wide range of issues ranging from online radicalization to vaccine hesitancy. Extracting and analyzing moral content in messages, narratives, and other forms of public discourse is a critical step toward understanding how the psychological influence of moral judgments unfolds at a global scale. Extant approaches for extracting moral content are limited in their ability to capture the intuitive nature of moral sensibilities, constraining their usefulness for understanding and predicting human moral behavior. Here we introduce the extended Moral Foundations Dictionary (eMFD), a dictionary-based tool for extracting moral content from textual corpora. The eMFD, unlike previous methods, is constructed from text annotations generated by a large sample of human coders. We demonstrate that the eMFD outperforms existing approaches in a variety of domains. We anticipate that the eMFD will contribute to advance the study of moral intuitions and their influence on social, psychological, and communicative processes. All data and code for our analyses can be found here. We also created software -eMFDscore - for easy scoring of text via our eMFD. The software is available at https://github.com/medianeuroscience/emfdscore.

Generic language (e.g., “Birds fly”) conveys generalizations about categories and is essential for learning beyond our direct experience. The meaning of generic language is notoriously hard to specify, however (e.g., penguins don’t fly). Tessler and Goodman (2019) proposed a model for generics that is mathematically equivalent to Bayesian belief-updating based on a single pedagogical example, suggesting a deep connection be- tween learning from experience and learning from language. Relatedly, Csibra and Shamsudheen (2015) argue that generics are inherently pedagogical, understood by infants as referring to a member of a kind. In two experiments with adults, we quantify the exchange-rate between generics and observations by relating their belief-updating capacity, varying both the number of observations and whether they are presented pedagogically or incidentally. We find generics convey stronger generalizations than single pedagogical observations (Expt. 1), even when the property is explicitly demarcated (Expt. 2). We suggest revisions to the vague quantifier model of generics that would allow it to accommodate this intriguing exchange-rate.

The mind and consciousness are pretty complex to observe and analyze quantitatively since the complexity of neuron and mystery of consciousness. Recent years, many scientists try to explain our mind from physical viewpoint or elucidate the physics of the mind. In the paper, we reveal a new approach to understand our mind and present a tentative theory to describe how our mind works respect to physics. Two significant concept of time, namely, the internal time of mind and external time of the outside are introduced to measure our mind quantitatively on some degree. The related law of these two types of time is similar to the special relativity proposed by Einstein. Our theory aims to provide a new perspective to understand and know ourselves and lead our life better. At the end of paper, we design an experiment to check the correctness of our results.

Evidence accumulation models (EAMs) have become the dominant explanation of how the decision-making process operates, proposing that decisions are the result of a process of evidence accumulation. The primary use of EAMs has been as "measurement tools" of the underlying decision-making process, where researchers apply EAMs to empirical data to estimate participants' task ability (i.e., the "drift rate"), response caution (i.e., the "decision threshold"), and the time taken for other processes (i.e., the "non-decision time"), making EAMs a powerful tool for discriminating between competing psychological theories. Recent studies have brought into question the mapping between the latent parameters of EAMs and the theoretical constructs that they are thought to represent, showing that emphasizing urgent responding -- which intuitively should selectively influence decision threshold -- may also influence drift rate and/or non-decision time. However, these findings have been mixed, leading to differences in opinion between experts in the field. The current study aims to provide a more conclusive answer to the implications of emphasizing urgent responding, providing a re-analyse of 6 data sets from previous studies using two different EAMs -- the diffusion model and the linear ballistic accumulator (LBA) -- with state-of-the-art methods for model selection based inference. The findings display clear evidence for a difference in conclusions between the two models, with the diffusion model suggesting that decision threshold and non-decision time decrease when urgency is emphasized, and the LBA suggesting that decision threshold and drift rate decrease when urgency is emphasized. Furthermore, although these models disagree regarding whether non-decision time or drift rate decrease under urgency emphasis, both show clear evidence that emphasizing urgency does not selectively influence decision threshold. These findings suggest that researchers should revise their assumptions about certain experimental manipulations, the specification of certain EAMs, or perhaps both.

The analysis of longitudinal neuroimaging data within the massively univariate framework provides the opportunity to study empirical questions about neurodevelopment. Missing outcome data are an all-to-common feature of any longitudinal study, a feature that, if handled improperly, can reduce statistical power and lead to biased parameter estimates. The goal of this paper is to provide conceptual clarity of the issues and non-issues that arise from analyzing incomplete data in longitudinal studies with particular focus on neuroimaging data. This paper begins with a review of the hierarchy of missing data mechanisms and their relationship to likelihood-based methods, a review that is necessary not just for likelihood-based methods, but also for multiple-imputation methods. Next, the paper provides a series of simulation studies with designs common in longitudinal neuroimaging studies to help illustrate missing data concepts regardless of interpretation. Finally, two applied examples are used to demonstrate the sensitivity of inferences under different missing data assumptions and how this may change the substantive interpretation. The paper concludes with a set of guidelines for analyzing incomplete longitudinal data that can improve the validity of research findings in developmental neuroimaging research. Keywords: Neuroimaging, Missing data, Likelihood, Longitudinal data

An important tool in the advancement of cognitive science are quantitative models that represent different cognitive variables in terms of model parameters. To evaluate such models, their parameters are typically tested for relationships with behavioral and physiological variables that are thought to reflect specific cognitive processes. However, many models do not come equipped with the statistical framework needed to relate model parameters to covariates. Instead, researchers often revert to classifying participants into groups depending on their values on the covariates, and subsequently comparing the estimated model parameters between these groups. Here we develop a comprehensive solution to the covariate problem in the form of a Bayesian regression framework. Our framework can be easily added to existing cognitive models and allows researchers to quantify the evidential support for relationships between covariates and model parameters using Bayes factors. Moreover, we present a simulation study that demonstrates the superiority of the Bayesian regression framework to the conventional classification-based approach.

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

Evaluation of model fit is critically important for every structural equation model (SEM), and sophisticated methods have been developed for this task. Among them are the χ² goodness-of-fit test, decomposition of the χ², derived measures like the popular root mean square error of approximation (RMSEA) or comparative fit index (CFI), or inspection of residuals or modification indices. Many of these methods provide a global approach to model fit evaluation: A single index is computed that quantifies the fit of the entire SEM to the data. In contrast, graphical criteria like d-separation or trek-separation allow derivation of implications that can be used for local fit evaluation, an approach that is hardly ever applied. We provide an overview of local fit evaluation from the viewpoint of SEM practitioners. In the presence of model misfit, local fit evaluation can potentially help in pinpointing where the problem with the model lies. For models that do fit the data, local tests can identify the parts of the model that are corroborated by the data. Local tests can also be conducted before a model is fitted at all, and they can be used even for models that are globally underidentified. We discuss appropriate statistical local tests, and provide applied examples. We also present novel software in R that automates this type of local fit evaluation. (PsycINFO Database Record

According to Spearman’s law of diminishing returns (SLODR), IQ test scores are more g saturated among those with low, compared to those with high, ability. The present simulation shows that such difference in saturation can be observed if test scores are affected by a disturbing factor, for example, low motivation, illness, or linguistic confusion, that varies in magnitude between individuals. More contemporary criteria of SLODR can also be satisfied if test scores are affected by disturbance, especially if the disturbance variable is negatively skewed. This indicates a possible threat against the validity of findings supporting SLODR and points at the importance for researchers to try to eliminate the influence of such disturbing factors from their studies.

Understanding the temporal dynamics of affect is crucial for our understanding human emotions in general. In this study, we empirically test a computational model of affective dynamics by analyzing a large-scale dataset of Facebook status updates using text analysis techniques. Our analyses support the central assumptions of our model: After stimulation, affective states, quantified as valence and arousal, exponentially return to an individual-specific baseline. On average, this baseline is at a slightly positive valence value and at a moderate arousal point below the midpoint. Furthermore, affective expression, in this case posting a status update on Facebook, immediately pushes arousal and valence towards the baseline by a proportional value. These results are robust to the choice of the text analysis technique and illustrate the fast timescale of affective dynamics through social media text. These outcomes are of high relevance for affective computing, the detection and modeling of collective emotions, the refinement of psychological research methodology, and the detection of abnormal, and potentially pathological, individual affect dynamics.

The human affect system is responsible for producing the positive and negative feelings that color and guide our lives. At the same time, when disrupted, its workings lie at the basis of the occurrence of mood disorder. Understanding the functioning and dynamics of the affect system is therefore crucial to understand the feelings that people experience on a daily basis, their dynamics across time, and how they can become dysregulated in mood disorder. In this paper, a nonlinear stochastic model for the dynamics of positive and negative affect is proposed called the Affective Ising Model (AIM). It incorporates principles of statistical mechanics, is inspired by neurophysiological and behavioral evidence about auto-excitation and mutual inhibition of the positive and negative affect dimensions, and is intended to better explain empirical phenomena such as skewness, multimodality, and non-linear relations of positive and negative affect. The AIM is applied to two large experience sampling studies on the occurrence of positive and negative affect in daily life in both normality and mood disorder. It is examined to what extent the model is able to reproduce the aforementioned non-Gaussian features observed in the data, using two sightly different continuous-time vector autoregressive (VAR) models as benchmarks. The predictive performance of the models is also compared by means of leave-one-out cross-validation. The results indicate that the AIM is better at reproducing non-Gaussian features while their performance is comparable for strictly Gaussian features. The predictive performance of the AIM is also shown to be better for the majority of the affect time series. The potential and limitations of the AIM as a computational model approximating the workings of the human affect system are discussed., Author summary Feelings color and guide our lives. Understanding their dynamics is a crucial step on the way to eventually understanding mood disorders such as depression. In this paper, we propose a model for the dynamics of positive and negative affect, called the Affective Ising Model (AIM). Starting from a neurobiologically inspired yet abstract microscopic representation of how affect is generated, the model predicts the presence of a number of nonlinear phenomena in the dynamics of positive and negative affect. These nonlinear phenomena include skewed distributions, bimodality (people’s affect can fluctuate around one of two possible states) and a V-shaped relation between positive and negative affect. These nonlinear signature features have been empirically established, but have thus far not been integrated into a single computation model. The AIM can be used in the future to explain both normal and dysfunctional affect.

People express quantities using a remarkably small set of units – digits. Confusing digits could be costly, and not all confusions are equal; confusing a price tag of 2 dollars with 9 dollars is naturally more costly than confusing 2 with 3. Confusion patterns are intimately related to the distances between mental representations, which are hypothetical internal symbols said to stand for, or represent, ‘real’ external stimuli. The distance between the mental representations of two digits could be determined by their numerical distance. Alternatively, it could be driven by visual similarity. In an English speaking cohort, we investigated the mental representations of familiar and unfamiliar numbers (4 sets: Arabic, Chinese, Thai, and non-symbolic dots) through a set of identification experiments, using multi-dimensional scaling and cluster analysis. We controlled for undesired effects of response bias using Luce’s choice model. Our findings show Arabic, Chinese and Thai numerals were represented in the mental space by perceptual similarities. We also find non-symbolic dots were represented by perceptual and numerical similarities. This work is a novel contribution to the literature and lays the foundation for further investigations into the mental representation of numerals across cultures.

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

A key phenomenon in inductive reasoning is the diversity effect, whereby a novel property is more likely to be generalized when it is shared by an evidence sample composed of diverse instances than a sample composed of similar instances. We outline a Bayesian model and an experimental study that show that the diversity effect depends on the assumption that samples of evidence were selected by a helpful agent (strong sampling). Inductive arguments with premises containing either diverse or nondiverse evidence samples were presented under different sampling conditions, where instructions and filler items indicated that the samples were selected intentionally (strong sampling) or randomly (weak sampling). A robust diversity effect was found under strong sampling, but was attenuated under weak sampling. As predicted by our Bayesian model, the largest effect of sampling was on arguments with nondiverse evidence, where strong sampling led to more restricted generalization than weak sampling. These results show that the characteristics of evidence that are deemed relevant to an inductive reasoning problem depend on beliefs about how the evidence was generated. Electronic supplementary material The online version of this article (10.3758/s13423-018-1562-2) contains supplementary material, which is available to authorized users.

Neurocognitive tasks are frequently used to assess disordered decision making, and cognitive models of these tasks can quantify performance in terms related to decision makers' underlying cognitive processes. In many cases, multiple cognitive models purport to describe similar processes, but it is difficult to evaluate whether they measure the same latent traits or processes. In this article, we develop methods for modeling behavior across multiple tasks by connecting cognitive model parameters to common latent constructs. This approach can be used to assess whether 2 tasks measure the same dimensions of cognition, or actually improve the estimates of cognitive models when there are overlapping cognitive processes between 2 related tasks. The approach is then applied to connecting decision data on 2 behavioral tasks that evaluate clinically relevant deficits, the delay discounting task and Cambridge gambling task, to determine whether they both measure the same dimension of impulsivity. We find that the discounting rate parameters in the models of each task are not closely related, although substance users exhibit more impulsive behavior on both tasks. Instead, temporal discounting on the delay discounting task as quantified by the model is more closely related to externalizing psychopathology like aggression, while temporal discounting on the Cambridge gambling task is related more to response inhibition failures. The methods we develop thus provide a new way to connect behavior across tasks and grant new insights onto the different dimensions of impulsivity and their relation to substance use. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

In recent years, a growing chorus of researchers have argued that psychological theory is in a state of crisis: theories are rarely developed in a way that indicates an accumulation of knowledge and they are often absent from our research entirely. More than 40 years ago, Paul Meehl raised these very concerns. Yet, in the ensuing decades, little has improved. We aim to chart a better path forward for psychological theory by revisiting Meehl's criticisms, his proposed solution, and the reasons his solution failed to meaningful change the status of psychological theory. We argue that Meehl identified serious shortcomings in our evaluation of psychological theories and that his proposed solution would substantially strengthen theory testing. However, we also argue that he failed to provide researchers a set of tools for theory construction. To advance psychological theory, we must equip researchers with tools to better generate, evaluate, and develop their theories. We argue that formal theories provide this much needed set of tools, equipping researchers with tools for thinking, evaluating explanation, informing theory development, strengthening measurement, and moving toward collaborative construction of psychological theories that allow us to explain, predict, and control psychological phenomena.

Quasi-orders are reflexive and transitive binary relations and have many applications. Examples are the dependencies of mastery among the problems of a psychological test, or methods such as item tree or Boolean analysis that mine for quasi-orders in empirical data. Data mining techniques are typically tested based on simulation studies with unbiased samples of randomly generated quasi-orders. In this paper, we develop techniques for the approximately representative sampling of quasi-orders. Polynomial regression curves are fitted for the mean and standard deviation of quasi-order size as a function of item number. The resulting regression graphs are seen to be quadratic and linear functions, respectively. The extrapolated values for the mean and standard deviation are used to propose two quasi-order sampling techniques. The discrete method matches these location and scale measures with a transformed discrete distribution directly obtained from the sample. The continuous method uses the normal density function with matched expectation and variance. The quasi-orders are constructed according to the biased randomized doubly inductive construction, however they are resampled to become approximately representative following the matched discrete and continuous distributions. In simulations, we investigate the usefulness of these methods. The location-scale matching approach can cope with very large item sets. Close to representative samples of random quasi-orders are constructed for item numbers up to n = 400.

BayeFit is a module for Python that allows users to fit, plot, and extract parameter estimates from models of psychophysical data using Bayesian inference. The module makes extensive use of the module PyStan, which in turn makes extensive use of the package Stan, a powerful tool for Bayesian inference using a No-U-Turn sampler. Although PyStan provides an interface between Python and Stan, defining models for use with PyStan can be difficult for most users given the models are defined in C++ language. BayesFit removes this barrier to entry by providing researchers with functions that streamline the process of defining psychophysical models, obtaining fits, extracting outputs, and visualizing fitted models. The source code for BayesFit is available at https://github.com/slugocm/bayesfit. This module can be expanded and reused as newer versions of Python are developed to ensure researchers always have a tool available to ease the process of fitting models to psychophysical data.

Cue-based retrieval theories in sentence processing predict two classes of interference effect: (i) Inhibitory interference is predicted when multiple items match a retrieval cue: cue-overloading leads to a decrease in the probability of retrieving the correct target; and (ii) Facilitatory interference arises when a retrieval target as well as a distractor only partially match the retrieval cues; this partial matching leads to an overall speedup in retrieval time. Inhibitory interference effects are widely observed, but facilitatory interference apparently have an exception: reflexives have been claimed to show no facilitatory interference effects. Because the finding is based on underpowered studies, we conducted a large-sample experiment that investigated both facilitatory and inhibitory interference. In contrast to previous studies, we find facilitatory interference effects in reflexives. We also present a comprehensive quantitative evaluation of a cue-based retrieval model (Engelmann et al., 2019, available as a Shiny App: https://engelmann.shinyapps.io/inter-act/) with respect to the large-sample data.

In working memory research, individual items are sometimes said to be in the "focus of attention". According to one view, this occurs for the last item in a sequentially presented list (last-item benefit). According to a second view, this occurs when items are externally cued during the retention interval (retro-cue benefit). We investigated both phenomena at the same time to determine whether both result from the same cognitive mechanisms. If that were the case, retro-cue benefits should be reduced when the retro-cue is directed to the item that already benefits from being presented last. We measured speed-accuracy-tradeoff functions with the response-deadline paradigm to measure retrieval dynamics in a short-term recognition task. Across three experiments, we found that retro-cues benefited the last item and other items to the same extent. The additivity of the last-item benefit and the retro-cue benefit points towards the co-existence of at least two distinct forms of attentional prioritization in working memory.

With the development of increasingly automated vehicles (AVs) comes the increasingly difficult challenge of comprehensively validating these for acceptable, and ideally beneficial, impacts on the transport system. There is a growing consensus that virtual testing, where simulated AVs are deployed in simulated traffic, will be key for cost-effective testing and optimization. The least mature model components in such simulations are those generating the behavior of human agents in or around the AVs. In this paper, human models and virtual testing applications are presented for two example scenarios: (i) a human pedestrian deciding whether to cross a street in front of an approaching automated vehicle, with or without external human–machine interface elements, and (ii) an AV handing over control to a human driver in a critical rear-end situation. These scenarios have received much recent research attention, yet simulation-ready human behavior models are lacking. They are discussed here in the context of existing models of perceptual decision-making, situational awareness, and traffic interactions. It is argued that the human behavior in question might be usefully conceptualized as a number of interrelated decision processes, not all of which are necessarily directly associated with externally observable behavior. The results show that models based on this type of framework can reproduce qualitative patterns of behavior reported in the literature for the two addressed scenarios, and it is demonstrated how computer simulations based on the models, once these have been properly validated, could allow prediction and optimization of AV impacts on traffic flow and traffic safety.

Background Participant retention strategies that minimise attrition in longitudinal cohort studies have evolved considerably in recent years. This study aimed to assess, via systematic review and meta-analysis, the effectiveness of both traditional strategies and contemporary innovations for retention adopted by longitudinal cohort studies in the past decade. Methods Health research databases were searched for retention strategies used within longitudinal cohort studies published in the 10-years prior, with 143 eligible longitudinal cohort studies identified (141 articles; sample size range: 30 to 61,895). Details on retention strategies and rates, research designs, and participant demographics were extracted. Meta-analyses of retained proportions were performed to examine the association between cohort retention rate and individual and thematically grouped retention strategies. Results Results identified 95 retention strategies, broadly classed as either: barrier-reduction, community-building, follow-up/reminder, or tracing strategies. Forty-four of these strategies had not been identified in previous reviews. Meta-regressions indicated that studies using barrier-reduction strategies retained 10% more of their sample (95%CI [0.13 to 1.08]; p = .01); however, studies using follow-up/reminder strategies lost an additional 10% of their sample (95%CI [− 1.19 to − 0.21]; p = .02). The overall number of strategies employed was not associated with retention. Conclusions Employing a larger number of retention strategies may not be associated with improved retention in longitudinal cohort studies, contrary to earlier narrative reviews. Results suggest that strategies that aim to reduce participant burden (e.g., flexibility in data collection methods) might be most effective in maximising cohort retention. Electronic supplementary material The online version of this article (10.1186/s12874-018-0586-7) contains supplementary material, which is available to authorized users.

Cross-cultural analysis of song It is unclear whether there are universal patterns to music across cultures. Mehr et al. examined ethnographic data and observed music in every society sampled (see the Perspective by Fitch and Popescu). For songs specifically, three dimensions characterize more than 25% of the performances studied: formality of the performance, arousal level, and religiosity. There is more variation in musical behavior within societies than between societies, and societies show similar levels of within-society variation in musical behavior. At the same time, one-third of societies significantly differ from average for any given dimension, and half of all societies differ from average on at least one dimension, indicating variability across cultures. Science , this issue p. eaax0868 ; see also p. 944

Carrots and sticks motivate behavior, and people can teach new behaviors to other organisms, such as children or nonhuman animals, by tapping into their reward learning mechanisms. But how people teach with reward and punishment depends on their expectations about the learner. We examine how people teach using reward and punishment by contrasting two hypotheses. The first is evaluative feedback as reinforcement, where rewards and punishments are used to shape learner behavior through reinforcement learning mechanisms. The second is evaluative feedback as communication, where rewards and punishments are used to signal target behavior to a learning agent reasoning about a teacher's pedagogical goals. We present formalizations of learning from these 2 teaching strategies based on computational frameworks for reinforcement learning. Our analysis based on these models motivates a simple interactive teaching paradigm that distinguishes between the two teaching hypotheses. Across 3 sets of experiments, we find that people are strongly biased to use evaluative feedback communicatively rather than as reinforcement. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

We present a mixed-effects location scale model (MELSM) for examining the daily dynamics of affect in dyads. The MELSM includes person and time-varying variables to predict the location, or individual means, and the scale, or within-person variances. It also incorporates a submodel to account for between-person variances. The dyadic specification can accommodate individual and partner effects in both the location and the scale components, and allows random effects for all location and scale parameters. All covariances among the random effects, within and across the location and the scale are also estimated. These covariances offer new insights into the interplay of individual mean structures, intra-individual variability, and the influence of partner effects on such factors. To illustrate the model, we use data from 274 couples who provided daily ratings on their positive and negative emotions toward their relationship - up to 90 consecutive days. The model is fit using Hamiltonian Monte Carlo methods, and includes subsets of predictors in order to demonstrate the flexibility of this approach. We conclude with a discussion on the usefulness and the limitations of the MELSM for dyadic research.

The same persuasive message can be interpreted in a positive or negative way, challenging our ability to predict its effectiveness. Here, we analyze theoretically and experimentally how causal reasoning contributes to this process of interpretation and can produce attitude reversals due to the network structure of beliefs. We conduct two vignette experiments, one based on the famous slogan of the car rental agency Avis ("We're No. 2—that means we try harder"), and the other based on online product reviews. When participants' contextual beliefs about the economic environment are manipulated, message effectiveness changes as predicted by a Bayesian mechanism in which seemingly negative information is "explained away" in a more positive light, or vice versa. Thus causal reasoning may help account for certain counterintuitive kinds of high-level attitude change.

Reliability is a fundamental problem for measurement in all of science. Al- though defined in multiple ways, and estimated in even more ways, the basic concepts are straight forward and need to be understood by methodologists as well as practioners. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. Easy to use, open source software is applied to examples of real data, and comparisons are made between the many types of reliability

The present study examined the dimensional structure underlying the Multidimensional Schizotypy Scale (MSS) and its brief version (MSS-B). The MSS and MSS-B were developed to assess current multidimensional conceptualizations of schizotypy. We used Exploratory Graph Analysis (EGA) to evaluate the dimensional structure of the scales in two large, independent samples (n = 6,265 and n = 1,000). We then used Confirmatory Factor Analysis (CFA) to compare the fit of the theoretical dimensions with the EGA dimensions. For the MSS, EGA identified four dimensions: positive schizotypy, two dimensions of negative schizotypy (affective and social anhedonia), and disorganized schizotypy. For the MSS-B, EGA identified three dimensions, which corresponded to the theorized positive, negative, and disorganized dimensions. Based on the MSS’s EGA dimensions, we also estimated a four-factor model for the MSS-B. The CFA comparison found that the four-factor model fit significantly better than the theoretical three-factor model for both the MSS and MSS-B. In short, we propose that the four-factor model supports the theoretical model and offers a more nuanced interpretation of the negative schizotypy dimension. Our findings offer new implications for future research on the MSS and MSS-B dimensions that may provide differential associations with interview and questionnaire measures.

Individual differences in general speed lead to a positive correlation between the mean and standard deviation of mean latency. This “coarse” scaling effect causes the mean latency difference (MLD) to be spuriously correlated with general speed. Within individuals, the correlation between the mean and standard deviation of trial latencies leads contrasted distributions to increase their overlap as an MLD of fixed width is translated to the right. To address this “fine” scaling effect, contrasts based on within subject latency transformations including the logarithm, standardization, and ranking were evaluated and turned out to be distinctly superior to the MLD. Notably, the mean gaussian rank latency difference was internally consistent, eliminated fine scaling, meliorated coarse scaling, reduced correlations with general speed, increased statistical power to detect within subject and between group effects, and has the potential to increase the validity of inferences drawn from response latency data.

Cross-validation (CV) is increasingly popular as a generic method to adjudicate between mathematical models of cognition and behavior. In order to measure model generalizability, CV quantifies out-of-sample predictive performance, and the CV preference goes to the model that predicted the out-of-sample data best. The advantages of CV include theoretic simplicity and practical feasibility. Despite its prominence, however, the limitations of CV are often underappreciated. Here we demonstrate the limitations of a particular form of CV --Bayesian leave-one-out cross-validation or LOO-- with three concrete examples. In each example, a data set of infinite size is perfectly in line with the predictions of a simple model (i.e., a general law or invariance). Nevertheless, LOO shows bounded and relatively modest support for the simple model. We conclude that CV is not a panacea for model selection.

Creativity assessment with open-ended production tasks relies heavily on scoring the quality of a subject’s ideas. This creates a faceted measurement structure involving persons, tasks (and ideas within tasks), and raters. Most studies, however, do not model possible systematic differences among raters. The present study examines individual rater differences in the context of a planned-missing design and its association with reliability and validity of creativity assessments. It applies the many-facet Rasch model (MFRM) to model and correct for these differences. We reanalyzed data from 2 studies (Ns = 132 and 298) where subjects produced metaphors, alternate uses for common objects, and creative instances. Each idea was scored by several raters. We simulated several conditions of reduced load on raters where they scored subsets of responses. We then compared the reliability and validity of IRT estimated scores (original vs. IRT adjusted scores) on various conditions of missing data. Results show that (a) raters vary substantially on the lenient-severity dimension, so rater differences should be modeled; (b) when different combinations of raters assess different subsets of ideas, systematic rater differences confound subjects’ scores, increasing measurement error and lowering criterion validity with external variables; and (c) MFRM adjustments effectively correct for rater effects, thus increasing correlations of scores obtained from partial with scores obtained with full data. We conclude that MFRM is a powerful means to model rater differences and reduce rater load in creativity research.

Mediation analysis plays a central role in the investigation of causal mechanisms, where the total, direct and indirect influence of one variable on another is typically assessed using path-tracing rule}. Two recent developments promise to revolutionize mediation analysis: first, the interventionist causal inference framework has allowed researchers to define and investigate mediation in terms of interventions on variables rather than path-tracing rules; second, researchers have begun to use Continuous-Time (CT) models to investigate mediation in longitudinal data. This paper shows how CT mediation can be treated using an interventionist approach. The links between interventionist and path-tracing definitions in traditional longitudinal SEM models and in CT models are explored in detail. The treatment of mediation from diverse strands of literature is unified, by showing how total, direct and indirect effects in different frameworks can be defined as qualitatively different types of interventions on the values of variables.

Stress is considered to be among the leading causes of various psychopathological problems, whose outcomes are often difficult to foresee. In order to prevent consequent maladaptive behaviors, the present paper introduces a novel, three-phase architecture for analyzing the formation, handling and manifestation of stress in individuals. We specifically model the process of transforming stressful stimuli to cognitive schemas, and from these to subconscious defense mechanism manifestations. Such three-phase model has been implemented and validated via the PERSONA-80 test. Psychometric properties of PERSONA-80 have been evaluated on a sample of 83 individuals from general and clinical populations. The test showed good internal consistency and excellent predictive validity results, from which we derived the purified scale PERSONA-16. The potential of the developed instrument is the detection of the inability to produce mature defense mechanisms from an arbitrary state of mind, or the inability to preserve schemas when exposed to stressful conditions, useful in high-stress occupation screening and monitoring. To encourage widespread use and validation in different domains, we provide at the end of this text the complete PERSONA-16 instrument.

Psychological constructs, such as emotions, thoughts, and attitudes are often measured by asking individuals to reply to questions using closed-ended numerical rating scales. However, when asking people about their state of mind in a natural context ("How are you?"), we receive open-ended answers using words ("Fine and happy!") and not closed-ended answers using numbers ("7") or categories ("A lot"). Nevertheless, to date it has been difficult to objectively quantify responses to open-ended questions. We develop an approach using open-ended questions in which the responses are analyzed using natural language processing (Latent Semantic Analyses). This approach of using open-ended, semantic questions is compared with traditional rating scales in nine studies (N = 92-854), including two different study paradigms. The first paradigm requires participants to describe psychological aspects of external stimuli (facial expressions) and the second paradigm involves asking participants to report their subjective well-being and mental health problems. The results demonstrate that the approach using semantic questions yields good statistical properties with competitive, or higher, validity and reliability compared with corresponding numerical rating scales. As these semantic measures are based on natural language and measure, differentiate, and describe psychological constructs, they have the potential of complementing and extending traditional rating scales. (PsycINFO Database Record (c) 2019 APA, all rights reserved).