Search

Reliability is a crucial concept in psychometrics. Although it is typically estimated as a single fixed quantity, previous work suggests that reliability can vary across persons, groups, and covariates. We propose a novel method for estimating and modeling case-specific reliability without repeated measurements or parallel tests. The proposed method employs a “Reliability Factor” that models the error variance of each case across multiple indicators, thereby producing case-specific reliability estimates. Additionally, we use Gaussian process modeling to a estimate non-linear, non-monotonic function between the latent factor itself and the reliability of the measure, providing an analogue to test information functions in item response theory. The reliability factor model is a new tool for examining latent regions with poor conditional reliability, and correlates thereof, in a classical test theory framework.

Measurement reliability is a fundamental concept in psychology. It is traditionally considered a stable property of a questionnaire, measurement device, or experimental task. Although intraclass correlation coefficients (ICC) are often used to assess reliability in repeated measure designs, their descriptive nature depends upon the assumption of a common within-person variance. This work focuses on the presumption that each individual is adequately described by the average within-person variance in hierarchical models. And thus whether reliability generalizes to the individual level, which leads directly into the notion of individually varying ICCs. In particular, we introduce a novel approach, using the Bayes factor, wherein a researcher can directly test for homogeneous within-person variance in hierarchical models. Additionally, we introduce a membership model that allows for classifying which (and how many) individuals belong to the common variance model. The utility of our methodology is demonstrated on cognitive inhibition tasks. We find that heterogeneous within-person variance is a defining feature of these tasks, and in one case, the ratio between the largest to smallest within-person variance exceeded 20. This translates into a tenfold difference in person-specific reliability! We also find that few individuals belong to the common variance model, and thus traditional reliability indices are potentially masking important individual variation. We discuss the implications of our findings and possible future directions. The methods are implemented in the R package vICC

Missing data are ubiquitous in psychological research. They may come about as an unwanted result of coding or computer error, participants' non-response or absence, or missing values may be intentional, as in planned missing designs. We discuss the effects of missing data on χ²-based goodness-of-fit indices in Structural Equation Modeling (SEM), specifically on the Root Mean Squared Error of Approximation (RMSEA). We use simulations to show that naive implementations of the RMSEA have a downward bias in the presence of missing data and, thus, overestimate model goodness-of-fit. Unfortunately, many state-of-the-art software packages report the biased form of RMSEA. As a consequence, the scientific community may have been accepting a much larger fraction of models with non-acceptable model fit. We propose a bias-correction for the RMSEA based on information-theoretic considerations that take into account the expected misfit of a person with fully observed data. The corrected RMSEA is asymptotically independent of the proportion of missing data for misspecified models. Importantly, results of the corrected RMSEA computation are identical to naive RMSEA if there are no missing data.

Ordinal variables, while extremely common in Psychology, are almost exclusively analysed with statistical models that falsely assume them to be metric. This practice can lead to distorted effect size estimates, inflated error rates, and other problems. We argue for the application of ordinal models that make appropriate assumptions about the variables under study. In this tutorial article, we first explain the three major ordinal model classes; the cumulative, sequential and adjacent category models. We then show how to fit ordinal models in a fully Bayesian framework with the R package brms, using data sets on stem cell opinions and marriage time courses. Two appendices provide detailed mathematical derivations of the models, and a third practical example that connects ordinal models to Signal Detection Theory with confidence rating data. Ordinal models provide better theoretical interpretation and numerical inference from ordinal data, and we recommend their widespread adoption in Psychology.

[Correction Notice: An Erratum for this article was reported in Vol 115(6) of Journal of Personality and Social Psychology (see record 2018-63189-002). In the article, the original supplemental material has been revised to include a clarifying note to the Tests of model fit over larger sample (Samples C-G) section and post-peer review analyses added to the Post-peer review Factor Analytic Tests section. All versions of this article have been corrected.] Philosophers and behavioral scientists refer to wisdom as unbiased reasoning that guides one toward a balance of interests and promotes a good life. However, major instruments developed to test wisdom appear biased, and it is unclear whether they capture balance-related tendencies. We examined whether shifting from global, de-contextualized reports to state-level reports about concrete situations provides a less biased method to assess wise reasoning (e.g., intellectual humility, recognition of uncertainty and change, consideration of the broader context at hand and perspectives of others, integration of these perspectives or compromise), which may be aligned with the notion of balancing interests. Results of a large-scale psychometric investigation (N = 4,463) revealed that the novel Situated WIse Reasoning Scale (SWIS) is reliable and appears independent of psychological biases (attribution bias, bias blind spot, self-deception, and impression management), whereas global wisdom reports are subject to such biases. Moreover, SWIS scores were positively related to indices of living well (e.g., adaptive emotion regulation, mindfulness), and balancing of cooperative and self-protective interests, goals (influence-vs.-adjustment), and causal inferences about conflict (attribution to the self-vs.-other party). In contrast, global wisdom reports were unrelated or negatively related to balance-related measures. Notably, people showed modest within-person consistency in wise reasoning across situations or over time, suggesting that a single-shot measurement may be insufficient for whole understanding of trait-level wisdom. We discuss theoretical and practical implications for research on wisdom, judgment and decision making, well-being, and prosociality. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Background. Studies investigating the link between depressive symptoms and inflammation have yielded inconsistent results, which may be due to two factors. First, studies differed regarding the specific inflammatory markers studied and covariates accounted for. Second, specific depressive symptoms may be differentially related to inflammation. We address both challenges using network psychometrics.Methods. We estimated seven regularized mixed graphical models in the NESDA data (N=2321) to explore shared variances among 1) depression severity, modeled via depression sum-score, 9 DSM-5 symptoms, or 28 individual depressive symptoms; 2) inflammatory markers C-reactive protein (CRP), interleukin 6 (IL-6), and tumor necrosis factor α (TNF-α); 3) before and after adjusting for sex, age, body mass index (BMI), exercise, smoking, alcohol, and chronic diseases.Results. The depression sum-score was related to both IL-6 and CRP before, and only to IL-6 after covariate adjustment. When modeling the DSM-5 symptoms and CRP in a conceptual replication of Jokela et al. (2016), CRP was associated with ‘sleep problems’, ‘energy level’, and ‘weight/appetite changes’; only the first two links survived covariate adjustment. In a conservative model with all 38 variables, symptoms and markers were unrelated. Following recent psychometric work, we re-estimated the full model without regularization: the depressive symptoms ‘insomnia, ‘hypersomnia’, and ‘aches and pain’ showed unique positive relations to all inflammatory markers.Conclusions. We found evidence for differential relations between markers, depressive symptoms, and covariates. Associations between symptoms and markers were attenuated after covariate adjustment; BMI and sex consistently showed strong relations with inflammatory markers.

This article introduces the NetworkToolbox package for R. Network analysis offers an intuitive perspective on complex phenomena via models depicted by nodes (variables) and edges (correlations). The ability of networks to model complexity has made them the standard approach for modeling the intricate interactions in the brain. Similarly, networks have become an increasingly attractive model for studying the complexity of psychological and psychopathological phenomena. NetworkToolbox aims to provide psychological researchers with state-of-the-art methods and measures for estimating and analyzing brain, cognitive, and psychometric data. In this article, I introduce NetworkToolbox and provide a manual for applying the package’s functions to psychometric (personality) data.

Psychological data often violate the normality assumptions made by commonly used statistical methods. These violations are addressed in a variety of ways such as transformations or assuming the employed method is robust to violations. Here we argue that data transformations are unnecessary at best and severely misleading at worst. An alternative approach is to use a Bayesian model that permits skewness and other perturbations to classical assumptions (e.g., heteroskedasticity). Through simulation, we demonstrate that a Bayesian skew-normal model has optimal frequentist properties (i.e., "type 1" error, "power", unbiasedness) compared to normal-assumptive models with or without transformation. Furthermore, the Bayesian skew-normal model has greater predictive utility, as indicated by posterior predictive checking and approximate leave-one-out cross-validation. After an applied example, we discuss practical implications of our findings for psychological science in general, and specifically how Bayesian modeling can improve reproducibility in psychology.

Preregistration is an open science practice that requires the specification of research hypotheses and analysis plans before the data are inspected. Here, we discuss the benefits of preregistration for hypothesis-driven, confirmatory bilingualism research. Using examples from psycholinguistics and bilingualism, we illustrate how non-peer reviewed preregistrations can serve to implement a clean distinction between hypothesis testing and data exploration. This distinction helps researchers avoid casting post-hoc hypotheses and analyses as confirmatory ones. We argue that, in keeping with current best practices in the experimental sciences, preregistration, along with sharing data and code, should be an integral part of hypothesis-driven bilingualism research.

Mixed-effects models are becoming common in psychological science. Although they have many desirable features, there is still untapped potential. It is customary to view homogeneous variance as an assumption to satisfy. We argue to move beyond that perspective, and to view modeling within-person variance as an opportunity to gain a richer understanding of psychological processes. The technique to do so is based on the mixed-effects location scale model that can simultaneously estimate mixed-effects submodels to both the mean (location) and within-person variance (scale). We develop a framework that goes beyond assessing the submodels in isolation of one another and introduce a novel Bayesian hypothesis test for mean—variance correlations in the distribution of random effects. We first present a motivating example, which makes clear how the model can characterize mean—variance relations. We then apply the method to reaction times (RTs) gathered from 2 cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean—variance relations. This stands in contrast to the dominant view of within-person variance (i.e., “noise”). The results also point toward paradoxical within-person, as opposed to between-person, effects: several people had slower and less variable incongruent responses. This contradicts the typical pattern, wherein larger means tend to be associated with more variability. We conclude with future directions, spanning from methodological to theoretical inquires, that can be answered with the presented methodology.Impact StatementTranslational Abstract: Mixed-effects models are becoming common in psychological science. Although researchers typically focus on explaining the dependent variable, characterizing individual differences in within-person variance can provide valuable insights into behavioral consistency. The technique to do so is based on an extension to the mixed-effect model that can simultaneously predict both the mean and within-person variance. We introduce Bayesian methodology to test the covariance among individual difference parameters that capture mean and within-person variance relations. This is motivated by theoretical models that predict larger means will be associated with more variability. Our approach allows for confirmatory hypothesis testing of this proposed relation in the distribution of random effects. Hence, researchers can begin to test the interplay between an experimental effect on the mean and within-person variability in their repeated measurements. We apply our method to RTs gathered from two cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean-variance relations. We also uncovered strikingly large correlations between the experimental effect on the mean and within-person variability. Substantively this novel finding suggests that cognitive inhibition has important and highly interrelated signatures on both RT and RT consistency. The method is implemented in the user-friendly R package hypMuVar.

Decision making is a vital aspect of our everyday functioning, from simple perceptual demands to more complex and meaningful decisions. The strategy adopted to make such decisions is often viewed as balancing elements of speed and caution, i.e. making fast or careful decisions. Using sequential sampling models to analyse decision making data can allow us to tease apart strategic differences, such as being more or less cautious, from processing differences, which would otherwise be indistinguishable in behavioural data. Our study used a multiple object tracking task where student participants and a highly skilled military group were compared on their ability to track several items at once. Using a mathematical model of decision making (the linear ballistic accumulator), we show the underpinnings of how two groups differ in performance. Results showed a large difference between the groups on accuracy, with the Royal Australian Air Force (RAAF) group outperforming students. An interaction effect was observed between groups and level of difficulty in response times, where RAAF response times slowed at a greater rate than the student group as difficulty increased. Model results indicated that the RAAF personnel were more cautious in their decisions than students, and had faster processing in some conditions. Our study shows the strength of sequential sampling models, as well as providing a first attempt at fitting a sequential sampling model to data from a multiple object tracking task.

Objective Psychopathology research has relied on discrete diagnoses, which neglects the unique manifestations of each individual's pathology. Borderline personality disorder combines interpersonal, affective, and behavioral regulation impairments making it particularly ill-suited to a "one size fits all" diagnosis. Clinical assessment and case formulation involve understanding and developing a personalized model for each patient's contextualized dynamic processes, and research would benefit from a similar focus on the individual. Method We use group iterative multiple model estimation, which estimates a model for each individual and identifies general or shared features across individuals, in both a mixed-diagnosis sample (N = 78) and a subsample with a single diagnosis (n = 24). Results We found that individuals vary widely in their dynamic processes in affective and interpersonal domains both within and across diagnoses. However, there was some evidence that dynamic patterns relate to transdiagnostic baseline measures. We conclude with descriptions of 2 person-specific models as an example of the heterogeneity of dynamic processes. Conclusions The idiographic models presented here join a growing literature showing that the individuals differ dramatically in the total patterning of these processes, even as key processes are shared across individuals. We argue that these processes are best estimated in the context of person-specific models, and that so doing may advance our understanding of the contextualized dynamic processes that could identify maintenance mechanisms and treatment targets. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Network psychometrics is undergoing a time of methodological reflection. In part, this was spurred by the revelation that l1-regularization does not reduce spurious associations in partial correlation networks. In this work, we address another motivation for the widespread use of regularized estimation: the thought that it is needed to mitigate overfitting. We first clarify important aspects of overfitting and the bias-variance tradeoff that are especially relevant for the network literature, where the number of nodes or items in a psychometric scale are not largecompared to the number of observations (i.e., a low p/n ratio). This revealed that bias and especially variance are most problematic in p=n ratios rarely encountered. We then introduce a nonregularized method, based on classical hypothesis testing, that fulfills two desiderata: (1) reducing or controlling the false positives rate and (2) quelling concerns of overfitting by providing accurate predictions. These were the primary motivations for initially adopting the graphical lasso (glasso). In several simulation studies, our nonregularized method provided more than competitive predictive performance, and, in many cases, outperformed glasso. Itappears to be nonregularized, as opposed to regularized estimation, that best satisfies these desiderata. We then provide insights into using our methodology. Here we discuss the multiple comparisons problem in relation to prediction: stringent alpha levels, resulting in a sparse network, can deteriorate predictive accuracy. We end by emphasizing key advantages of our approach that make it ideal for both inference and prediction in network analysis.

This dissertation deals with the problem of modeling psychopathology. Its first part focuses on statistical (data) models and introduces a number of models for cross-sectional and time series data that can be visualized as a network. This includes Mixed Graphical Models (MGMs), which allow one to include variables of different types in a statistical network model, Moderated Network Models (MNMs) which allow pairwise interactions to be moderated by other variables in the model, and time-varying Vector Autoregressive (VAR) models and MGMs that relax the standard assumption of stationarity. In addition, I discuss several methodological issues related to statistical network models such as the importance of considering predictability, model selection between AR and VAR models, and how the interpretation of the Ising model depends on its domain. The second part focuses on formal theories of psychopathology and how to develop them using data models. I first illustrate the fundamental difficulties in obtaining a formal theory with a purely statistical approach, by trying to recover an assumed bistable system for emotion dynamics with currently popular time series analyses. Next, I present a formal theory of panic disorder, based on an extensive review of the literature on the phenomenology of panic disorder and existing theories. Finally, I discuss three different ways to use data models to construct formal theories about psychopathological phenomena. Based on this discussion, I put forward an abductive framework for constructing formal theories for psychological and psychopathological phenomena.

La santé psychique des étudiants est un problème de santé publique qui semble sous-considéré, en partie en raison de l’hétérogénéité des résultats obtenus dans les différentes enquêtes. Cette hétérogénéité a différentes causes : imprécision dans les termes utilisés ; confusion dans les périodes de temps étudiées ; variabilité des questionnaires utilisés qui ne mesurent pas les mêmes symptômes et dont les seuils de détection varient selon les études ; modélisation d’analyse des données conduisant à des construits narratifs trop simplistes. Ces problématiques sont inhérentes tant au processus scientifique qu’à son utilisation dans le discours social., Students’ mental health is a public health problem that seems under-considered, partly because of the heterogeneity of the results obtained in the different surveys. This heterogeneity has different causes: inaccuracy in the terms used; confusion in the time periods studied; variability of questionnaires used that do not measure the same symptoms and whose detection thresholds vary by study; data analysis modeling leading to overly simplistic narrative constructs. This problematic is inherent in both the scientific process and its use in social discourse.

Agents that can make better use of computation, experience, time, and memory can solve a greater range of problems more effectively. A crucial ingredient for managing such finite resources is intelligently chosen abstract representations. But, how do abstractions facilitate problem solving under limited resources? What makes an abstraction useful? To answer such questions, we review several trends in recent reinforcement-learning research that provide insight into how abstractions interact with learning and decision making. During learning, abstraction can guide exploration and generalization as well as facilitate efficient tradeoffs---e.g., time spent learning versus the quality of a solution. During computation, good abstractions provide simplified models for computation while also preserving relevant information about decision-theoretic quantities. These features of abstraction are not only key for scaling up artificial problem solving, but can also shed light on what pressures shape the use of abstract representations in humans and other organisms.

Conventional meta-analytic procedures assume that effect sizes are independent. When effect sizes are not independent, conclusions based on these conventional procedures can be misleading or even wrong. Traditional approaches, such as averaging the effect sizes and selecting one effect size per study, are usually used to avoid the dependence of the effect sizes. These ad-hoc approaches, however, may lead to missed opportunities to utilize all available data to address the relevant research questions. Both multivariate meta-analysis and three-level meta-analysis have been proposed to handle non-independent effect sizes. This paper gives a brief introduction to these new techniques for applied researchers. The first objective is to highlight the benefits of using these methods to address non-independent effect sizes. The second objective is to illustrate how to apply these techniques with real data in R and Mplus. Researchers may modify the sample R and Mplus code to fit their data. Electronic supplementary material The online version of this article (10.1007/s11065-019-09415-6) contains supplementary material, which is available to authorized users.

In this article we describe the origins of the subtests that appear on the modern Stanford-Binet Intelligence Scales (SB5), Wechsler Preschool and Primary Scale of Intelligence (WPPSI-IV), Wechsler Intelligence Scale for Children (WISC-V), and Wechsler Adult Intelligence Scale (WAIS-IV). We found that the majority of these subtest formats were first created in 1908 or earlier and that only three have been created since 1980. We discuss the implications of these findings, which are that (1) many subtests have lengthy research histories that support their use in measuring intelligence; (2) many subtests have formats that predate modern theories of test creation, cognitive psychology, and intelligence; and (3) the history of many subtests is more complex than psychologists probably realize.

A wide variety of effect size indices have been proposed for quantifying the magnitude of treatment effects in single-case designs. Commonly used measures include parametric indices such as the standardized mean difference as well as nonoverlap measures such as the percentage of nonoverlapping data, improvement rate difference, and nonoverlap of all pairs. Currently, little is known about the properties of these indices when applied to behavioral data collected by systematic direct observation, even though systematic direct observation is the most common method for outcome measurement in single-case research. This study uses Monte Carlo simulation to investigate the properties of several widely used single-case effect size measures when applied to systematic direct observation data. Results indicate that the magnitude of the nonoverlap measures and of the standardized mean difference can be strongly influenced by procedural details of the study's design, which is a significant limitation to using these indices as effect sizes for meta-analysis of single-case designs. A less widely used parametric index, the log response ratio, has the advantage of being insensitive to sample size and observation session length, although its magnitude is influenced by the use of partial interval recording. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

There is a body of research on how driver behaviour might change in response to bicyclists’ appearance. In 2007, Walker published a study suggesting motorists drove closer on average when passing a bicyclist if the rider wore a helmet, potentially increasing the risk of a collision. Olivier and Walter re-analysed the same data in 2013 and claimed helmet wearing was not associated with close vehicle passing. Here we show how Olivier and Walter’s analysis addressed a subtly, but importantly, different question than Walker’s. Their conclusion was based on omitting information about variability in driver behaviour and instead dividing overtakes into two binary categories of ‘close’ and ‘not close’; we demonstrate that they did not justify or address the implications of this choice, did not have sufficient statistical power for their approach, and moreover show that slightly adjusting their definition of ‘close’ would reverse their conclusions. We then present a new analysis of the original dataset, measuring directly the extent to which drivers changed their behaviour in response to helmet wearing. This analysis confirms that drivers did, overall, get closer when the rider wore a helmet. The distribution of overtaking events shifted just over one-fifth of a standard deviation closer to the rider – a potentially important behaviour if, as theoretical frameworks suggest, near-misses and collisions lie on a continuum. The paper ends by considering wider issues surrounding this topic and suggests public health research might be best served by shifting focus to risk elimination rather than harm mitigation.

State trace analyses assess the latent dimensionality of a cognitive process by asking whether the means of two dependent variables conform to a monotonic function across a set of conditions. Using an assumption of independence between the measures, recently proposed statistical tests address bivariate measurement error, allowing both frequentist and Bayesian analyses of monotonicity (e.g., Davis-Stober, Morey, Gretton, & Heathcote, 2016; Kalish, Dunn, Burdakov, & Sysoev, 2016). However, statistical inference can be biased by unacknowledged dependencies between measures, particularly when the data are insufficient to overwhelm an incorrect prior assumption of independence. To address this limitation, we developed a hierarchical Bayesian model that explicitly models the separate roles of subject, item, and trial-level dependencies between two measures. Assessment of monotonicity is then performed by fitting separate models that do or do not allow a non-monotonic relation between the condition effects (i.e., same versus different rank orders). The Widely Applicable Information Criterion (WAIC) and Pseudo Bayesian Model Averaging – both cross validation measures of model fit – are used for model comparison, providing an inferential conclusion regarding the dimensionality of the latent psychological space. We validated this new state trace analysis technique using model recovery simulation studies, which assumed different ground truths regarding monotonicity and the direction/magnitude of the subject- and trial-level dependence. We also provide an example application of this new technique to a visual object learning study that compared performance on a visual retrieval task (forced choice part recognition) versus a verbal retrieval task (cued recall).

Regularization methods such as the least absolute shrinkage and selection operator (LASSO) are commonly used in high dimensional data to achieve sparser solutions. Recently, methods such as regularized structural equation modeling (SEM) and penalized likelihood SEM have been proposed, trying to transfer the benefits of regularization to models commonly used in social and behavioral research. These methods allow researchers to estimate large models even in the presence of small sample sizes. However, some drawbacks of the LASSO, such as high false positive rates (FPRs) and inconsistency in selection results, persist at the same time. We propose the application of stability selection, a method based on repeated resampling of the data to select stable coefficients, to regularized SEM as a mechanism to overcome these limitations. Across 2 simulation studies, we find that stability selection greatly improves upon the LASSO in selecting the correct paths, specifically through reducing the number of false positives. We close the article by demonstrating the application of stability selection in 2 empirical examples and presenting several future research directions. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.

International audience; We elaborated an index, the Interference Distribution Index, which allows quantifying the relation between response times and the size of the interference effect. This index is associated with an intuitive graphical representation, the Lorenz-interference plot. We show that this index has some convenient properties in terms of sensitivity to changes in the distribution of the interference effect and to aggregation of individual data. Moreover, it turns out that this index is the only one (up to an arbitrary increasing transformation) possessing these properties. The relevance of this index is illustrated through simulations of a cognitive model of interference effects and reanalysis of experimental data.

Mediation analysis is a popular statistical method throughout psychology, communication, business and other behavioral sciences. However, previous research has demonstrated that that typical sample sizes are too small to have high power to detect indirect effects in between-subject designs; however, some inferential methods have higher power than others. Montoya and Hayes (2017) developed new methods for estimating and conducting inference on indirect effects in within-subject designs. For tests of means, within-subject designs boast power advantages over between-subject designs; however, this advantage has not been demonstrated for mediation analysis. Monte Carlo simulation is used to compare six inferential methods, two designs, and a broad range of sample sizes, effect sizes, and correlations among repeated measurements. The results suggest within-subject designs require about half the sample size of between-subject designs to detect indirect effects of the same size, and the effect of design is much greater than the effect of inferential method. Factors which can impact power (e.g. highly correlated measurements) are discussed, and how to determine if a within-subject design is appropriate for a given research question. MEMORE is an easy to use tool for estimating and conducting inference on indirect effects in two-condition within-subject designs.

Data analysis is a risky endeavor, particularly among those unaware of its dangers. In the words of Cook and Campbell (1976; see also Cook, Campbell, and Shadish 2002), “Statistical Conclusions Validity” threatens all experiments that subject themselves to the dark arts of statistical magic. Although traditional statistics classes may advise against certain practices (e.g., multiple comparisons, small sample sizes, violating normality), they may fail to cover others (e.g., outlier detection and violating linearity). More common, perhaps, is that researchers may fail to remember them. In this paper, rather than rehashing old warnings and diatribes against this practice or that, I instead advocate a general statistical analysis strategy. This graphically-based eight step strategy promises to resolve the majority of statistical traps researchers may fall in without having to remember large lists of problematic statistical practices. These steps will assist in preventing both Type I and Type II errors and yield critical insights about the data that would have otherwise been missed. I conclude with an applied example that shows how the eight steps highlight data problems that would not be detected with standard statistical practices.

JASP, Jeffrey’s Amazing Statistics Program (official webpage: https://jasp-stats.org/), version 0.12.2, is an open-source, free statistical software package available on Windows, MacOS, and Linux platforms. It was developed by a group of quantitative methodologists who are interested in improving statistical testing and analysis methods used in the fields of psychological sciences. It implements various analysis methods, particularly those in Bayesian analysis and data science. The implemented methods include t-tests, ANOVA, regression analysis, factor analysis, machine learning, meta-analysis, network analysis, and SEM. JASP also includes a data editor for visual inspection and pre-processing. It supports data importing and exporting from/to various data sources. The statistical analysis program R constitutes the basis of JASP, but for users who are not familiar with programming, JASP is featured with a graphical user interface (GUI) so that the users can select analysis modules and modify options conveniently. To support users in academia, especially those in psychology, JASP provides functionalities for visualizing analysis results in tables and figures according to the American Psychological Association (APA) convention and presents options for modification once they are created.

This study develops a new way to model the emotional impact on a particular individual (person, group,...) that may have feelings, to any event or any information. The significance of an event or an information depends on the person undergoing it. Each person reacts emotionally differently to a same situation in terms of intensity and/or type of emotion. A particular same event is therefore perceived subjectively different by each individual. The reason is that each person has his own personality, his own experience, his own history, his own education and tradition ... In this study, we assume that all the Events / Information have common interpretable points relative to each individual: Who is in the event ?, What is done? When it is done? Where it is done? How often/much it is done? Why it is done? and How it is done? Each element of the vector [Who; What; When; Where; How Much; Why; How] will be called as an emotional stimulus and will be mathematically modeled relatively to the studied individual. These stimuli, interpreted differently by each individual, will be the starting point for the creation of emotions at different intensities. The objective of this study is to mathematically link these different stimuli to the intensity of the resulting emotions following an event relatively to an individual.

Universities around the world are incorporating online learning, often relying upon videos (asynchronous multimedia). We systematically reviewed the effects of video on learning in higher education. We searched five databases using 27 keywords to find randomised trials that measured the learning effects of video among college students. We conducted full-text screening, data-extraction, and risk of bias in duplicate. We calculated pooled effect-sizes using multi-level random-effects meta-analysis. Searches retrieved 9,677 unique records. After screening 329 full-texts, 103 met inclusion criteria, with a pooled sample of 7,776 students. Swapping video for existing teaching methods led to small improvements in student learning (g = 0.28). Adding video to existing teaching led to strong learning benefits (g = 0.80). Although results may be subject to some experimental and publication biases, they suggest that videos are unlikely to be detrimental and usually improve student learning.

The mind and consciousness are pretty complex to observe and analyze quantitatively since the complexity of neuron and mystery of consciousness. Recent years, many scientists try to explain our mind from physical viewpoint or elucidate the physics of the mind. In the paper, we reveal a new approach to understand our mind and present a tentative theory to describe how our mind works respect to physics. Two significant concept of time, namely, the internal time of mind and external time of the outside are introduced to measure our mind quantitatively on some degree. The related law of these two types of time is similar to the special relativity proposed by Einstein. Our theory aims to provide a new perspective to understand and know ourselves and lead our life better. At the end of paper, we design an experiment to check the correctness of our results.

This paper presents a new idea to understand human's thoughts and behaviors. Here, we propose a tentative theory of human's thoughts and behaviors, including the relativity of the thoughts, the relativity of the behaviors with thoughts, the inertia of the behaviors with thoughts and its corollary, the spiral advance of the human's thoughts, the procedure of the behaviors with thoughts, and a universal identity. This paper aims to provide a new method to know ourselves and lead human's life better. We design some experiments to check the theory lastly.

Response time (RT) information has recently attracted a significant amount of attention in the literature as it may provide meaningful information about item preknowledge. In this study, a Deterministic Gated Lognormal Response Time (DG-LNRT) model is proposed to identify examinees with potential item preknowledge using RT information. The proposed model is applied to a real experimental dataset provided by Toton and Maynes (2019) in which item preknowledge was manipulated, and its performance is demonstrated. Then, the performance of the DG-LNRT model is investigated through a simulation study. The model is estimated using the Bayesian framework via Stan. The results indicate that the proposed model is viable and has the potential to be useful in detecting cheating by using response time differences between compromised and uncompromised items.

Network theory has emerged as a popular framework for conceptualizing psychological constructs and mental disorders. Initially, network analysis was motivated in part by the thought that it can be used for hypothesis generation. Although the customary approach for network modeling is inherently exploratory, we argue that there is untapped potential for confirmatory hypothesis testing. In this work, we bring to fruition the potential of Gaussian graphical models for generating testable hypotheses. This is accomplished by merging exploratory and confirmatory analyses into a cohesive framework built around Bayesian hypothesis testing of partial correlations. We first present a motivating example based on a customary, exploratory analysis, where it is made clear how information encoded by the conditional (in)dependence structure can be used to formulate hypotheses. Building upon this foundation, we then provide several empirical examples that unify exploratory and confirmatory testing in psychopathology symptom networks. In particular, we (1) estimate exploratory graphs; (2) derive hypotheses based on the most central structures; and (3) test those hypotheses in a confirmatory setting. Our confirmatory results uncovered an intricate web of relations, including an order to edge weights within comorbidity networks. This illuminates the rich and informative inferences that can be drawn with the proposed approach. We conclude with recommendations for applied researchers, in addition to discussing how our methodology answers recent calls to begin developing formal models related to the conditional (in)dependence structure of psychological networks.

A common way to form scores from multiple-item scales is to sum responses of all items. Though sum scoring is often contrasted with factor analysis as a competing method, we review how factor analysis and sum scoring both fall under the larger umbrella of latent variable models, with sum scoring being a constrained version of a factor analysis. Despite similarities, reporting of psychometric properties for sum scored or factor analyzed scales are quite different. Further, if researchers use factor analysis to validate a scale but subsequently sum score the scale, this employs a model that differs from validation model. By framing sum scoring within a latent variable framework, our goal is to raise awareness that (a) sum scoring requires rather strict constraints, (b) imposing these constraints requires the same type of justification as any other latent variable model, and (c) sum scoring corresponds to a statistical model and is not a model-free arithmetic calculation. We discuss how unjustified sum scoring can have adverse effects on validity, reliability, and qualitative classification from sum score cut-offs. We also discuss considerations for how to use scale scores in subsequent analyses and how these choices can alter conclusions. The general goal is to encourage researchers to more critically evaluate how they obtain, justify, and use multiple-item scale scores.

Poor mental health is the fifth greatest contributor to the global burden of disease (1), and mental health systems worldwide require strengthening to address this need. Utilising ‘big data’ offers novel and promising ways of analysing problems, relationships, and solutions. However, several problems unique to mental health-based data exist. One of these problems is the activity of off-label or ‘grey prescribing’ of psychotropic medications. Although a long-term practice in the medical community, the standards for how this is captured and analysed in mental health data are non-existent at this stage. To adequately make use of available mental health data to improve outcomes, first problems like this must be identified and overcome.

The past few years were marked by increased online offensive strategies perpetrated by state and non-state actors to promote their political agenda, sow discord, and question the legitimacy of democratic institutions in the US and Western Europe. In 2016, the US congress identified a list of Russian state-sponsored Twitter accounts that were used to try to divide voters on a wide range of issues. Previous research used latent Dirichlet allocation (LDA) to estimate latent topics in data extracted from these accounts. However, LDA has characteristics that may limit the effectiveness of its use on data from social media: The number of latent topics must be specified by the user, interpretability of the topics can be difficult to achieve, and it does not model short-term temporal dynamics. In the current paper, we propose a new method to estimate latent topics in texts from social media termed Dynamic Exploratory Graph Analysis (DynEGA). In a Monte Carlo simulation, we compared the ability of DynEGA and LDA to estimate the number of simulated latent topics. The results show that DynEGA is substantially more accurate than several different LDA algorithms when estimating the number of simulated topics. In an applied example, we performed DynEGA on a large dataset with Twitter posts from state-sponsored right- and left-wing trolls during the 2016 US presidential election. DynEGA revealed topics that were pertinent to several consequential events in the election cycle, demonstrating the coordinated effort of trolls capitalizing on current events in the USA. This example demonstrates the potential power of our approach for revealing temporally relevant information from qualitative text data.

The COVID-19 Psychological Research Consortium (C19PRC) Study aims to assess and monitor the psychological and social impact of the coronavirus (COVID-19) pandemic in the general population, using longitudinal surveys and mixed-methods studies in multiple countries. The first strand of the study, an internet-based panel survey, was launched in the UK in March 2020 during the earliest stages of the pandemic in that country (hereafter referred to as C19PRC-UKW1). This paper describes (1) the development, design and content for C19PRC-UKW1, which was informed by the extant evidence base on the psychosocial impact of previous global outbreaks of similar severe acute respiratory syndromes (e.g. SARS, H1N1, MERS); (2) the specific socio-economic and political context of the C19PRC-UKW1; (3) the recruitment of a large sample of UK adults aged 18 years and older (n=2025) via an internet-based panel survey; (4) the representativeness of the C19PRC-UKW1 sample compared to the UK adult population in terms of important sociodemographic characteristics (e.g. age, sex, household income, etc.); and (5) future plans for C19PRC Study including follow-up survey waves in the UK, supplementary non-survey based study strands linking from the C19PRC-UKW1 and the roll-out of the study to other countries.

What are the socio-political consequences of infectious diseases? Humans have evolved to avoid disease and infection, resulting in a set of psychological mechanisms that promote disease-avoidance, referred to as the behavioural immune system (BIS). One manifestation of the BIS is the cautious avoidance of unfamiliar, foreign, or potentially contaminating stimuli. Specifically, when disease infection risk is salient or prevalent, authoritarian attitudes can emerge that seek to avoid and reject foreign outgroups while favoring homogenous, familiar ingroups. In the largest study conducted on the topic to date (N>240,000), elevated regional levels of infectious pathogens were related to more authoritarian attitudes on three geographical levels: across US metropolitan regions, US states, and cross-culturally across 47 countries. The link between pathogen prevalence and authoritarian psychological dispositions predicted conservative voting behavior in the 2016 US Presidential Election as well as more authoritarian governance and state laws, in which one group of people imposes asymmetrical laws on others in a hierarchical structure. Furthermore, cross-cultural analysis illustrated that the relationship between infectious diseases and authoritarianism was pronounced for infectious diseases that can be acquired from other humans (nonzoonotic), and does not generalize to other infectious diseases that can only be acquired from non-human species (zoonotic diseases). At a time of heightened awareness of infectious diseases, these findings are important reminders that public health and ecology can have ramifications for socio-political attitudes, and this can shape how citizens vote as well as how they govern and are governed.

Psychology studies frequently involve the analysis of binary and count dependent variables. These are typically conducted using generalized linear models (GLMs) that specify non-linear relations between predictors and outcomes. Interactions are central within many research applications of these models. To date, typical practice in evaluating interaction effects in GLMs extends directly from linear approaches, in which the product term coefficient between variables of interest are used to provide evidence of an interaction effect. However, unlike linear models, interaction effects in GLMs are not equal to product terms between predictor variables and are instead a function of all predictors of a model. Here, we applied a partial derivative and finite difference framework to propose solutions for computing and conducting hypothesis testing on interaction effects in binary and count GLMs. We provided concrete guidelines and simulated examples to describe how interaction effects should be estimated and interpreted when GLMs are employed. We further illustrated via simulation how using the product term coefficient as an estimator of the interaction can result in bias and inefficiency of the interaction parameter. We concluded with an example using the Adolescent Brain Cognitive Development Study demonstrating how to correctly evaluate interaction effects in a logistic model.

Despite the widespread and rising popularity of structural equation modeling (SEM) in psychology, there is still much confusion surrounding how to choose an appropriate sample size for SEM. Currently available guidance primarily consists of sample-size rules of thumb that are not backed up by research and power analyses for detecting model misspecification. Missing from most current practices is power analysis for detecting a target effect (e.g., a regression coefficient between latent variables). In this article, we (a) distinguish power to detect model misspecification from power to detect a target effect, (b) report the results of a simulation study on power to detect a target regression coefficient in a three-predictor latent regression model, and (c) introduce a user-friendly Shiny app, pwrSEM, for conducting power analysis for detecting target effects in structural equation models.

Standard multilevel models focus on variables that predict the mean while the within-group variability is largely treated as a nuisance. Recent work has shown the advantage of including predictors for both the mean (the location submodel) and the variability (the scale submodel) within a single model. Constrained versions of the model can be fit in standard mixed effect model software, but the most general version with random effects in each of the location and scale submodels has been noted for being difficult to fit and estimate in software. However, the latest release of Mplus includes new capabilities that facilitate fitting the general version of the model as a multilevel SEM. This paper introduces the general form of the model that includes location and scale random effects (called the location-scale model) and notes how it can be envisioned as a multilevel SEM. We provide a tutorial with example analyses and Mplus code for the model with two-level cross-sectional data and three-level repeated measures data and discuss how such a model has potential to extend recent developments in organizational science.

In multilevel modeling decomposing variables into between and within components is often required in the analysis. This decomposition should not ignore the possible unreliability of an observed group mean (i.e. arithmetic mean) that is due to small cluster sizes and can lead to substantially biased estimates. Adjustment procedures that allow unbiased estimation have been defined and implemented in software for two-level model. This study introduces a two-stage adjustment procedure to implement the procedure for balanced three-level data and shows how to obtain information necessary for calculating adjusted means in unbalanced designs. A simulation study showed that the adjustment procedure provides unbiased estimates. An illustration is also provided to demonstrate how the adjustment procedure can change results in a real data context.

Political involvement is accompanied by a range of positive and negative emotional experiences, including stress, sadness, anger, regret, purpose, and empowerment. These experiences may be especially potent during adolescence when youth are forming political opinions and are becoming more integrated within the political system. Little research has examined adolescents’ political emotional experiences and how these experiences are connected with concordant and future political engagement. Using an ecological momentary assessment design, this study tested cross-sectional and longitudinal within-day associations between adolescents’ political emotional costs and benefits and political engagement. Participants (N=291, Mage=16.09) completed four assessments of political emotional experiences (stress/sadness, anger, regret, empowerment) and multiple forms of political engagement (intent to protest, intent to vote, political interest, political efficacy, political participation) across one day. Greater political empowerment was cross-sectionally associated with all forms of political engagement and longitudinally associated with greater next-moment intent to vote and protest, political interest, and political efficacy. Greater political anger was associated with greater cross-sectional political interest, political efficacy, and political participation, but lower next-moment political efficacy. Greater political regret was associated with lower cross-sectional political efficacy and political interest and greater political stress/sadness was associated with greater cross-sectional intent to vote. Findings emphasize the importance of empowerment for adolescent political engagement and demonstrate nuanced connections between adolescents’ and subsequent involvement.

How humans perceive and represent emotional facial expressions critically guides social interactions and has a profound impact on social cognition. However, much emotion research to date is potentially flawed by relying on the assumption that people represent categorical emotions in the same way, using standardised stimulus sets and therefore overlooking important individual differences in emotion processing. To resolve this problem, we developed a task using Genetic Algorithms and derived participant-generated emotional expressions from 105 individuals. A separate group (N=108) was then asked to identify the expression on these faces. Taken together, results showed relative population-level consistency in the representation of happy faces alongside a high degree of variability in the representation of fear and sadness. High test-retest reliability was observed. This novel method enables the efficient quantification of variation in how people represent emotional expressions, presenting promising advances for the study of individual differences in social cognition and emotion processing.

The multitude of gambling activities corresponds to heterogeneous ways of analyzing behavioral outcomes and may partially underlie the lack of replication in gambling research. The current study incorporated complementary analyses to provide an approach to investigate associations with multi-dimensional gambling data that we demonstrated in a discovery/community sample of 2,116 twins (54.86% female; Mage=24.90) and a replication/clinical sample of 619 siblings (30.37% female; Mage=28.00). Our proposed approach was twofold. First, we used confirmatory factor analyses (CFAs) to derive a general gambling frequency measure across eight gambling activities and to test the common liability hypothesis. Second, we conducted latent class analyses (LCAs) to derive gambling frequency subtypes and investigated their theoretical correspondence with the Pathways Model. Our study identified robust support for the common liability hypothesis of gambling, suggesting a shared mechanism for multiple gambling behaviors – including activities controversially defined as “gambling.” Using LCAs, we identified four novel gambling frequency subtypes with analogous behavioral profiles and correlates across samples and some resemblance with the Pathways Model. The subtype with the highest rates of problem gambling had a frequent appetite for particular gambling activities and demonstrated externalizing psychopathology comparable to the “antisocial impulsivist” pathway. Using co-twin control models, we determined that risk-taking, sensation seeking and antisocial personality disorder predicted gambling frequency above and beyond shared genetic and environmental factors. In sum, we illustrated the utility of multi-dimensional statistical techniques for disentangling the structure and typology of different gambling behaviors and discussed our results in context of the psychometric, empirical and theoretical implications.

Dual-process theories posit that separate kinds of intuitive (Type 1) and reflective (Type 2) processes contribute to reasoning. Under this view, inductive judgments are more heavily influenced by Type 1 processing, and deductive judgments are more strongly influenced by Type 2 processing. Alternatively, single-process theories propose that both types of judgments are based on a common form of assessment. The competing accounts were respectively instantiated as two-dimensional and one-dimensional signal detection models, and their predictions were tested against specifically targeted novel data using signed difference analysis. In two experiments, participants evaluated valid and invalid arguments, under induction or deduction instructions. Arguments varied in believability and type of conditional argument structure. Additionally, we used logic training to strengthen Type 2 processing in deduction (Experiments 1 & 2) and belief training to strengthen Type 1 processing in induction (Experiment 2). The logic training successfully improved validity-discrimination, and differential effects on induction and deduction judgments were evident in Experiment 2. While such effects are consistent with popular dual-process accounts, crucially, a one-dimensional model successfully accounted for the results. We also demonstrate that the one-dimensional model is psychologically interpretable, with the model parameters varying sensibly across conditions. We argue that single-process accounts have been prematurely discounted, and formal modeling approaches are important for theoretical progress in the reasoning field.

Machine learning (i.e., data mining, artificial intelligence, big data) has seen an increase in application in psychological science. Although some areas of research have benefited tremendously from a new set of statistical tools, most often in the use of biological or genetic variables, the hype has not been substantiated in more traditional areas of research. We offer an explanation for this phenomena: namely that poor measurement prevents machine learning algorithms from accurately modeling nonlinear relationships, if they exist. This is showcased across a set of simulated examples, demonstrating that model selection between a machine learning algorithm and regression depends on the measurement quality, regardless of sample size. We conclude with a set of recommendations and a discussion of ways to better integrate machine learning with statistics as traditionally practiced in psychological science.

In the context of high-stakes tests, test takers who do not have enough time to complete a test rush toward the end and may engage in speeded behavior when tests do not penalize guessing. Using mathematical derivations and simulations, previous research showed that random guessing responses should attenuate interitem correlations, and therefore, decrease estimates of reliability. Meanwhile, other researchers showed that random guessing could in fact inflate reliability estimates using real data. We provide analytical derivations on how speededness could affect correlations between two dichotomous items in multiple ways, depending on the manifestation and prevalence of test speededness. Furthermore, we provide two simulation studies that evaluate the magnitude of impact of test speededness on interitem correlations and Cronbach’s alpha. We found that the impact of test speededness can vary between item pairs and that it depends on the manifestation of test speededness and item level characteristics. Furthermore, speeded responses will, in general, attenuate or not affect reliability estimates, depending on the prevalence of such responses and conceptual interpretation of speeded responses. Implications of the findings are discussed.

The Savage-Dickey density ratio is a simple method for computing the Bayes factor for an equality constraint on one or more parameters of a statistical model. In regression analysis, this includes the important scenario of testing whether one or more of the covariates have an effect on the dependent variable. However, the Savage-Dickey ratio only provides the correct Bayes factor if the prior distribution of the nuisance parameters under the nested model is identical to the conditional prior under the full model given the equality constraint. This condition is violated for multiple regression models with a Jeffreys-Zellner-Siow (JZS) prior, which is often used as a default prior in psychology. Besides linear regression models, the limitation of the Savage-Dickey ratio is especially relevant when analytical solutions for the Bayes factor are not available. This is the case for generalized linear models, nonlinear models, or cognitive process models with regression extensions. As a remedy, the correct Bayes factor can be computed using a generalized version of the Savage-Dickey density ratio.

The analysis of longitudinal neuroimaging data within the massively univariate framework provides the opportunity to study empirical questions about neurodevelopment. Missing outcome data are an all-to-common feature of any longitudinal study, a feature that, if handled improperly, can reduce statistical power and lead to biased parameter estimates. The goal of this paper is to provide conceptual clarity of the issues and non-issues that arise from analyzing incomplete data in longitudinal studies with particular focus on neuroimaging data. This paper begins with a review of the hierarchy of missing data mechanisms and their relationship to likelihood-based methods, a review that is necessary not just for likelihood-based methods, but also for multiple-imputation methods. Next, the paper provides a series of simulation studies with designs common in longitudinal neuroimaging studies to help illustrate missing data concepts regardless of interpretation. Finally, two applied examples are used to demonstrate the sensitivity of inferences under different missing data assumptions and how this may change the substantive interpretation. The paper concludes with a set of guidelines for analyzing incomplete longitudinal data that can improve the validity of research findings in developmental neuroimaging research. Keywords: Neuroimaging, Missing data, Likelihood, Longitudinal data