Search

Missing data are ubiquitous in psychological research. They may come about as an unwanted result of coding or computer error, participants' non-response or absence, or missing values may be intentional, as in planned missing designs. We discuss the effects of missing data on χ²-based goodness-of-fit indices in Structural Equation Modeling (SEM), specifically on the Root Mean Squared Error of Approximation (RMSEA). We use simulations to show that naive implementations of the RMSEA have a downward bias in the presence of missing data and, thus, overestimate model goodness-of-fit. Unfortunately, many state-of-the-art software packages report the biased form of RMSEA. As a consequence, the scientific community may have been accepting a much larger fraction of models with non-acceptable model fit. We propose a bias-correction for the RMSEA based on information-theoretic considerations that take into account the expected misfit of a person with fully observed data. The corrected RMSEA is asymptotically independent of the proportion of missing data for misspecified models. Importantly, results of the corrected RMSEA computation are identical to naive RMSEA if there are no missing data.

Ordinal variables, while extremely common in Psychology, are almost exclusively analysed with statistical models that falsely assume them to be metric. This practice can lead to distorted effect size estimates, inflated error rates, and other problems. We argue for the application of ordinal models that make appropriate assumptions about the variables under study. In this tutorial article, we first explain the three major ordinal model classes; the cumulative, sequential and adjacent category models. We then show how to fit ordinal models in a fully Bayesian framework with the R package brms, using data sets on stem cell opinions and marriage time courses. Two appendices provide detailed mathematical derivations of the models, and a third practical example that connects ordinal models to Signal Detection Theory with confidence rating data. Ordinal models provide better theoretical interpretation and numerical inference from ordinal data, and we recommend their widespread adoption in Psychology.

[Correction Notice: An Erratum for this article was reported in Vol 115(6) of Journal of Personality and Social Psychology (see record 2018-63189-002). In the article, the original supplemental material has been revised to include a clarifying note to the Tests of model fit over larger sample (Samples C-G) section and post-peer review analyses added to the Post-peer review Factor Analytic Tests section. All versions of this article have been corrected.] Philosophers and behavioral scientists refer to wisdom as unbiased reasoning that guides one toward a balance of interests and promotes a good life. However, major instruments developed to test wisdom appear biased, and it is unclear whether they capture balance-related tendencies. We examined whether shifting from global, de-contextualized reports to state-level reports about concrete situations provides a less biased method to assess wise reasoning (e.g., intellectual humility, recognition of uncertainty and change, consideration of the broader context at hand and perspectives of others, integration of these perspectives or compromise), which may be aligned with the notion of balancing interests. Results of a large-scale psychometric investigation (N = 4,463) revealed that the novel Situated WIse Reasoning Scale (SWIS) is reliable and appears independent of psychological biases (attribution bias, bias blind spot, self-deception, and impression management), whereas global wisdom reports are subject to such biases. Moreover, SWIS scores were positively related to indices of living well (e.g., adaptive emotion regulation, mindfulness), and balancing of cooperative and self-protective interests, goals (influence-vs.-adjustment), and causal inferences about conflict (attribution to the self-vs.-other party). In contrast, global wisdom reports were unrelated or negatively related to balance-related measures. Notably, people showed modest within-person consistency in wise reasoning across situations or over time, suggesting that a single-shot measurement may be insufficient for whole understanding of trait-level wisdom. We discuss theoretical and practical implications for research on wisdom, judgment and decision making, well-being, and prosociality. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Background. Studies investigating the link between depressive symptoms and inflammation have yielded inconsistent results, which may be due to two factors. First, studies differed regarding the specific inflammatory markers studied and covariates accounted for. Second, specific depressive symptoms may be differentially related to inflammation. We address both challenges using network psychometrics.Methods. We estimated seven regularized mixed graphical models in the NESDA data (N=2321) to explore shared variances among 1) depression severity, modeled via depression sum-score, 9 DSM-5 symptoms, or 28 individual depressive symptoms; 2) inflammatory markers C-reactive protein (CRP), interleukin 6 (IL-6), and tumor necrosis factor α (TNF-α); 3) before and after adjusting for sex, age, body mass index (BMI), exercise, smoking, alcohol, and chronic diseases.Results. The depression sum-score was related to both IL-6 and CRP before, and only to IL-6 after covariate adjustment. When modeling the DSM-5 symptoms and CRP in a conceptual replication of Jokela et al. (2016), CRP was associated with ‘sleep problems’, ‘energy level’, and ‘weight/appetite changes’; only the first two links survived covariate adjustment. In a conservative model with all 38 variables, symptoms and markers were unrelated. Following recent psychometric work, we re-estimated the full model without regularization: the depressive symptoms ‘insomnia, ‘hypersomnia’, and ‘aches and pain’ showed unique positive relations to all inflammatory markers.Conclusions. We found evidence for differential relations between markers, depressive symptoms, and covariates. Associations between symptoms and markers were attenuated after covariate adjustment; BMI and sex consistently showed strong relations with inflammatory markers.

This article introduces the NetworkToolbox package for R. Network analysis offers an intuitive perspective on complex phenomena via models depicted by nodes (variables) and edges (correlations). The ability of networks to model complexity has made them the standard approach for modeling the intricate interactions in the brain. Similarly, networks have become an increasingly attractive model for studying the complexity of psychological and psychopathological phenomena. NetworkToolbox aims to provide psychological researchers with state-of-the-art methods and measures for estimating and analyzing brain, cognitive, and psychometric data. In this article, I introduce NetworkToolbox and provide a manual for applying the package’s functions to psychometric (personality) data.

Psychological data often violate the normality assumptions made by commonly used statistical methods. These violations are addressed in a variety of ways such as transformations or assuming the employed method is robust to violations. Here we argue that data transformations are unnecessary at best and severely misleading at worst. An alternative approach is to use a Bayesian model that permits skewness and other perturbations to classical assumptions (e.g., heteroskedasticity). Through simulation, we demonstrate that a Bayesian skew-normal model has optimal frequentist properties (i.e., "type 1" error, "power", unbiasedness) compared to normal-assumptive models with or without transformation. Furthermore, the Bayesian skew-normal model has greater predictive utility, as indicated by posterior predictive checking and approximate leave-one-out cross-validation. After an applied example, we discuss practical implications of our findings for psychological science in general, and specifically how Bayesian modeling can improve reproducibility in psychology.

A test score on a psychological test is usually expressed as a normed score, representing its position relative to test scores in a reference population. These typically depend on predictor(s) such as age. The test score distribution conditional on predictors is estimated using regression, which may need large normative samples to estimate the relationships between the predictor(s) and the distribution characteristics properly. In this study, we examine to what extent this burden can be alleviated by using prior information in the estimation of new norms with Bayesian Gaussian distributional regression. In a simulation study, we investigate to what extent this norm estimation is more efficient and how robust it is to prior model deviations. We varied the prior type, prior misspecification and sample size. In our simulated conditions, using a fixed effects prior resulted in more efficient norm estimation than a weakly informative prior as long as the prior misspecification was not age dependent. With the proposed method and reasonable prior information, the same norm precision can be achieved with a smaller normative sample, at least in empirical problems similar to our simulated conditions. This may help test developers to achieve cost-efficient high-quality norms. The method is illustrated using empirical normative data from the IDS-2 intelligence test.

Network psychometrics is undergoing a time of methodological reflection. In part, this was spurred by the revelation that l1-regularization does not reduce spurious associations in partial correlation networks. In this work, we address another motivation for the widespread use of regularized estimation: the thought that it is needed to mitigate overfitting. We first clarify important aspects of overfitting and the bias-variance tradeoff that are especially relevant for the network literature, where the number of nodes or items in a psychometric scale are not largecompared to the number of observations (i.e., a low p/n ratio). This revealed that bias and especially variance are most problematic in p=n ratios rarely encountered. We then introduce a nonregularized method, based on classical hypothesis testing, that fulfills two desiderata: (1) reducing or controlling the false positives rate and (2) quelling concerns of overfitting by providing accurate predictions. These were the primary motivations for initially adopting the graphical lasso (glasso). In several simulation studies, our nonregularized method provided more than competitive predictive performance, and, in many cases, outperformed glasso. Itappears to be nonregularized, as opposed to regularized estimation, that best satisfies these desiderata. We then provide insights into using our methodology. Here we discuss the multiple comparisons problem in relation to prediction: stringent alpha levels, resulting in a sparse network, can deteriorate predictive accuracy. We end by emphasizing key advantages of our approach that make it ideal for both inference and prediction in network analysis.

This dissertation deals with the problem of modeling psychopathology. Its first part focuses on statistical (data) models and introduces a number of models for cross-sectional and time series data that can be visualized as a network. This includes Mixed Graphical Models (MGMs), which allow one to include variables of different types in a statistical network model, Moderated Network Models (MNMs) which allow pairwise interactions to be moderated by other variables in the model, and time-varying Vector Autoregressive (VAR) models and MGMs that relax the standard assumption of stationarity. In addition, I discuss several methodological issues related to statistical network models such as the importance of considering predictability, model selection between AR and VAR models, and how the interpretation of the Ising model depends on its domain. The second part focuses on formal theories of psychopathology and how to develop them using data models. I first illustrate the fundamental difficulties in obtaining a formal theory with a purely statistical approach, by trying to recover an assumed bistable system for emotion dynamics with currently popular time series analyses. Next, I present a formal theory of panic disorder, based on an extensive review of the literature on the phenomenology of panic disorder and existing theories. Finally, I discuss three different ways to use data models to construct formal theories about psychopathological phenomena. Based on this discussion, I put forward an abductive framework for constructing formal theories for psychological and psychopathological phenomena.

A wide variety of effect size indices have been proposed for quantifying the magnitude of treatment effects in single-case designs. Commonly used measures include parametric indices such as the standardized mean difference as well as nonoverlap measures such as the percentage of nonoverlapping data, improvement rate difference, and nonoverlap of all pairs. Currently, little is known about the properties of these indices when applied to behavioral data collected by systematic direct observation, even though systematic direct observation is the most common method for outcome measurement in single-case research. This study uses Monte Carlo simulation to investigate the properties of several widely used single-case effect size measures when applied to systematic direct observation data. Results indicate that the magnitude of the nonoverlap measures and of the standardized mean difference can be strongly influenced by procedural details of the study's design, which is a significant limitation to using these indices as effect sizes for meta-analysis of single-case designs. A less widely used parametric index, the log response ratio, has the advantage of being insensitive to sample size and observation session length, although its magnitude is influenced by the use of partial interval recording. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

State trace analyses assess the latent dimensionality of a cognitive process by asking whether the means of two dependent variables conform to a monotonic function across a set of conditions. Using an assumption of independence between the measures, recently proposed statistical tests address bivariate measurement error, allowing both frequentist and Bayesian analyses of monotonicity (e.g., Davis-Stober, Morey, Gretton, & Heathcote, 2016; Kalish, Dunn, Burdakov, & Sysoev, 2016). However, statistical inference can be biased by unacknowledged dependencies between measures, particularly when the data are insufficient to overwhelm an incorrect prior assumption of independence. To address this limitation, we developed a hierarchical Bayesian model that explicitly models the separate roles of subject, item, and trial-level dependencies between two measures. Assessment of monotonicity is then performed by fitting separate models that do or do not allow a non-monotonic relation between the condition effects (i.e., same versus different rank orders). The Widely Applicable Information Criterion (WAIC) and Pseudo Bayesian Model Averaging – both cross validation measures of model fit – are used for model comparison, providing an inferential conclusion regarding the dimensionality of the latent psychological space. We validated this new state trace analysis technique using model recovery simulation studies, which assumed different ground truths regarding monotonicity and the direction/magnitude of the subject- and trial-level dependence. We also provide an example application of this new technique to a visual object learning study that compared performance on a visual retrieval task (forced choice part recognition) versus a verbal retrieval task (cued recall).

A majority of statistically educated scientists draw incorrect conclusions based on the most commonly used statistical technique: null hypothesis significance testing (NHST). Frequentist techniques are often claimed to be incorrectly interpreted as Bayesian outcomes, which suggests that a Bayesian framework may fit better to inferences researchers frequently want to make (Briggs, 2012). The current study set out to test this proposition. Firstly, we investigated whether there is a discrepancy between what researchers think they can conclude and what they want to be able to conclude from NHST. Secondly, we investigated to what extent researchers want to incorporate prior study results and their personal beliefs in their statistical inference. Results show the expected discrepancy between what researchers think they can conclude from NHST and what they want to be able to conclude. Furthermore, researchers were interested in incorporating prior study results, but not their personal beliefs, into their statistical inference.

Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.

This paper presents a new idea to understand human's thoughts and behaviors. Here, we propose a tentative theory of human's thoughts and behaviors, including the relativity of the thoughts, the relativity of the behaviors with thoughts, the inertia of the behaviors with thoughts and its corollary, the spiral advance of the human's thoughts, the procedure of the behaviors with thoughts, and a universal identity. This paper aims to provide a new method to know ourselves and lead human's life better. We design some experiments to check the theory lastly.

The analysis of longitudinal neuroimaging data within the massively univariate framework provides the opportunity to study empirical questions about neurodevelopment. Missing outcome data are an all-to-common feature of any longitudinal study, a feature that, if handled improperly, can reduce statistical power and lead to biased parameter estimates. The goal of this paper is to provide conceptual clarity of the issues and non-issues that arise from analyzing incomplete data in longitudinal studies with particular focus on neuroimaging data. This paper begins with a review of the hierarchy of missing data mechanisms and their relationship to likelihood-based methods, a review that is necessary not just for likelihood-based methods, but also for multiple-imputation methods. Next, the paper provides a series of simulation studies with designs common in longitudinal neuroimaging studies to help illustrate missing data concepts regardless of interpretation. Finally, two applied examples are used to demonstrate the sensitivity of inferences under different missing data assumptions and how this may change the substantive interpretation. The paper concludes with a set of guidelines for analyzing incomplete longitudinal data that can improve the validity of research findings in developmental neuroimaging research. Keywords: Neuroimaging, Missing data, Likelihood, Longitudinal data

An important tool in the advancement of cognitive science are quantitative models that represent different cognitive variables in terms of model parameters. To evaluate such models, their parameters are typically tested for relationships with behavioral and physiological variables that are thought to reflect specific cognitive processes. However, many models do not come equipped with the statistical framework needed to relate model parameters to covariates. Instead, researchers often revert to classifying participants into groups depending on their values on the covariates, and subsequently comparing the estimated model parameters between these groups. Here we develop a comprehensive solution to the covariate problem in the form of a Bayesian regression framework. Our framework can be easily added to existing cognitive models and allows researchers to quantify the evidential support for relationships between covariates and model parameters using Bayes factors. Moreover, we present a simulation study that demonstrates the superiority of the Bayesian regression framework to the conventional classification-based approach.

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

This Tutorial explains the statistical concept known as likelihood and discusses how it underlies common frequentist and Bayesian statistical methods. The article is suitable for researchers interested in understanding the basis of their statistical tools and is also intended as a resource for teachers to use in their classrooms to introduce the topic to students at a conceptual level.

Evaluation of model fit is critically important for every structural equation model (SEM), and sophisticated methods have been developed for this task. Among them are the χ² goodness-of-fit test, decomposition of the χ², derived measures like the popular root mean square error of approximation (RMSEA) or comparative fit index (CFI), or inspection of residuals or modification indices. Many of these methods provide a global approach to model fit evaluation: A single index is computed that quantifies the fit of the entire SEM to the data. In contrast, graphical criteria like d-separation or trek-separation allow derivation of implications that can be used for local fit evaluation, an approach that is hardly ever applied. We provide an overview of local fit evaluation from the viewpoint of SEM practitioners. In the presence of model misfit, local fit evaluation can potentially help in pinpointing where the problem with the model lies. For models that do fit the data, local tests can identify the parts of the model that are corroborated by the data. Local tests can also be conducted before a model is fitted at all, and they can be used even for models that are globally underidentified. We discuss appropriate statistical local tests, and provide applied examples. We also present novel software in R that automates this type of local fit evaluation. (PsycINFO Database Record

The topic of replicability has recently captivated the emerging field of networkpsychometrics. Although methodological practice (e.g., p-hacking) has been identified as aroot cause of unreliable research findings in psychological science, the statistical modelitself has come under attack in the partial correlation network literature. In a motivatingexample, I first describe how sampling variability inherent to partial correlations can merelygive the appearance of unreliability. For example, when going from zero-order to partialcorrelations there is necessarily more sampling variability that translates into reducedstatistical power. I then introduce novel methodology for deriving expected networkreplicability (ENR), wherein replication is modeled with the Poisson-binomial distribution.This analytic solution can be used with the Pearson, Spearman, Kendall, and polychoricpartial correlation coefficient. I first employed the method to estimate ENR for a variety ofdatasets from the network literature. Here it was determined that partial correlationnetworks do not have inherent limitations, given current estimates of replicability wereconsistent with ENR. I then highlighted sources that can reduce replicability, that is, whengoing from continuous to ordinal data with few categories and employing a multiplecomparisons correction. To address these challenges, I described a strategy for using theproposed method to plan for network replication. I end with recommendations that includethe importance of the network literature repositioning itself with gold-standard approachesfor assessing replication, including explicit consideration of type I and type II error rates.The method for computing ENR is implemented in the R package GGMnonreg.

Psychometric models for longitudinal test scores typically estimate quantities associated with single-administration tests, like ability at each time-point. However, models for longitudinal tests have not considered opportunities to estimate new quantities that are unavailable from single-administration tests. Specifically, we discuss dynamic measurement models – which combine aspects of longitudinal IRT, nonlinear growth models, and dynamic assessment – to directly estimate capacity, defined as the expected future score once the construct has fully developed. After discussing the history and connecting these areas into a single framework, we apply the model to verbal test scores from the Berkeley Growth Study, which follows 494 people from 3 to 72 years old. The goal is to predict adult verbal scores (Age ≥ 34) from adolescent scores (Age ≤ 20). We held-out the adult data for prediction and compared predictions from traditional longitudinal IRT ability scores and proposed dynamic measurement capacity scores from models fit to the adolescent data. Results showed that the R2 from capacity scores were 2.5 times larger than the R2 from longitudinal IRT ability scores (43% vs. 16%), providing some evidence that exploring new quantities available from longitudinal testing could be worthwhile when an interest in testing is forecasting future performance.

Gaussian graphical models (GGM) allow for learning conditional independence structures that are encoded by partial correlations. Whereas there are several \proglang{R} packages for classical (i.e., frequentist) methods, there are only two that implement a Bayesian approach. These are exclusively focused on identifying the graphical structure; that is, detecting non-zero effects. The \proglang{R} package \pkg{BGGM} not only fills this gap, but it also includes novel Bayesian methodology for extending inference beyond identifying non-zero relations. \pkg{BGGM} is built around two Bayesian approaches for inference--estimation and hypothesis testing. The former focuses on the posterior distribution and includes extensions to assess predictability, as well as methodology to compare partial correlations. The latter includes methods for Bayesian hypothesis testing, in both exploratory and confirmatory contexts, with the novel matrix-$F$ prior distribution. This allows for testing order and equality constrained hypotheses, as well as a combination of both with the Bayes factor. Further, there are two approaches for comparing any number of GGMs with either the posterior predictive distribution or Bayesian hypothesis testing. This work describes the software implementation of these methods. We end by discussing future directions for \pkg{BGGM}.

The study of memory for texts has had an long tradition of research in psychology. According to most general accounts, the recognition or recall of items in a text is based on querying a memory representation that is built up on the basis of background knowledge. The objective of this paper is to describe and thoroughly test a Bayesian model of these general accounts. In particular, we present a model that describes how we use our background knowledge to form memories in terms of Bayesian inference of statistical patterns in the text, followed by posterior predictive inference of the words that are typical of those inferred patterns. This provides us with precise predictions about which words will be remembered, whether veridically or erroneously, from any given text. We tested these predictions using behavioural data from a memory experiment using a large sample of randomly chosen texts from a representative corpus of British English. The results show that the probability of remembering any given word in the text, whether falsely or veridically, is well predicted by the Bayesian model. Moreover, compared to nontrivial alternative models of text memory, by every measure used in the analyses, the predictions of the Bayesian model were superior, often overwhelmingly so. We conclude that these results provide strong evidence in favour of the Bayesian account of text memory that we have presented in this paper.

Utility functions are heavily used for modeling economic decision-making such as intertemporal and risky choice. Researchers have used both parametric and non-parametric approaches to model utility functions. Parametric models are simple and easy to fit, but they often require formal assumptions that can be hard to justify, especially given the myriad of different models that exist with no consensus. Non-parametric approaches require relatively large datasets and specially designed procedures to elicit the utility functions. Here we present a novel method – cubic Bezier splines (CBS) – as an ‘agnostic’ model that approximates utility functions without rigid formal assumptions. CBS is flexible like non-parametric approaches but does not require large or specially designed datasets. CBS provides smooth functions like parametric models but without many of their assumptions. We demonstrate via simulation and real data that CBS accurately recovers a diverse variety of utility models and shows higher predictive accuracy than alternatives. We also demonstrate that CBS can detect important heterogeneity and consistent granular patterns in choice data that cannot be detected with many parametric models.

A Gibbs sampler for the hierarchical Dirichlet process mixture model (HDPMM) when used with multinomial data.

Gaussian graphical models are commonly used to characterize conditional (in)dependence structures (i.e., partial correlation networks) of psychological constructs. Recently attention has shifted from estimating single networks to those from various subpopulations. The focus is primarily to detect differences or demonstrate replicability. We introduce two novel Bayesian methods for comparing networks that explicitly address these aims. The first is based on the posterior predictive distribution, with a symmetric version of Kullback-Leibler divergence as the discrepancy measure, that tests differences between two (or more) multivariate normal distributions. The second approach makes use of Bayesian model comparison, with the Bayes factor, and allows for gaining evidence for invariant network structures. This overcomes limitations of current approaches in the literature that use classical hypothesis testing, where it is only possible to determine whether groups are significantly different from each other. With simulation we show the posterior predictive method is approximately calibrated under the null hypothesis (alpha = .05) and has more power to detect differences than alternative approaches. We then examine the necessary sample sizes for detecting invariant network structures with Bayesian hypothesis testing, in addition to how this is influenced by the choice of prior distribution. The methods are applied to posttraumatic stress disorder symptoms that were measured in 4 groups. We end by summarizing our major contribution, that is proposing 2 novel methods for comparing Gaussian graphical models (GGMs), which extends beyond the social-behavioral sciences. The methods have been implemented in the R package BGGM.Translational AbstractGaussian graphical models are becoming popular in the social-behavioral sciences. Recently attention has shifted from estimating single networks to those from various subpopulations (e.g., males vs. females). We introduce Bayesian methodology for comparing networks estimated from any number of groups. The first approach is based on the posterior predictive distribution and it allows for determining whether networks are different from one another. This is ideal for testing the null hypothesis of group equality, say, in the context of testing for network replicability (or lack thereof). The second approach is based on Bayesian hypothesis testing and it allows for gaining evidence for network invariances or equality of partial correlations for any number of groups. This is ideal for focusing on specific aspects of the network such as individual partial correlations. In a series of simulations and illustrative examples we demonstrate the utility of the proposed methodology for comparing Gaussian graphical models. The methods have been implemented in the R package BGGM.

BGGM is built around two approaches for Bayesian inference–estimation and hypothesis testing. This distinction is arbitrary (see Rouder, Haaf, & Vandekerckhove, 2018), but is used to organize this work. The former focuses on the posterior distribution and includes extensions to assess predictability (Haslbeck & Waldorp, 2018), as well as methodology to compare partial correlations. The latter includes methods for Bayesian hypothesis testing, in both exploratory and confirmatory contexts, with the novel matrix-F prior distribution (Mulder & Pericchi, 2018). This allows for testing the null hypothesis of conditional independence, as well as inequality and equality constrained hypotheses. Further, there are several approaches for comparing GGMs across any number of groups. The package also includes a suite of options for model checking. Together, BGGM is a comprehensive toolbox for Gaussian graphical modeling in R.

BayeFit is a module for Python that allows users to fit, plot, and extract parameter estimates from models of psychophysical data using Bayesian inference. The module makes extensive use of the module PyStan, which in turn makes extensive use of the package Stan, a powerful tool for Bayesian inference using a No-U-Turn sampler. Although PyStan provides an interface between Python and Stan, defining models for use with PyStan can be difficult for most users given the models are defined in C++ language. BayesFit removes this barrier to entry by providing researchers with functions that streamline the process of defining psychophysical models, obtaining fits, extracting outputs, and visualizing fitted models. The source code for BayesFit is available at https://github.com/slugocm/bayesfit. This module can be expanded and reused as newer versions of Python are developed to ensure researchers always have a tool available to ease the process of fitting models to psychophysical data.

Gaussian graphical models (GGM; partial correlation networks) have become increasingly popular in the social and behavioral sciences for studying conditional (in)dependencies between variables. In this work, we introduce exploratory and confirmatory Bayesian tests for partial correlations. For the former, we first extend the customary GGM formulation that focuses on conditional dependence to also consider the null hypothesis of conditional independence for each partial correlation. Here a novel testing strategy is introduced that can provide evidence for a null, negative, or positive effect. We then introduce a test for hypotheses with order constraints on partial correlations. This allows for testing theoretical and clinical expectations in GGMs. The novel matrix$-F$ prior distribution is described that provides increased flexibility in specification compared to the Wishart prior. The methods are applied to PTSD symptoms. In several applications, we demonstrate how the exploratory and confirmatory approaches can work in tandem: hypotheses are formulated from an initial analysis and then tested in an independent dataset. The methodology is implemented in the R package BGGM.

With the development of increasingly automated vehicles (AVs) comes the increasingly difficult challenge of comprehensively validating these for acceptable, and ideally beneficial, impacts on the transport system. There is a growing consensus that virtual testing, where simulated AVs are deployed in simulated traffic, will be key for cost-effective testing and optimization. The least mature model components in such simulations are those generating the behavior of human agents in or around the AVs. In this paper, human models and virtual testing applications are presented for two example scenarios: (i) a human pedestrian deciding whether to cross a street in front of an approaching automated vehicle, with or without external human–machine interface elements, and (ii) an AV handing over control to a human driver in a critical rear-end situation. These scenarios have received much recent research attention, yet simulation-ready human behavior models are lacking. They are discussed here in the context of existing models of perceptual decision-making, situational awareness, and traffic interactions. It is argued that the human behavior in question might be usefully conceptualized as a number of interrelated decision processes, not all of which are necessarily directly associated with externally observable behavior. The results show that models based on this type of framework can reproduce qualitative patterns of behavior reported in the literature for the two addressed scenarios, and it is demonstrated how computer simulations based on the models, once these have been properly validated, could allow prediction and optimization of AV impacts on traffic flow and traffic safety.

Background Participant retention strategies that minimise attrition in longitudinal cohort studies have evolved considerably in recent years. This study aimed to assess, via systematic review and meta-analysis, the effectiveness of both traditional strategies and contemporary innovations for retention adopted by longitudinal cohort studies in the past decade. Methods Health research databases were searched for retention strategies used within longitudinal cohort studies published in the 10-years prior, with 143 eligible longitudinal cohort studies identified (141 articles; sample size range: 30 to 61,895). Details on retention strategies and rates, research designs, and participant demographics were extracted. Meta-analyses of retained proportions were performed to examine the association between cohort retention rate and individual and thematically grouped retention strategies. Results Results identified 95 retention strategies, broadly classed as either: barrier-reduction, community-building, follow-up/reminder, or tracing strategies. Forty-four of these strategies had not been identified in previous reviews. Meta-regressions indicated that studies using barrier-reduction strategies retained 10% more of their sample (95%CI [0.13 to 1.08]; p = .01); however, studies using follow-up/reminder strategies lost an additional 10% of their sample (95%CI [− 1.19 to − 0.21]; p = .02). The overall number of strategies employed was not associated with retention. Conclusions Employing a larger number of retention strategies may not be associated with improved retention in longitudinal cohort studies, contrary to earlier narrative reviews. Results suggest that strategies that aim to reduce participant burden (e.g., flexibility in data collection methods) might be most effective in maximising cohort retention. Electronic supplementary material The online version of this article (10.1186/s12874-018-0586-7) contains supplementary material, which is available to authorized users.

We present a mixed-effects location scale model (MELSM) for examining the daily dynamics of affect in dyads. The MELSM includes person and time-varying variables to predict the location, or individual means, and the scale, or within-person variances. It also incorporates a submodel to account for between-person variances. The dyadic specification can accommodate individual and partner effects in both the location and the scale components, and allows random effects for all location and scale parameters. All covariances among the random effects, within and across the location and the scale are also estimated. These covariances offer new insights into the interplay of individual mean structures, intra-individual variability, and the influence of partner effects on such factors. To illustrate the model, we use data from 274 couples who provided daily ratings on their positive and negative emotions toward their relationship - up to 90 consecutive days. The model is fit using Hamiltonian Monte Carlo methods, and includes subsets of predictors in order to demonstrate the flexibility of this approach. We conclude with a discussion on the usefulness and the limitations of the MELSM for dyadic research.

A new system for subjective rating of responses to divergent thinking tasks was tested using raters recruited from Amazon Mechanical Turk. The rationale for the study was to determine if such raters could provide reliable (aka generalizable) ratings from the perspective of generalizability theory. To promote reliability across the Alternative Uses and Consequence task prompts often used by researchers as measures of Divergent Thinking, two parallel scales were developed to facilitate feasibility and validity of ratings performed by laypeople. Generalizability and dependability studies were conducted separately for two scoring systems: the average-rating system and the snapshot system. Results showed that it is difficult to achieve adequate reliability using the snapshot system, while good reliability can be achieved on both task families using the average-rating system and a specific number of items and raters. Additionally, the construct validity of the average-rating system is generally good, with less validity for certain Consequences items. Recommendations for researchers wishing to adopt the new scales are discussed, along with broader issues of generalizability of subjective creativity ratings.

Using complete enumeration (e.g., generating all possible subsets of item combinations) to evaluate clustering problems has the benefit of locating globally optimal solutions automatically without the concern of sampling variability. The proposed method is meant to combine clustering variables in such a way as to create groups that are maximally different on a theoretically sound derivation variable(s). After the population of all unique sets is permuted, optimization on some predefined, user-specific function can occur. We apply this technique to optimizing the diagnosis of Alcohol Use Disorder. This is a unique application, from a clustering point of view, in that the decision rule for clustering observations into the diagnosis group relies on both the set of items being considered and a predefined threshold on the number of items required to be endorsed for the diagnosis to occur. In optimizing diagnostic rules, criteria set sizes can be reduced without a loss of significant information when compared to current and proposed, alternative, diagnostic schemes.

Reliability is a fundamental problem for measurement in all of science. Al- though defined in multiple ways, and estimated in even more ways, the basic concepts are straight forward and need to be understood by methodologists as well as practioners. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. Easy to use, open source software is applied to examples of real data, and comparisons are made between the many types of reliability

The present study examined the dimensional structure underlying the Multidimensional Schizotypy Scale (MSS) and its brief version (MSS-B). The MSS and MSS-B were developed to assess current multidimensional conceptualizations of schizotypy. We used Exploratory Graph Analysis (EGA) to evaluate the dimensional structure of the scales in two large, independent samples (n = 6,265 and n = 1,000). We then used Confirmatory Factor Analysis (CFA) to compare the fit of the theoretical dimensions with the EGA dimensions. For the MSS, EGA identified four dimensions: positive schizotypy, two dimensions of negative schizotypy (affective and social anhedonia), and disorganized schizotypy. For the MSS-B, EGA identified three dimensions, which corresponded to the theorized positive, negative, and disorganized dimensions. Based on the MSS’s EGA dimensions, we also estimated a four-factor model for the MSS-B. The CFA comparison found that the four-factor model fit significantly better than the theoretical three-factor model for both the MSS and MSS-B. In short, we propose that the four-factor model supports the theoretical model and offers a more nuanced interpretation of the negative schizotypy dimension. Our findings offer new implications for future research on the MSS and MSS-B dimensions that may provide differential associations with interview and questionnaire measures.

Individual differences in general speed lead to a positive correlation between the mean and standard deviation of mean latency. This “coarse” scaling effect causes the mean latency difference (MLD) to be spuriously correlated with general speed. Within individuals, the correlation between the mean and standard deviation of trial latencies leads contrasted distributions to increase their overlap as an MLD of fixed width is translated to the right. To address this “fine” scaling effect, contrasts based on within subject latency transformations including the logarithm, standardization, and ranking were evaluated and turned out to be distinctly superior to the MLD. Notably, the mean gaussian rank latency difference was internally consistent, eliminated fine scaling, meliorated coarse scaling, reduced correlations with general speed, increased statistical power to detect within subject and between group effects, and has the potential to increase the validity of inferences drawn from response latency data.

Cross-validation (CV) is increasingly popular as a generic method to adjudicate between mathematical models of cognition and behavior. In order to measure model generalizability, CV quantifies out-of-sample predictive performance, and the CV preference goes to the model that predicted the out-of-sample data best. The advantages of CV include theoretic simplicity and practical feasibility. Despite its prominence, however, the limitations of CV are often underappreciated. Here we demonstrate the limitations of a particular form of CV --Bayesian leave-one-out cross-validation or LOO-- with three concrete examples. In each example, a data set of infinite size is perfectly in line with the predictions of a simple model (i.e., a general law or invariance). Nevertheless, LOO shows bounded and relatively modest support for the simple model. We conclude that CV is not a panacea for model selection.

Creativity assessment with open-ended production tasks relies heavily on scoring the quality of a subject’s ideas. This creates a faceted measurement structure involving persons, tasks (and ideas within tasks), and raters. Most studies, however, do not model possible systematic differences among raters. The present study examines individual rater differences in the context of a planned-missing design and its association with reliability and validity of creativity assessments. It applies the many-facet Rasch model (MFRM) to model and correct for these differences. We reanalyzed data from 2 studies (Ns = 132 and 298) where subjects produced metaphors, alternate uses for common objects, and creative instances. Each idea was scored by several raters. We simulated several conditions of reduced load on raters where they scored subsets of responses. We then compared the reliability and validity of IRT estimated scores (original vs. IRT adjusted scores) on various conditions of missing data. Results show that (a) raters vary substantially on the lenient-severity dimension, so rater differences should be modeled; (b) when different combinations of raters assess different subsets of ideas, systematic rater differences confound subjects’ scores, increasing measurement error and lowering criterion validity with external variables; and (c) MFRM adjustments effectively correct for rater effects, thus increasing correlations of scores obtained from partial with scores obtained with full data. We conclude that MFRM is a powerful means to model rater differences and reduce rater load in creativity research.

Multiple-choice and continuous-response tasks pose a number of challenges for models of the decision process, from empirical challenges like context effects to computational demands imposed by choice sets with a large number of outcomes. This paper develops a general framework for constructing models of the cognitive processes underlying both inferential and preferential choice among any arbitrarily large number of alternatives. This geometric approach represents the alternatives in a choice set along with a decision maker's beliefs or preferences in a ``decision space,'' simultaneously capturing the support for different alternatives along with the similarity relations between the alternatives in the choice set. Support for the alternatives (represented as vectors) shifts over time according to the dynamics of the belief / preference state (represented as a point) until a stopping rule is met (state crosses a hyperplane) and the corresponding selection is made. I review stopping rules that guarantee optimality in multi-alternative inferential choice, minimizing response time for a desired level of accuracy, as well as methods for constructing the decision space, which can result in context effects when applied to preferential choice.

Mediation analysis plays a central role in the investigation of causal mechanisms, where the total, direct and indirect influence of one variable on another is typically assessed using path-tracing rule}. Two recent developments promise to revolutionize mediation analysis: first, the interventionist causal inference framework has allowed researchers to define and investigate mediation in terms of interventions on variables rather than path-tracing rules; second, researchers have begun to use Continuous-Time (CT) models to investigate mediation in longitudinal data. This paper shows how CT mediation can be treated using an interventionist approach. The links between interventionist and path-tracing definitions in traditional longitudinal SEM models and in CT models are explored in detail. The treatment of mediation from diverse strands of literature is unified, by showing how total, direct and indirect effects in different frameworks can be defined as qualitatively different types of interventions on the values of variables.

Stress is considered to be among the leading causes of various psychopathological problems, whose outcomes are often difficult to foresee. In order to prevent consequent maladaptive behaviors, the present paper introduces a novel, three-phase architecture for analyzing the formation, handling and manifestation of stress in individuals. We specifically model the process of transforming stressful stimuli to cognitive schemas, and from these to subconscious defense mechanism manifestations. Such three-phase model has been implemented and validated via the PERSONA-80 test. Psychometric properties of PERSONA-80 have been evaluated on a sample of 83 individuals from general and clinical populations. The test showed good internal consistency and excellent predictive validity results, from which we derived the purified scale PERSONA-16. The potential of the developed instrument is the detection of the inability to produce mature defense mechanisms from an arbitrary state of mind, or the inability to preserve schemas when exposed to stressful conditions, useful in high-stress occupation screening and monitoring. To encourage widespread use and validation in different domains, we provide at the end of this text the complete PERSONA-16 instrument.

Psychological constructs, such as emotions, thoughts, and attitudes are often measured by asking individuals to reply to questions using closed-ended numerical rating scales. However, when asking people about their state of mind in a natural context ("How are you?"), we receive open-ended answers using words ("Fine and happy!") and not closed-ended answers using numbers ("7") or categories ("A lot"). Nevertheless, to date it has been difficult to objectively quantify responses to open-ended questions. We develop an approach using open-ended questions in which the responses are analyzed using natural language processing (Latent Semantic Analyses). This approach of using open-ended, semantic questions is compared with traditional rating scales in nine studies (N = 92-854), including two different study paradigms. The first paradigm requires participants to describe psychological aspects of external stimuli (facial expressions) and the second paradigm involves asking participants to report their subjective well-being and mental health problems. The results demonstrate that the approach using semantic questions yields good statistical properties with competitive, or higher, validity and reliability compared with corresponding numerical rating scales. As these semantic measures are based on natural language and measure, differentiate, and describe psychological constructs, they have the potential of complementing and extending traditional rating scales. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Paper details and compares 6 different modifications to the k-means algorithm on simulated data and applies disparate algorithms to a college risk behavior dataset to understand risk groups among college populations. Study was MS thesis at University of Miami, 2014.

We investigated the latent structure of narcissistic personality disorder by comparing dimensional, hybrid, and categorical latent variable models, using confirmatory factor analysis (CFA), nonparametric and semiparametric factor analysis, and latent class analysis, respectively. We first explored these models in a clinical sample and then preregistered replication analyses in 4 additional data sets (with national, undergraduate, community, and mixed community/clinical samples) to test whether the best-fitting model would generalize across different data sets with different sample compositions. A 1-factor CFA outperformed categorical models in fit and reliability, suggesting the criteria do not serve to distinguish a narcissist class or subtypes; rather, a narcissistic dimension underlies the narcissistic personality disorder construct. The CFA also outperformed hybrid models, indicating that people fall within the same continuous distribution, rather than composing homogenous groups of relative severity (nonparametric factor analysis) or pulling apart into mixtures of discrete distributions (semiparametric factor analysis) along that spectrum. (PsycINFO Database Record

Psychologists wrestle with how to best handle multiple comparisons, while maintaining a balance between false positives and false negatives. Undercorrection, such as ignoring the presence of multiple comparisons altogether, is known to yield an unacceptably high rate of false positives. Overcorrection, such as treating all tests as independent when they are not, results in overly conservative evaluations of statistical significance. This tutorial demonstrates $M_{eff}$ correction, a method for adjusting statistical significance thresholds for multiple comparisons, without the assumption of independence of tests. This method, in which the effective number of tests ($M_{eff}$) is estimated from the correlations among the variables being tested, was developed and validated in the field of genetics, but is based on statistical concepts (eigenvalues) that are very familiar to psychologists. $M_{eff}$ correction can be applied in psychological research to balance the necessity of correction for multiple comparisons with the concerns that arise from complex, correlated tests.

An important goal for psychological science is developing methods to characterize relationships between variables. The customary approach uses structural equation models to connect latent factors to a number of observed measurements. More recently, regularized partial correlation networks have been proposed as an alternative approach for characterizing relationships among variables through covariances in the precision matrix. While the graphical lasso (glasso) method has merged as the default network estimation method, it was optimized in fields outside of psychology with very different needs, such as high dimensional data where the number of variables (p) exceeds the number of observations (n). In this paper, we describe the glasso method in the context of the fields where it was developed, and then we demonstrate that the advantages of regularization diminish in settings where psychological networks are often fitted (p ≪ n). We first show that improved properties of the precision matrix, such as eigenvalue estimation, and predictive accuracy with cross-validation are not always appreciable. We then introduce non-regularized methods based on multiple regression, after which we characterize performance with extensive simulations. Our results demonstrate that the non-regularized methods consistently outperform glasso with respect to limiting false positives, and they provide more consistent performance across sparsity levels, sample composition (p=n), and partial correlation size. We end by reviewing recent findings in the statistics literature that suggest alternative methods often have superior than glasso, as well as suggesting areas for future research in psychology.

*THIS PAPER HAS NOT YET BEEN PEER REVIEWED* Listening to music is a strategy many people use to regulate their emotions, especially sadness. However, there is disagreement about whether listening to music is a healthy way to regulate emotions, with some research finding that sad music worsens a sad state, especially for people high in rumination. To further explore the immediate consequences of music listening when sad 128 young adults (41% male, aged 18 to 25 years) were induced into a sad emotional state prior to random assignment to listening of either self-selected music, experimenter-selected sad music, or no music. Results revealed that listening to either self-selectedor experimenter-selected music led to a decrease in sadness. No difference was found between groups at post-listening. However, participants who listened to self-selected music reported a return to baseline levels of sadness, while this did not occur for participants who listened to experimenter-selected or were in the no music control. Rumination was also measured but did not moderate the impact of music listening on sadness for either musiccondition. Furthermore, there was no impact of rumination on participants’ perceptions of sadness in music. These results support the notion that listening to sad music does not worsen a sad state—even for those high in rumination—although it does appear to slow the emotion regulation process in cases where sad music is not self-selected.

Scholars have often characterized wisdom as a trait—a stable and invariable individual disposition. However, recent advances in behavioral sciences suggest that many features of wisdom can be viewed as situation-specific states—concrete instances of wise and not so wise behavior. We critically examine the evidence concerning the trait vs. state aspects of wisdom. First, we review the common individual qualities attributed to wisdom. The extent to which these qualities are seen as stable traits vs. variable states varies among both laypeople and scientists, and new evidence shows that pertinent folk theories about wisdom vary substantially across cultures as well. Next, we present empirical research on wisdom in everyday situations which demonstrates the systematic variability of wisdom-related characteristics as a function of situational demands and induced mindsets. To resolve the trait vs. state debate in wisdom, we argue for a whole trait theoretical framework, conceptualizing traits as a density distribution of states. Based on theoretical and empirical insights, we conclude by providing recommendations for best practices when measuring trait- and state-components of wisdom.

As the debate over best statistical practices continues in academic journals, conferences, and the blogosphere, working researchers (e.g., psychologists) need to figure out how much time and effort to invest in attending to experts’ arguments, how to design their next project, and how to craft a sustainable long-term strategy for data analysis and inference. The present special issue of the American Statistician promises help. In this article, we offer a modest proposal for a continued and informed use of the conventional p value without the pitfalls of statistical rituals. Other statistical indices should complement reporting, and extra-statistical (e.g., theoretical) judgments ought to be made with care and clarity.