Search

In applications involving human forecasters, proper scoring rules such as the Brier score or logarithmic score are the gold standards for forecaster evaluation. They provide incentives for forecaster honesty; they are simple to compute; they are well studied; and they provide a mechanism for forecaster selection and aggregation. But these scoring rules are somewhat inflexible and cannot immediately handle various attributes of forecasting data, including missing forecasts and nested forecasts. In light of these issues, we consider the scoring of forecasters via statistical models that are grounded in the logarithmic scoring rule. We specifically discuss the connection between model log-likelihoods and the logarithmic scoring rule, which allows us to apply theoretical results from the statistics literature to human forecaster evaluation. We use publicly-available data from the Good Judgment Project to illustrate the flexibility of model-based evaluations and to compare these evaluations to traditional approaches. We also consider implications for forecaster selection, recalibration, and aggregation.

Bayesian methods are becoming increasingly used in applied psychological research. Previous researchers have thoroughly written about much of the details already, including the philosophy underlying Bayesian methods, computational issues associated with Bayesian model estimation, Bayesian model development and summary, and the role of Bayesian methods in the so-called replication crisis. In this paper, we seek to provide case studies comparing the use of frequentist methods to the use of Bayesian methods in applied psychological research. These case studies are intended to 'illustrate by example' the ways that Bayesian modelling differs from frequentist modelling and the differing conclusions that one may arrive at using the two methods. The intended audience is applied psychological researchers who have been trained in the traditional frequentist framework, who are familiar with mixed-effects models and who are curious about how statistical results might look in a Bayesian context. Along with our case studies, we provide general opinions and guidance on the use of Bayesian methods in applied psychological research.

Multidimensional item response models are known to be difficult to estimate, with a variety of estimation and modeling strategies being proposed to handle the difficulties. While some previous studies have considered the performance of these estimation methods, they typically include only one or two methods, or a small number of factors. In this paper, we report on a large simulation study of between-item multidimensional IRT estimation methods, considering five different methods, a variety of sample sizes, and up to eight factors. This study provides a comprehensive picture of the methods’ relative performance, as well as each individual method’s strengths and weaknesses. The study results lead us to make recommendations for applied research, related to which estimation methods should be used under various scenarios.

It is well known that, in traditional SEM applications, a scale must be set for each latent variable: typically, either the latent variance or a factor loading is fixed to one. While this has no impact on the fit metrics in ML estimation, it can potentially lead to varying Bayesian model comparison metrics due to the use of different prior distributions under each parameterization. This is a problem, because a researcher could artificially improve one’s preferred model simply by changing the identification constraint. Using a single-factor CFA as motivation for study, we first show that Bayesian model comparison metrics can systematically change depending on constraints used. We then study principled methods for setting the scale of the latent variable that stabilize the model comparison metrics. These methods involve (i) the placement of priors on ratios of factor loadings, as opposed to individual loadings; and (ii) use of effect coding. We illustrate the methods via simulation and application.

Cross-level interactions among fixed effects in linear mixed models (also known as multilevel models) can be complicated by heterogeneity stemming from random effects and residuals. When heterogeneity is present, tests of fixed effects (including cross-level interaction terms) are subject to inflated type I or type II error. While the impact of variance change/heterogeneity has been noticed in the literature, few methods have been proposed to detect this heterogeneity in a simple, systematic way. In addition, when heterogeneity among clusters is detected, researchers often wish to know which clusters' variances differed from the others. In this study, we utilize a recently proposed family of score-based tests to distinguish between cross-level interactions and heterogeneity in variance components, also providing information about specific clusters that exhibit heterogeneity. These score-based tests only require estimation of the null model (when variance homogeneity is assumed to hold), and they have been previously applied to psychometric models to detect measurement invariance. In this paper, we extend the tests to linear mixed models, allowing us to use the tests to differentiate between interaction and heterogeneity. We detail the tests' implementation and performance via simulation, provide an empirical example of the tests' use in practice, and provide code illustrating the tests' general application.

Latent variable models (LVMs) are incredibly flexible tools that allow users to address research questions they might otherwise never be able to answer (McDonald, 2013). However, one major limitation of LVMs is evaluating model fit. There is no universal consensus about how to evaluate model fit, either globally or locally. Part of the reason evaluating these models is difficult is because fit is typically reduced to a handful of statistics that may or may not reflect the model’s adequacy and/or assumptions. In this paper we argue that proper evaluation of model fit must include visualizing both the raw data and the model-implied fit. Visuals reveal, at a glance, the fit of the model and whether the model’s assumptions have been met. Unfortunately, tools for visualizing LVMs have historically been limited. In this paper, we introduce new plots and reframe existing plots that provide necessary resources for evaluating LVMs. These plots are available in a new open- source R package called flexplavaan, which combines the model plotting capabilities of flexplot with the latent variable modeling capabilities oflavaan.

It is well known that, in traditional SEM applications, a scale must be set for each latent variable: either the latent variance or a factor loading is typically fixed to one. While this has no impact on the fit metrics in ML estimation, it can potentially lead to varying Bayesian model comparison metrics due to the use of different priors under each parameterization. Using a single-factor CFA as motivation for study, we first show that Bayesian model comparison metrics systematically change depending on constraints used. We then study principled methods for setting the latent variable scale that stabilize the model comparison metrics. These methods involve (i) the placement of priors on ratios of factor loadings, as opposed to individual loadings, and (ii) use of effect coding. We illustrate the methods via simulation and application.

In this paper, we apply Vuong's (1989) general approach of model selection to the comparison of nested and non-nested unidimensional and multidimensional item response theory (IRT) models. Vuong's approach of model selection is useful because it allows for formal statistical tests of both nested and non-nested models. However, only the test of non-nested models has been applied in the context of IRT models to date. After summarizing the statistical theory underlying the tests, we investigate the performance of all three distinct Vuong tests in the context of IRT models using simulation studies and real data. In the non-nested case we observed that the tests can reliably distinguish between the graded response model and the generalized partial credit model. In the nested case, we observed that the tests typically perform as well as or sometimes better than the traditional likelihood ratio test. Based on these results, we argue that Vuong's approach provides a useful set of tools for researchers and practitioners to effectively compare competing nested and non-nested IRT models., This is an original manuscript / preprint of an article published by Taylor & Francis in Multivariate Behavioral Research on the 18th September 2019, available online: http://www.tandfonline.com/10.1080/00273171.2019.1664280. 49 pages, 3 tables, 5 figures

Comparative measures such as paired comparisons and rankings are frequently used to evaluate health states and quality of life. The present article introduces log-linear Bradley-Terry (LLBT) models to evaluate intervention effectiveness when outcomes are measured as paired comparisons or rankings and presents a combination of the LLBT model and model-based recursive partitioning (MOB) to detect treatment effect heterogeneity. The MOB LLBT approach enables researchers to identify subgroups that differ in the preference order and in the effect an intervention has on choice behavior. Applicability of MOB LLBT models is demonstrated using an artificial data example with known data-generating mechanism and a real-world data example focusing on drug-harm perception among music festival visitors. In the artificial data example, the MOB LLBT model is able to adequately recover the “true” (population) model. In the real-world data example, the standard LLBT model confirms the existence of a situational willingness among festival visitors to trivialize drug harm when peer consumption behavior is made cognitively accessible. In addition, MOB LLBT results suggest that this trivialization effect is highly context-dependent and most pronounced for participants with low-to-moderate alcohol intoxication who also proactively contacted a substance counselor at the festival venue. Both data examples suggest that MOB LLBT models allow for more nuanced statements about the effectiveness of interventions. We provide R code examples to implement MOB LLBT models for paired comparisons, rankings, and rating (Likert-type) data.

Maximum likelihood estimation of generalized linear mixed models (GLMMs) is difficult due to marginalization of the random effects. Derivative computations of a fitted GLMM's likelihood are also difficult, especially because the derivatives are not by-products of popular estimation algorithms. In this paper, we first describe theoretical results related to GLMM derivatives along with a quadrature method to efficiently compute the derivatives, focusing on fitted lme4 models with a single clustering variable. We describe how psychometric results related to item response models are helpful for obtaining the derivatives, as well as for verifying the derivatives' accuracies. We then provide a tutorial on the many possible uses of these derivatives, including robust standard errors, score tests of fixed effect parameters, and likelihood ratio tests of non-nested models. The derivative computation methods and applications described in the paper are all available in easily obtained R packages.

OBJECTIVE: Alcohol-impaired driving is a significant public safety concern and is highly prevalent among young adults. Considerable research has examined between-person predictors of alcohol-impaired driving, but there has been little research on factors that predict alcohol-impaired driving at the event-level. This pilot/feasibility study was designed to identify within-person, event-level predictors of alcohol-impaired driving intentions in the natural environment using an ecological momentary assessment (EMA) design. METHOD: Thirty-six young adult, moderate drinkers (M age = 22.9 years; 72.2% female; M drinks per occasion = 3.2) were recruited from a university area to complete 2 weeks of EMA. They reported on their subjective levels of intoxication, perceived dangerousness of driving, and driving intentions during real-world drinking episodes. Breath alcohol concentrations were collected with a portable breath alcohol analyzer. RESULTS: Event-level perceived danger and subjective intoxication most strongly predicted intentions to drive after drinking, such that higher perceived danger and intoxication predicted lower willingness to drive, after adjusting for baseline alcohol-impaired driving attitudes (ps < .001). When we accounted for perceived danger during drinking episodes at the event and person level, baseline attitudes were no longer predictive of willingness to drive. Higher event-level breath alcohol concentration also predicted lower willingness to drive (p = .003). CONCLUSIONS: This study is the first to demonstrate that event-level risks of alcohol-impaired driving can be collected during drinking episodes in the natural environment. Findings indicate that subjective perceptions of intoxication and risk more strongly predict alcohol-impaired driving intentions than objective intoxication. Findings also suggest that event-level perceptions of intoxication and driving risk may be fruitful targets for interventions to reduce alcohol-impaired driving.

Invariant item ordering refers to the statement that if one item is harder than another for one person, then it is harder for all people. Whether item ordering holds is a psychological statement because it describes how people may qualitatively vary. Yet, modern item response theory (IRT) makes an a priori commitment to item ordering. The Rasch model, for example, posits that items must order. Conversely, the 2PL model posits that items never order. Needed is an IRT model where item ordering or its violation is a function of the data rather than an a priori commitment. We develop two-parameter shift-scale models for this purpose, and find that the two-parameter uniform offers many advantages. We show how item ordering may be assessed using Bayes factor model comparison, and discuss computational issues with shift-scale IRT models.

Studies of recognition memory often demonstrate a recency effect on behavioral performance, whereby response times (RTs) are faster for stimuli that were previously presented recently as opposed to more remotely in the past. This relationship between performance and presentation lag has been taken to reflect that memories are accessed by serially searching backwards in time, such that RT indicates the self-terminating moment of such a process. Here, we investigated the conditions under which this serial search gives way to more efficient means of retrieving memories. Event-related potentials (ERPs) were recorded during a continuous recognition task in which subjects made binary old/new judgments to stimuli that were each presented up to four times across a range of lags. Stimulus repetition and shorter presentation lag both gave rise to speeded RTs, consistent with previous findings, and we novelly extend these effects to a robust latency measure of the left parietal ERP effect associated with retrieval success. Importantly, the relationship between repetition and recency was further elucidated, such that repetition attenuated lag-related differences that were initially present in both the behavioral and neural latency data. These findings are consistent with the idea that a serial search through recent memory can quickly be abandoned in favor of relying on more efficient ‘time-independent’ cognitive processes or neural signals.

A family of score-based tests has been proposed in the past years for assessing the invariance of model parameters in several models of item response theory. These tests were originally developed in a maximum likelihood framework. This study aims to extend the theoretical framework of these tests to Bayesian maximum-a-posteriori estimates and to multiple group IRT models. We propose two families of statistical tests, which are based on a) an approximation using a pooled variance method, or b) a simulation-based approach based on asymptotic results. The resulting tests were evaluated by a simulation study, which investigated their sensitivity against differential item functioning with respect to a categorical or continuous person covariate in the two- and three-parametric logistic models. Whereas the method based on pooled variance was found to be practically useful with maximum likelihood as well as maximum-a-posteriori estimates, the simulation-based approach was found to require large sample sizes to lead to satisfactory results.

Structural equation models comprise a large class of popular statistical models, including factor analysis models, certain mixed models, and extensions thereof. Model estimation is complicated by the fact that we typically have multiple interdependent response variables and multiple latent variables (which may also be called random effects or hidden variables), often leading to slow and inefficient MCMC samples. In this paper, we describe and illustrate a general, efficient approach to Bayesian SEM estimation in Stan, contrasting it with previous implementations in R package blavaan (Merkle & Rosseel, 2018). After describing the approaches in detail, we conduct a practical comparison under multiple scenarios. The comparisons show that the new approach is clearly better. We also discuss ways that the approach may be extended to other models that are of interest to psychometricians., 21 pages, 5 figures

Cannabis use has been rising despite recognition of the negative consequences associated with heavy use. The severity of these consequences has been shown to differ across racial/ethnic groups, even when controlling for consumption levels. The present study conducted an item response theory (IRT) analysis of the Cannabis Use Disorders Identification Test (CUDIT) to better understand the patterns of problematic cannabis use and their relation with other substance use across ethnic groups in the Healthy Life in an Urban Setting (HELIUS) study. CUDIT responses from 1,960 cannabis-using African Surinamese, South-Asian Surinamese, Dutch, Moroccan, and Turkish ethnic origin participants were used to test for differential item functioning (DIF) within an IRT framework. After restricting the sample to men because of low frequency of use among women, several instances of uniform DIF were identified. Multiple-group IRT analysis yielded a harmonized cannabis use phenotype that was used to estimate ethnic group differences in problematic cannabis use and its relation to alcohol and tobacco co-use. These analyses suggested that cannabis users from certain ethnic minority groups experienced higher rates of problematic use than the majority group despite lower rates of cannabis use. Further, cannabis and tobacco use were positively related across groups, whereas only ethnic minority groups showed a positive relation between cannabis and alcohol use. These results demonstrate the importance of accounting for DIF when examining group differences in problematic cannabis use, and support prior evidence suggesting that certain ethnic minority groups may be more likely to experience problematic cannabis use and alcohol co-use relative to the majority group. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Forecasters are typically evaluated via proper scoring rules such as the Brier score. These scoring rules use only the reported forecasts for assessment, neglecting related variables such as the specific questions that a person chose to forecast. In this paper, we study whether information related to question selection influences our estimates of forecaster ability. In other words, do good and bad forecasters tend to select questions in different ways? If so, can we capitalize on these selections when estimating forecaster ability? We address these questions by extending a recently-developed psychometric model of forecasts to include question selection data. We compare the extended psychometric model to a simpler model, studying its unidimensionality assumption and highlighting the unique information that it can provide. We find that the model can make use of the fact that good forecasters tend to select more questions than bad forecasters, and we conclude that question selection data can be beneficial above and beyond reported forecasts. As a side benefit, the resulting model can potentially provide unique incentives for forecaster participation.

Methods to determine the direction of a regression line, that is, to determine the direction of dependence in reversible linear regression models (e.g., x→y vs. y→x), have experienced rapid development within the last decade. However, previous research largely rested on the assumption that the true predictor is measured without measurement error. The present paper extends the direction dependence principle to measurement error models. First, we discuss asymmetric representations of the reliability coefficient in terms of higher moments of variables and the attenuation of skewness and excess kurtosis due to measurement error. Second, we identify conditions where direction dependence decisions are biased due to measurement error and suggest method of moments (MOM) estimation as a remedy. Third, we address data situations in which the true outcome exhibits both regression and measurement error, and propose a sensitivity analysis approach to determining the robustness of direction dependence decisions against unreliably measured outcomes. Monte Carlo simulations were performed to assess the performance of MOM-based direction dependence measures and their robustness to violated measurement error assumptions (i.e., non-independence and non-normality). An empirical example from subjective well-being research is presented. The plausibility of model assumptions and links to modern causal inference methods for observational data are discussed.

Recently, a dynamic-interactive model of person construal (DI model) has been proposed, whereby the social categories a person represents are determined on the basis of an iterative integration of bottom-up and top-down influences. The current study sought to test this model by leveraging the high temporal resolution of event-related brain potentials (ERPs) as 65 participants viewed male faces that varied by race (White vs Black), fixating either between the eyes or on the forehead. Within face presentations, the effect of fixation, meant to vary bottom-up visual input, initially was large but decreased across early latency neural responses identified by a principal components analysis (PCA). In contrast, the effect of race, reflecting a combination of top-down and bottom-up factors, initially was small but increased across early latency principal components. These patterns support the DI model prediction that bottom-up and top-down processes are iteratively integrated to arrive at a stable construal within 230 ms. Additionally, exploratory multilevel modeling of single trial ERP responses representing a component linked to outgroup categorization (the P2) suggests change in effects of the manipulations over the course of the experiment. Implications of the findings for the DI model are considered.

While likelihood-based derivatives and related facilities are available in R for many types of statistical models, the facilities are notably lacking for models estimated via lme4. This is because the necessary statistical output, including the Hessian, Fisher information and casewise contributions to the model gradient, is not immediately available from lme4 and is not trivial to obtain. In this article, we describe merDeriv, an R package which supplies new functions to obtain analytic output from Gaussian mixed models. We discuss the theoretical results implemented in the code, focusing on calculation of robust standard errors via package sandwich. We also use the sleepstudy data to illustrate the package and to compare it to a benchmark from package lavaan.

In this article, we apply Vuong's (1989) likelihood ratio tests of nonnested models to the comparison of nonnested structural equation models (SEMs). Similar tests have been previously applied in SEM contexts (especially to mixture models), though the nonstandard output required to conduct the tests has limited their use and study. We review the theory underlying the tests and show how they can be used to construct interval estimates for differences in nonnested information criteria. Through both simulation and application, we then study the tests' performance in nonmixture SEMs and describe their general implementation via free R packages. The tests offer researchers a useful tool for nonnested SEM comparison, with barriers to test implementation now removed. (PsycINFO Database Record

In a variety of real-world forecasting contexts, researchers have demonstrated that the unweighted average forecast is reasonably accurate and difficult to improve upon with more complex, model-based aggregation methods. We investigate this phenomenon by systematically examining the relationship between individual forecaster characteristics (e.g., bias, consistency) and aspects of the criterion being forecast (e.g., ``signal strength''). To this end, we develop a model inspired by Cultural Consensus Theory (Batchelder & Romney, 1988) that (i) allows us to jointly estimate both forecaster characteristics and environmental characteristics and (ii) contains the unweighted average as a special case. This allows us to use the model as a regularization method for forecast aggregation, where restrictions on forecaster parameters make the model similar to use of an unweighted average. Relatedly, the model allows us to apply existing results on optimal forecaster weighting to real data. We show how the model provides guidance for identifying prediction environments where the average forecast can potentially be beaten. We also conduct two simulation studies and illustrate the model's practical application using forecasts of Australian Football League point spreads.

Typical Bayesian methods for models with latent variables (or random effects) involve directly sampling the latent variables along with the model parameters. In high-level software code for model definitions (using, e.g., BUGS, JAGS, Stan), the likelihood is therefore specified as conditional on the latent variables. This can lead researchers to perform model comparisons via conditional likelihoods, where the latent variables are considered model parameters. In other settings, however, typical model comparisons involve marginal likelihoods where the latent variables are integrated out. This distinction is often overlooked despite the fact that it can have a large impact on the comparisons of interest. In this paper, we clarify and illustrate these issues, focusing on the comparison of conditional and marginal Deviance Information Criteria (DICs) and Watanabe-Akaike Information Criteria (WAICs) in psychometric modeling. The conditional/marginal distinction corresponds to whether the model should be predictive for the clusters that are in the data or for new clusters (where "clusters" typically correspond to higher-level units like people or schools). Correspondingly, we show that marginal WAIC corresponds to leave-one-cluster out (LOcO) cross-validation, whereas conditional WAIC corresponds to leave-one-unit out (LOuO). These results lead to recommendations on the general application of the criteria to models with latent variables., Comment: Manuscript in press at Psychometrika; 31 pages, 8 figures

EEG data, and specifically the event-related potential (ERP), provide psychologists with the power to examine quickly occurring cognitive processes at the native temporal resolution at which they occur. Despite the advantages conferred by ERPs to examine processes at different points in time, ERP researchers commonly ignore the trial-to-trial temporal dimension by collapsing across trials of similar types (i.e., the signal averaging approach) because of constraints imposed by repeated-measured ANOVA. Here, we present the advantages of using multilevel modeling (MLM) to examine trial-level data, to investigate change in neurocognitive processes across the course of an experiment. Two examples are presented to illustrate the usefulness of this technique. The first demonstrates decreasing differentiation in N170 amplitude to faces of different races across the course of a race categorization task. The second demonstrates attenuation of the ERN as participants commit more errors within a task designed to measure implicit racial bias. Although the examples presented here are within the realm of social psychology, the use of MLM to analyze trial-level EEG data has the potential to contribute to a number of different theoretical domains within psychology.

Introduction. Little is known about how physicians present diagnosis and treatment planning in routine practice in preference-sensitive treatment decisions. We evaluated completeness and quality of informed decision making in localized prostate cancer post biopsy encounters. Methods. We analyzed audio-recorded office visits of 252 men with presumed localized prostate cancer (Gleason 6 and Gleason 7 scores) who were seeing 45 physicians at 4 Veterans Affairs Medical Centers. Data were collected between September 2008 and May 2012 in a trial of 2 decision aids (DAs). Braddock’s previously validated Informed Decision Making (IDM) system was used to measure quality. Latent variable models for ordinal data examined the relationship of IDM score to treatment received. Results. Mean IDM score showed modest quality (7.61±2.45 out of 18) and high variability. Treatment choice and risks and benefits were discussed in approximately 95% of encounters. However, in more than one-third of encounters, physicians provided a partial set of treatment options and omitted surveillance as a choice. Informing quality was greater in patients treated with surveillance (β = 1.1, p = .04). Gleason score (7 vs 6) and lower age were often cited as reasons to exclude surveillance. Patient preferences were elicited in the majority of cases, but not used to guide treatment planning. Encounter time was modestly correlated with IDM score ( r = 0.237, p = .01). DA type was not associated with IDM score. Discussion. Physicians informed patients of options and risks and benefits, but infrequently engaged patients in core shared decision-making processes. Despite patients having received DAs, physicians rarely provided an opportunity for preference-driven decision making. More attention to the underused patient decision-making and engagement elements could result in improved shared decision making.

Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against "monotonic" violations of measurement invariance and lower power against "non-monotonic" ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.

We propose the use of signal detection theory (SDT) to evaluate the performance of both probabilistic forecasting systems and individual forecasters. The main advantage of SDT is that it provides a principled way to distinguish the response from system diagnosticity, which is defined as the ability to distinguish events that occur from those that do not. There are two challenges in applying SDT to probabilistic forecasts. First, the SDT model must handle judged probabilities rather than the conventional binary decisions. Second, the model must be able to operate in the presence of sparse data generated within the context of human forecasting systems. Our approach is to specify a model of how individual forecasts are generated from underlying representations and use Bayesian inference to estimate the underlying latent parameters. Given our estimate of the underlying representations, features of the classic SDT model, such as the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC), follow immediately. We show how our approach allows ROC curves and AUCs to be applied to individuals within a group of forecasters, estimated as a function of time, and extended to measure differences in forecastability across different domains. Among the advantages of this method is that it depends only on the ordinal properties of the probabilistic forecasts. We conclude with a brief discussion of how this approach might facilitate decision making.

It is known that the average of many forecasts about a future event tends to outperform the individual assessments. With the goal of further improving forecast performance, this paper develops and compares a number of models for calibrating and aggregating forecasts that exploit the well-known fact that individuals exhibit systematic biases during judgment and elicitation. All of the models recalibrate judgments or mean judgments via a two-parameter calibration function, and differ in terms of whether (1) the calibration function is applied before or after the averaging, (2) averaging is done in probability or log-odds space, and (3) individual differences are captured via hierarchical modeling. Of the non-hierarchical models, the one that first recalibrates the individual judgments and then averages them in log-odds is the best relative to simple averaging, with 26.7 % improvement in Brier score and better performance on 86 % of the individual problems. The hierarchical version of this model does slightly better in terms of mean Brier score (28.2 %) and slightly worse in terms of individual problems (85 %).

The issue of measurement invariance commonly arises in factor-analytic contexts, with methods for assessment including likelihood ratio tests, Lagrange multiplier tests, and Wald tests. These tests all require advance definition of the number of groups, group membership, and offending model parameters. In this paper, we study tests of measurement invariance based on stochastic processes of casewise derivatives of the likelihood function. These tests can be viewed as generalizations of the Lagrange multiplier test, and they are especially useful for: (i) identifying subgroups of individuals that violate measurement invariance along a continuous auxiliary variable without prespecified thresholds, and (ii) identifying specific parameters impacted by measurement invariance violations. The tests are presented and illustrated in detail, including an application to a study of stereotype threat and simulations examining the tests' abilities in controlled conditions.

A growing number of researchers have advocated for the advancement of cognitive neuroscience by blending cognitive models with neurophysiology. The recently proposed joint modeling framework is one way to bridge the gap between the abstractions assumed by cognitive models and the neurophysiology obtained by modern methods in neuroscience. Despite this advancement, the current method for linking the two domains is hindered by the dimensionality of the neural data. In this article, we present a new linking function based on factor analysis that allows joint models to grow linearly in complexity with increases in the number of neural features. The new linking function is then evaluated in two simulation studies. The first simulation study shows how the model parameters can be accurately recovered when there are many neural features, that mimics real-world applications. The second simulation shows how the new linking function can (1) properly recover a representation of the data generating model, even in the case of model misspecification, and (2) outperform the previous linking function in a cross-validation test. We close by applying a model equipped with the new linking function to real-world data from a perceptual decision making task. The model allows us to understand how differences in the model parameters emerge as a function of differences in brain function across speed and accuracy instruction.

Objectives The link between social capital and self-reported health has been widely explored. On the other hand, we know less about the relationship between social capital, community socioeconomic characteristics, and non-social capital–related individual differences, and about their impact on self-reported health in community settings. Methods Cross-sectional study design with a proportional sample of 7965 individuals from 20 US communities were analyzed using multilevel linear regression models, where individuals were nested within communities. The response rates ranged from 13.5% to 25.4%. Results Findings suggest that perceptions of the community and individual level socioeconomic characteristics were stronger predictors of self-reported health than were social capital or community socioeconomic characteristics. Conclusions Policy initiatives aimed at increasing social capital should first assess community member's perceptions of their communities to uncover potential assets to help increase social capital.

In this paper, we address the use of Bayesian factor analysis and structural equation models to draw inferences from experimental psychology data. While such application is non-standard, the models are generally useful for the unified analysis of multivariate data that stem from, e.g., subjects' responses to multiple experimental stimuli. We first review the models and the parameter identification issues inherent in the models. We then provide details on model estimation via JAGS and on Bayes factor estimation. Finally, we use the models to re-analyze experimental data on risky choice, comparing the approach to simpler, alternative methods.

Measurement invariance is a fundamental assumption in item response theory models, where the relationship between a latent construct (ability) and observed item responses is of interest. Violation of this assumption would render the scale misinterpreted or cause systematic bias against certain groups of persons. While a number of methods have been proposed to detect measurement invariance violations, they typically require advance definition of problematic item parameters and respondent grouping information. However, these pieces of information are typically unknown in practice. As an alternative, this paper focuses on a family of recently proposed tests based on stochastic processes of casewise derivatives of the likelihood function (i.e., scores). These score-based tests only require estimation of the null model (when measurement invariance is assumed to hold), and they have been previously applied in factor-analytic, continuous data contexts as well as in models of the Rasch family. In this paper, we aim to extend these tests to two-parameter item response models, with strong emphasis on pairwise maximum likelihood. The tests' theoretical background and implementation are detailed, and the tests' abilities to identify problematic item parameters are studied via simulation. An empirical example illustrating the tests' use in practice is also provided.

This article describes blavaan, an R package for estimating Bayesian structural equation models (SEMs) via JAGS and for summarizing the results. It also describes a novel parameter expansion approach for estimating specific types of models with residual covariances, which facilitates estimation of these models in JAGS. The methodology and software are intended to provide users with a general means of estimating Bayesian SEMs, both classical and novel, in a straightforward fashion. Users can estimate Bayesian versions of classical SEMs with lavaan syntax, they can obtain state-of-the-art Bayesian fit measures associated with the models, and they can export JAGS code to modify the SEMs as desired. These features and more are illustrated by example, and the parameter expansion approach is explained in detail.

This paper describes the application of finite-mixture general linear models based on the beta distribution to modeling response styles, polarization, anchoring, and priming effects in probability judgments. These models, in turn, enhance our capacity for explicitly testing models and theories regarding the aforementioned phenomena. The mixture model approach is superior in this regard to popular methods such as extremity scores, due to its incorporation of three submodels (location, dispersion, and relative composition), each of which can diagnose specific kinds of polarization and related effects. Three examples are elucidated using real data sets.