Search

Latent variable models (LVMs) are incredibly flexible tools that allow users to address research questions they might otherwise never be able to answer (McDonald, 2013). However, one major limitation of LVMs is evaluating model fit. There is no universal consensus about how to evaluate model fit, either globally or locally. Part of the reason evaluating these models is difficult is because fit is typically reduced to a handful of statistics that may or may not reflect the model's adequacy and/or assumptions. In this article we argue that proper evaluation of model fit must include visualizing both the raw data and the model-implied fit. Visuals reveal, at a glance, the fit of the model and whether the model's assumptions have been met. Unfortunately, tools for visualizing LVMs have historically been limited. In this article, we introduce new plots and reframe existing plots that provide necessary resources for evaluating LVMs. These plots are available in a new open-source R package called flexplavaan, which combines the model plotting capabilities of flexplot with the latent variable modeling capabilities of lavaan. There is a class of models frequently used by researchers called latent variable models, or LVMs. These models are extremely flexible and useful, especially when one is trying to assess a characteristic that is "unobserved," like intelligence or depression. However, historically, there have not been intuitive methods for evaluating how well these models fit the data. This article introduces a software application called "flexplavaan," which makes it easy to visualize the fits of these models. flexplavaan makes it easy to identify sources of misfit and to evaluate the model's assumptions. [ABSTRACT FROM AUTHOR]

Recent reform efforts have pushed toward a better understanding of the distinction between exploratory and confirmatory research, and appropriate use of each. As some utilize more exploratory tools, it may be tempting to employ multiple linear regression models. In this paper, we advocate for the use of random forest (RF) models. RF is able to obtain better predictive performance than traditional regression, while also inherently protecting against overfitting as well as detecting nonlinear effects and interactions among predictors. Given the advantages of RF compared to other statistical procedures, it is a tool commonly used within a plethora of industries, including stock trading, banking, pharmaceuticals, and patient healthcare planning. However, we find RF is used within the field of psychology comparatively less frequently. In the current paper, we advocate for RF as an important statistical tool within the context of behavioral and psychological research. In hopes of increasing the use of RF in the field of psychology, we provide information pertaining to the limitations one might confront in using RF and how to overcome such limitations. Moreover, we discuss various methods for how to optimally utilize RF with psychological data, such as nonparametric modeling, interaction and nonlinearity detection, variable selection, prediction and classification modeling, and assessing parameters of Monte Carlo simulations. Throughout, we illustrate the use of RF with visualization strategies, aimed to make RF models more comprehensible and intuitive. [ABSTRACT FROM AUTHOR]

Though much research and attention has been directed at assessing the correlation coefficient under range restriction, the assessment of reliability under range restriction has been largely ignored. This article uses item response theory to simulate dichotomous item-level data to assess the robustness of KR-20 ([alpha]), [omega], and test-retest under varying selection ratios. These estimators, both corrected and uncorrected for range restriction, were compared in terms of both bias and precision. Test-retest reliability was usually the best estimator of reliability across a variety of conditions. Only under indirect range restriction did KR-20 and [omega] performed well. All estimators suffered imprecision as a function of range restriction, above and beyond the reduction in sample size. Based on the results, a set of recommendations are proposed. (Contains 7 tables and 6 figures.)

Estimates of subgroup differences are routinely used as part of a comprehensive validation system, and these estimates serve a critical role, including evaluating adverse impact. Unfortunately, under direct range restriction, a selected mean (μ̂t′) is a biased estimator of the population mean μx as well as the selected true score mean μt′. This is due partly to measurement bias. This bias, as we show, is a factor of the selection ratio, the reliability of the measure, and the variance of the distribution. This measurement bias renders a subgroup comparison questionable when the subgroups have different selection ratios. The selected subgroup comparison is further complicated by the fact that the subgroup variances will be unequal in most situations where the selection ratios are not equal. We address these problems and present a corrected estimate of the mean difference, as well as an estimate of Cohen's d* that estimates the true score difference between two selected populations, (μtA′-μtB′)/σt. In addition, we show that the measurement bias is not present under indirect range restriction. Thus, the observed selected mean μ̂t′ is an unbiased estimator of selected true score mean μty′. However, it is not an unbiased estimator of the population mean μy. These results have important implications for selection research, particularly when validating instruments. [ABSTRACT FROM AUTHOR]

When estimating subgroup differences on incumbents, range restriction may bias estimates. Bobko, Roth, and Bobko recognized this problem and developed a Case II and Case III correction for Cohen's d. Subsequently, Li developed a Case IV correction, which seeks to estimate group differences on a predictor using only incumbent data but must assume that group membership (e.g., ethnicity) plays no role in selection decisions. In this paper, we extend Li's correction and relax this assumption. In addition, this new correction allows for the estimation of subgroup differences on both the criterion and predictor. Using Monte Carlo simulation, we study the performance of both estimators under situations where Li's assumptions are violated and demonstrate that this new procedure almost always outperforms Li's Case IV correction and does so with greater precision. We also provide R code to assist applied researchers in using these corrections. [ABSTRACT FROM AUTHOR]