Search

This study evaluated measurement invariance across males and females on the Neurobehavioral Symptom Inventory (NSI) in U.S. military veterans enrolled in the VA Million Veteran Program. Participants (N = 17,059; males: n = 15,450; females: n = 1,609) included Veterans who took part in the VA Traumatic Brain Injury (TBI) Screening and Evaluation Program and completed the NSI. Multiple-group confirmatory factor analyses investigated measurement invariance of the NSI 4-factor model. The configural (comparative fit index [CFI] = 0.948, root mean square error of approximation [RMSEA] = 0.060) and metric (CFI = 0.948, RMSEA = 0.058) invariance models showed acceptable fit. There was a minor violation of scalar invariance (Δχ2 = 232.50, p <.001); however, the degree of noninvariance was mild (ΔCFI = −0.002, Δ RMSEA = 0. 000). Our results demonstrate measurement invariance across sex, suggesting that the NSI 4-factor model can be used to accurately assess symptoms in males and females following TBI. Findings highlight the importance of considering validity of measurement across study groups to increase confidence that a measure is interpreted similarly by respondents from different subgroups. [ABSTRACT FROM AUTHOR]

Measurement invariance holds when a latent construct is measured in the same way across different levels of background variables (continuous or categorical) while controlling for the true value of that construct. Using Monte Carlo simulation, this paper compares the multiple indicators, multiple causes (MIMIC) model and MIMIC-interaction to a novel use of alignment optimization (AO) for detecting measurement noninvariance when the violator is a continuous variable. Results showed that MIMIC and MIMIC-interaction in sequential likelihood ratio tests and Wald tests with a Bonferroni correction provided a good balance between identifying invariant and noninvariant (linear violations) items when n ≥ 500 in terms of classification accuracy (CA). AO (CA ≥ .86) was as competitive as MIMIC and MIMIC-interaction to linear invariance violations but was far better under nonlinear quadratic violations when n ≥ 1,000 (i.e., 100 per group for 10 groups). [ABSTRACT FROM AUTHOR]

Measurement invariance research has focused on identifying biases in test indicators measuring a latent trait across two or more groups. However, relatively little attention has been devoted to the practical implications of noninvariance. An important question is whether noninvariance in indicators or items results in differences in observed composite scores across groups. The current study introduces the Bayesian region of measurement equivalence (ROME) as a framework for visualizing and testing the combined impact of partial invariance on the group difference in observed scores. Under the proposed framework, researchers first compute the highest posterior density intervals (HPDIs)-which contain the most plausible values-for the expected group difference in observed test scores over a range of latent trait levels. By comparing the HPDIs with a predetermined range of values that is practically equivalent to zero (i.e., region of measurement equivalence), researchers can determine whether a test instrument is practically invariant. The proposed ROME method can be used for both continuous indicators and ordinal items. We illustrated ROME using five items measuring mathematics-specific self-efficacy from a nationally representative sample of 10th graders. Whereas conventional invariance testing identifies a partial strict invariance model across gender, the statistically significant noninvariant items were found to have a negligible impact on the comparison of the observed scores. This empirical example demonstrates the utility of the ROME method for assessing practical significance when statistically significant item noninvariance is found. [ABSTRACT FROM AUTHOR]

Longitudinal measurement invariance—the consistency of measurement in data collected over time—is a prerequisite for any meaningful inferences of growth patterns. When one or more items measuring the construct of interest show noninvariant measurement properties over time, it leads to biased parameter estimates and inferences on the growth parameters. In this paper, I extend the recently developed alignment-within-confirmatory factor analysis (AwC) technique to adjust for measurement biases for growth models. The proposed AwC method does not require a priori knowledge of noninvariant items and the iterative searching of noninvariant items in typical longitudinal measurement invariance research. Results of a Monte Carlo simulation study comparing AwC with the partial invariance modeling method show that AwC largely reduces biases in growth parameter estimates and gives good control of Type I error rates, especially when the sample size is at least 1,000. It also outperforms the partial invariance method in conditions when all items are noninvariant. However, all methods give biased growth parameter estimates when the proportion of noninvariant parameters is over 25%. Based on the simulation results, I conclude that AO is a viable alternative to the partial invariance method in growth modeling when it is not clear whether longitudinal measurement invariance holds. The current paper also demonstrates AwC in an example modeling neuroticism over three time points using a public data set, which shows how researchers can compute effect size indices for noninvariance in AwC to assess to what degree invariance holds and whether AwC results are trustworthy. [ABSTRACT FROM AUTHOR]

There has been tremendous growth in research on measurement invariance over the past two decades. However, given that psychological tests are commonly used for making classification decisions such as personnel selections or diagnoses, surprisingly, there has been little research on how noninvariance impacts classification accuracy. Millsap and Kwok previously proposed a selection accuracy framework for that purpose, which has been recently extended to categorical data. Their framework, however, only deals with classification using a unidimensional test. In contrast, classification in practice usually involves multidimensional tests (e.g., personality) or multiple tests, with different weights assigned to each dimension. In the current paper, we extend Millsap and Kwok's framework for examining the impact of noninvariance to a multidimensional test on classification. We also provide an R script for the proposed method and illustrate it with a personnel selection example using data from a published report featuring a five-factor personality inventory. [ABSTRACT FROM AUTHOR]

Establishing measurement invariance, or that an instrument measures the same construct(s) in the same way across subgroups of respondents, is crucial in efforts to validate social and behavioral instruments. Although substantial previous research has focused on detecting the presence of noninvariance, less attention has been devoted to its practical significance and even less has been paid to its possible impact on diagnostic accuracy. In this article, we draw additional attention to the importance of measurement invariance and advance diagnostic research by introducing a novel approach for quantifying the impact of noninvariance with binary items (e.g., the presence or absence of symptoms). We illustrate this approach by testing measurement invariance and evaluating diagnostic accuracy across age groups using DSM alcohol use disorder items from a public national data set. By providing researchers with an easy-to-implement R program for examining diagnostic accuracy with binary items, this article sets the stage for future evaluations of the practical significance of partial invariance. Future work can extend our framework to include ordinal and categorical indicators, other measurement models in item response theory, settings with three or more groups, and via comparison to an external, "gold-standard" validator. [ABSTRACT FROM AUTHOR]

Much of the previous literature on partial measurement invariance has focused on (a) statistically detecting noninvariance, and (b) modeling partial invariance to obtain correct inferences for latent mean comparisons across groups in a single research study. However, very little guidance is provided on the practical implications of partial invariance on the instrument itself in the context of selection. In a frequently cited paper, Millsap and Kwok (2004) provided a framework for evaluating the impact of partial invariance by quantifying the magnitude of noninvariance on the efficacy of the test for selection purposes, yet our literature review found that only a few of the citations have fully captured the essence of Millsap and Kwok's method. In this article, we briefly review the selection accuracy analysis for partial invariance and provide a user-friendly R script (also available as a Web application) that takes parameter estimates as input, automatically produces summary statistics for evaluating selection accuracy, and generates a graph for visualizing the results. Hypothetical and real data examples are provided to illustrate the use of the R script. The goal of this article is to help readers understand Millsap and Kwok's framework of evaluating the impact of partial invariance through an accessible computer program and step-by-step demonstrations of the selection accuracy analysis. [ABSTRACT FROM AUTHOR]