Search

The new methodological and technological developments of the last decade make it possible to resolve or, at least, attenuate the psychometric problems of forced-choice (FC) tests for the measurement of personality. In these tests, the person being tested is shown blocks of two or more sentences of similar social desirability, from which he or she must choose which one best represents him or her. Thus, FC tests aim to reduce response bias in self-report questionnaires. However, their use is not without risks and complications if they are not created properly. Fortunately, new psychometric models make it possible to model responses in this type of test and to optimize their construction. Moreover, they allow the construction of "on the fly" computerized adaptive FC tests (CAT-FC), in which each item is constructed on the spot, optimally matching sentences from a previously calibrated bank. [ABSTRACT FROM AUTHOR]

As general factor modeling continues to grow in popularity, researchers have become interested in assessing how reliable general factor scores are. Even though omega hierarchical estimation has been suggested as a useful tool in this context, little is known about how to approximate it using modern bi-factor exploratory factor analysis methods. This study is the first to compare how omega hierarchical estimates were recovered by six alternative algorithms: Bi-quartimin, bi-geomin, Schmid-Leiman (SL), empirical iterative empirical target rotation based on an initial SL solution (SLiD), direct SL (DSL), and direct bi-factor (DBF). The algorithms were tested in three Monte-Carlo simulations including bi-factor and second-order structures and presenting complexities such as cross-loadings or pure indicators of the general factor and structures without a general factor. Results showed that SLiD provided the best approximation to omega hierarchical under most conditions. Overall, neither SL, bi-quartimin, nor bi-geomin produced an overall satisfactory recovery of omega hierarchical. Lastly, the performance of DSL and DBF depended upon the average discrepancy between the loadings of the general and the group factors. The re-analysis of eight classical datasets further illustrated how algorithm selection could influence judgments regarding omega hierarchical. [ABSTRACT FROM AUTHOR]

Contemporary models of personality assume a hierarchical structure in which broader traits contain narrower traits. Individual differences in response styles also constitute a source of score variance. In this study, the bifactor model is applied to separate these sources of variance for personality subscores. The procedure is illustrated using data for two personality inventories—NEO Personality Inventory–Revised and Zuckerman–Kuhlman–Aluja Personality Questionnaire. The inclusion of the acquiescence method factor generally improved the fit to acceptable levels for the Zuckerman–Kuhlman–Aluja Personality Questionnaire, but not for the NEO Personality Inventory–Revised. This effect was higher in subscales where the number of direct and reverse items is not balanced. Loadings on the specific factors were usually smaller than the loadings on the general factor. In some cases, part of the variance was due to domains being different from the main one. This information is of particular interest to researchers as they can identify which subscale scores have more potential to increase predictive validity. [ABSTRACT FROM AUTHOR]

The accuracy of factor retention methods for structures with one or more general factors, like the ones typically encountered in fields like intelligence, personality, and psychopathology, has often been overlooked in dimensionality research. To address this issue, we compared the performance of several factor retention methods in this context, including a network psychometrics approach developed in this study. For estimating the number of group factors, these methods were the Kaiser criterion, empirical Kaiser criterion, parallel analysis with principal components (PA PCA ) or principal axis, and exploratory graph analysis with Louvain clustering (EGA LV ). We then estimated the number of general factors using the factor scores of the first-order solution suggested by the best two methods, yielding a "second-order" version of PA PCA (PAP CA-FS ) and EGA LV (EGA LV-FS ). Additionally, we examined the direct multilevel solution provided by EGA LV . All the methods were evaluated in an extensive simulation manipulating nine variables of interest, including population error. The results indicated that EGA LV and PA PCA displayed the best overall performance in retrieving the true number of group factors, the former being more sensitive to high cross-loadings, and the latter to weak group factors and small samples. Regarding the estimation of the number of general factors, both PAP CA-FS and EGA LV-FS showed a close to perfect accuracy across all the conditions, while EGA LV was inaccurate. The methods based on EGA were robust to the conditions most likely to be encountered in practice. Therefore, we highlight the particular usefulness of EGA LV (group factors) and EGA LV-FS (general factors) for assessing bifactor structures with multiple general factors. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Background: The emergence of digital technology in the field of psychological and educational measurement and assessment broadens the traditional concept of pencil and paper tests. New assessment models built on the proliferation of smartphones, social networks and software developments are opening up new horizons in the field., Method: This study is divided into four sections, each discussing the benefits and limitations of a specific type of technology-based assessment: ambulatory assessment, social networks, gamification and forced-choice testing., Results: The latest developments are clearly relevant in the field of psychological and educational measurement and assessment. Among other benefits, they bring greater ecological validity to the assessment process and eliminate the bias associated with retrospective assessment., Conclusions: Some of these new approaches point to a multidisciplinary scenario with a tradition which has yet to be created. Psychometrics must secure a place in this new world by contributing sound expertise in the measurement of psychological variables. The challenges and debates facing the field of psychology as it incorporates these new approaches are also discussed.

Background: The inclusion of direct and reversed items in scales is a commonly-used strategy to control acquiescence bias. However, this is not enough to avoid the distortions produced by this response style in the structure of covariances and means of the scale in question. This simulation study provides evidence on the performance of two different procedures for modelling the influence of acquiescence bias on partially balanced multidimensional scales: a method based on exploratory factor analysis (EFA) with target rotation, and a method based on random intercept factor analysis (RIFA)., Method: The independent variables analyzed in a simulation study were sample size, number of items per factor, balance of substantive loadings of direct and reversed items, size and heterogeneity of acquiescence loadings, and inter-factor correlation., Results: The RIFA method had better performance over most of the conditions, especially for the balanced conditions, although the variance of acquiescence factor loadings had a certain impact. In relation to the EFA method, it was severely affected by a low degree of balance., Conclusions: RIFA seems the most robust approach, but EFA also remains a good alternative for medium and fully balanced scales.

Background: Unproctored Internet Tests (UIT) are vulnerable to cheating attempts by candidates to obtain higher scores. To prevent this, subsequent procedures such as a verification test (VT) is carried out. This study compares five statistics used to detect cheating in Computerized Adaptive Tests (CATs): Guo and Drasgow's Z-test, the Adaptive Measure of Change (AMC), Likelihood Ratio Test (LRT), Score Test, and Modified Signed Likelihood Ratio Test (MSLRT)., Method: We simulated data from honest and cheating candidates to the UIT and the VT. Honest candidates responded to the UIT and the VT with their real ability level, while cheating candidates responded only to the VT, and different levels of cheating were simulated. We applied hypothesis tests, and obtained type I error and power rates., Results: Although we found differences in type I error rates between some of the procedures, all procedures reported quite accurate results with the exception of the Score Test. The power rates obtained point to MSLRT's superiority in detecting cheating., Conclusions: We consider the MSLRT to be the best test, as it has the highest power rate and a suitable type I error rate.

Background: Due to its flexibility and statistical properties, bi-factor Exploratory Structural Equation Modeling (bi-factor ESEM) has become an often-recommended tool in psychometrics. Unfortunately, most recent methods for approximating these structures, such as the SLiD algorithm, are not available in the leading software for performing ESEM (i.e., Mplus). To resolve this issue, we present a novel, user-friendly Shiny application for integrating the SLiD algorithm in bi-factor ESEM estimation in Mplus. Thus, a two-stage framework for conducting SLiD-based bi-factor ESEM in Mplus was developed., Method: This approach was presented in a step-by-step guide for applied researchers, showing the utility of the developed SLiDApp application. Using data from the Open-Source Psychometrics Project (N = 2495), we conducted a bi-factor ESEM exploration of the Generic Conspiracist Beliefs Scale. We studied whether bi-factor modelling was appropriate and if both general and group factors were related to each personality trait., Results: The application of the SLiD algorithm provided unique information regarding this factor structure and its ESEM structural parameters., Conclusions: The results illustrated the usefulness and validity of SLiD-based bi-factor ESEM, and how the proposed Shiny app could make it eaiser for applied researchers to use these methods.

This study describes the parameter drift analysis conducted on eCAT (a Computerized Adaptive Test to assess the written English level of Spanish speakers). The original calibration of the item bank (N = 3224) was compared to a new calibration obtained from the data provided by most eCAT operative administrations (N = 7254). A Differential Item Functioning (DIF) study was conducted between the original and the new calibrations. The impact that the new parameters have on the trait level estimates was obtained by simulation. Results show that parameter drift is found especially for a and c parameters, an important number of bank items show DIF, and the parameter change has a moderate impact on high-level-English ? estimates. It is then recommended to replace the original estimates by the new set.

Although gains in generational intelligence test scores have been widely demonstrated around the world, researchers still do not know what has caused them. The cognitive stimulation and nutritional hypotheses summarize the several diverse potential causes that have been considered. This article analyses data for a sample of 499 children tested in 1930 and one equivalent sample of 710 children tested 72 years later, the largest gap ever considered. Both samples comprised children aged between 7 and 11 who were assessed by the Draw-a-Man test in the city of Belo Horizonte, Brazil. Further, one additional sample of 132 children was assessed in 2004 in a rural area very similar in several diverse factors to the 1930 urban sample. The results are consistent with both the cognitive stimulation and the nutritional hypotheses.

Item selection rules in a Computerized Adaptive Test for the assessment of written English. e-CAT is a Computerized Adaptive Test for the evaluation of written English knowledge, using the item selection rule most commonly employed: the maximum Fisher information criterion. Some of the problems of this criterion have a negative impact in the estimation accuracy and in the item bank security. In this study, the performance of this item selection rule is compared, by means of simulation, with two other rules: selecting the item with maximum Fisher information in an interval (Veerkamp y Berger, 1997) and a new criterion, called "maximum Fisher information in an interval with geometric mean". In general, this new rule shows smaller measurement error and smaller item overlap rates. It seems, thus, recommendable, as it allows the simultaneous improvement of estimation accuracy and the maintenance of the item bank security of e-CAT.