Search

The new methodological and technological developments of the last decade make it possible to resolve or, at least, attenuate the psychometric problems of forced-choice (FC) tests for the measurement of personality. In these tests, the person being tested is shown blocks of two or more sentences of similar social desirability, from which he or she must choose which one best represents him or her. Thus, FC tests aim to reduce response bias in self-report questionnaires. However, their use is not without risks and complications if they are not created properly. Fortunately, new psychometric models make it possible to model responses in this type of test and to optimize their construction. Moreover, they allow the construction of “on the fly” computerized adaptive FC tests (CAT-FC), in which each item is constructed on the spot, optimally matching sentences from a previously calibrated bank.

Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theory (IRT) models enable the estimation of nonipsative scores from FC responses. However, while some authors indicate that blocks composed of opposite-keyed items are necessary to retrieve normative scores, others suggest that these blocks may be less robust to faking, thus impairing the assessment validity. Accordingly, this article presents a simulation study to investigate whether it is possible to retrieve normative scores using only positively keyed items in pairwise FC computerized adaptive testing (CAT). Specifically, a simulation study addressed the effect of (a) different bank assembly (with a randomly assembled bank, an optimally assembled bank, and blocks assembled "on-the-fly" considering every possible pair of items), and (b) block selection rules (i.e., T, and Bayesian D and A-rules) over the estimate accuracy and ipsativity and overlap rates. Moreover, different questionnaire lengths (30 and 60) and trait structures (independent or positively correlated) were studied, and a nonadaptive questionnaire was included as baseline in each condition. In general, very good trait estimates were retrieved, despite using only positively keyed items. Although the best trait accuracy and lowest ipsativity were found using the Bayesian A-rule with questionnaires assembled "on-the-fly," the T-rule under this method led to the worst results. This points out to the importance of considering both aspects when designing FC CAT.

The nonparametric classification (NPC) method has been proven to be a suitable procedure for cognitive diagnostic assessments at a classroom level. However, its nonparametric nature impedes the obtention of a model likelihood, hindering the exploration of crucial psychometric aspects, such as model fit or reliability. Reporting the reliability and validity of scores is imperative in any applied context. The present study proposes the restricted deterministic input, noisy "and" gate (R-DINA) model, a parametric cognitive diagnosis model based on the NPC method that provides the same attribute profile classifications as the nonparametric method while allowing to derive a model likelihood and, subsequently, to compute fit and reliability indices. The suitability of the new proposal is examined by means of an exhaustive simulation study and a real data illustration. The results show that the R-DINA model properly recovers the posterior probabilities of attribute mastery, thus becoming a suitable alternative for comprehensive small-scale diagnostic assessments.

There has been increased interest in assessing the quality and usefulness of short versions of the Raven's Progressive Matrices. A recent proposal, composed of the last twelve matrices of the Standard Progressive Matrices (SPM-LS), has been depicted as a valid measure of "g." Nonetheless, the results provided in the initial validation questioned the assumption of essential unidimensionality for SPM-LS scores. We tested this hypothesis through two different statistical techniques. Firstly, we applied exploratory graph analysis to assess SPM-LS dimensionality. Secondly, exploratory bi-factor modelling was employed to understand the extent that potential specific factors represent significant sources of variance after a general factor has been considered. Results evidenced that if modelled appropriately, SPM-LS scores are essentially unidimensional, and that constitute a reliable measure of "g." However, an additional specific factor was systematically identified for the last six items of the test. The implications of such findings for future work on the SPM-LS are discussed.

Intelligence is composed of a set of cognitive abilities hierarchically organized. General and specific abilities capture distinguishable, but related, facets of the intelligence construct. Here, we analyze gray matter with three morphometric indices (volume, cortical surface area, and cortical thickness) at three levels of the intelligence hierarchy (tests, first-order factors, and a higher-order general factor, g). A group of one hundred and four healthy young adults completed a cognitive battery and underwent high-resolution structural MRI. Latent scores were computed for the intelligence factors and tests were also analyzed. The key finding reveals substantial variability in gray matter correlates at the test level, which is substantially reduced for the first-order and the higher-order factors. This supports a reversed hierarchy in the brain with respect to cognitive abilities at different psychometric levels: the greater the generality, the smaller the number of relevant gray matter clusters accounting for individual differences in intelligent performance.

The nonparametric classification (NPC) method has been proven to be a suitable procedure for cognitive diagnostic assessments at a classroom level. However, its nonparametric nature impedes the obtention of a model likelihood, hindering the exploration of crucial psychometric aspects, such as model fit or reliability. Reporting the reliability and validity of scores is imperative in any applied context. The present study proposes the restricted deterministic input, noisy "and" gate (R-DINA) model, a parametric cognitive diagnosis model based on the NPC method that provides the same attribute profile classifications as the nonparametric method while allowing to derive a model likelihood and, subsequently, to compute fit and reliability indices. The suitability of the new proposal is examined by means of an exhaustive simulation study and a real data illustration. The results show that the R-DINA model properly recovers the posterior probabilities of attribute mastery, thus becoming a suitable alternative for comprehensive small-scale diagnostic assessments. [ABSTRACT FROM AUTHOR]

The analysis of sex differences in cognitive abilities is largely confusing because these differences are masked by the pervasive influence of the general factor of intelligence (g). In this study a battery of five reasoning tests (abstract [AR], numerical [NR], verbal [VR], mechanical [MR], and spatial [SR]) was completed by a sample of 3233 young and old adolescents representative of the population. Using a latent variable approach, mean differences on the general factor were estimated after examining measurement invariance. Results show that the difference, favoring boys in latent "g" increases with age from two to four IQ points. Further, boys outperform girls in all the subtests and the observed differences were generally explained by "g". However, mechanical reasoning is a systematic and strong exception to this finding. For the young adolescents, the observed difference in MR is equivalent to 10 IQ points, and this difference increases to 13 IQ points for the old adolescents. Only 1 (young) or 2 (old) IQ points of the sex difference in MR can be accounted for by "g". The findings suggest that the persistent--and usually neglected average large advantage of boys in mechanical reasoning (MR)--orthogonal to "g"--might be behind their higher presence in STEM (science, technology, engineering, and math) disciplines. A new look at this relevant social issue is proposed in this study. (Contains 2 tables and 2 figures.)

Multidimensional forced-choice (FC) questionnaires have been consistently found to reduce the effects of socially desirable responding and faking in noncognitive assessments. Although FC has been considered problematic for providing ipsative scores under the classical test theory, item response theory (IRT) models enable the estimation of nonipsative scores from FC responses. However, while some authors indicate that blocks composed of opposite-keyed items are necessary to retrieve normative scores, others suggest that these blocks may be less robust to faking, thus impairing the assessment validity. Accordingly, this article presents a simulation study to investigate whether it is possible to retrieve normative scores using only positively keyed items in pairwise FC computerized adaptive testing (CAT). Specifically, a simulation study addressed the effect of (a) different bank assembly (with a randomly assembled bank, an optimally assembled bank, and blocks assembled on-the-fly considering every possible pair of items), and (b) block selection rules (i.e., T, and Bayesian D and A -rules) over the estimate accuracy and ipsativity and overlap rates. Moreover, different questionnaire lengths (30 and 60) and trait structures (independent or positively correlated) were studied, and a nonadaptive questionnaire was included as baseline in each condition. In general, very good trait estimates were retrieved, despite using only positively keyed items. Although the best trait accuracy and lowest ipsativity were found using the Bayesian A -rule with questionnaires assembled on-the-fly, the T -rule under this method led to the worst results. This points out to the importance of considering both aspects when designing FC CAT. [ABSTRACT FROM AUTHOR]

The distance between nonparametric and parametric item characteristic curves has been proposed as an index of goodness of fit in item response theory in the form of a root integrated squared error index. This article proposes to use the posterior distribution of the latent trait as the nonparametric model and compares the performance of an index based on this method with another approach based on the kernel-smoothing model. Error rates and power are evaluated using the two-parameter logistic model and three types of realistic misfitted items. Results show that for fitting items, the distance between parametric and nonparametric item characteristic curves decreased as the sample size increased for both procedures. Kernel-smoothing root integrated squared error also decreased as test length increased. Bootstrap methods are used to obtain a significance test. Both procedures performed adequately in terms of Type I error rates. Regarding power, the posterior probabilities method was superior, especially in small samples, although in short tests both procedures performed in a similar way. (Contains 2 tables and 1 figure.)

Despite strong evidence supporting the use of Velicer's minimum average partial (MAP) method to establish the dimensionality of continuous variables, little is known about its performance with categorical data. Seeking to fill this void, the current study takes an in-depth look at the performance of the MAP procedure in the presence of ordinal-level measurement. Using Monte Carlo methods, seven factors related to the data (sample size, factor loading, number of variables per factor, number of factors, factor correlation, number of response categories, and skewness) as well as two factors related to the MAP method (type of correlation matrix and power) were systematically manipulated. The results indicate that using polychoric correlations and the squared partial correlations leads to considerably more accurate estimations than using Pearson correlations and/or raising the partial correlations to the fourth power. Additionally, the MAP method is shown to be a biased estimator of dimensionality in two conditions: (a) for low factor loadings (0.40) and (b) for medium factor loadings (0.55) and a small number of variables per factor (less than or equal to 6). The applicability of this method with categorical variables is discussed in the context of these findings. (Contains 4 tables, 4 figures, and 1 note.)

This article deals with some of the problems that have hindered the application of Samejima's and Thissen and Steinberg's multiple-choice models: (a) parameter estimation difficulties owing to the large number of parameters involved, (b) parameter identifiability problems in the Thissen and Steinberg model, and (c) their treatment of omitted responses. The authors propose a new multiple-choice model (Restricted Samejima Multiple-Choice Model) that is more explicit about the assumed omitting mechanisms and takes advantage of this knowledge to improve the estimation of parameters. The three above-mentioned models and the Nominal Response Model were fitted to a 3,224-subject sample that took a Written English Test. Fit plots, chi[superscript 2] statistics, and information-based fit indexes were obtained to assess the goodness of fit. Results show that the new model proposed fits as well as the other two multiple-choice models and better than the Nominal model. A parameter recovery simulation study was also carried out to confirm that its estimation features are indeed improved. Parameters are better recovered in the new model than in the other two multiple-choice models. (Contains 1 figure, 8 tables, and 1 note.)

This article proposes a new test (called the EQEBI) for the measurement of epistemological beliefs, integrating and extending the Epistemological Questionnaire (EQ) and the Epistemic Beliefs Inventory (EBI). In Study 1, the two tests were translated and applied to a Spanish-speaking sample. A detailed dimensionality exploration, by means of the monotone homogeneity model and confirmatory factor analysis, provides unexpected dimensionality results. Therefore, a new test is proposed, and the dimensionality and psychometric properties of the test's scores are determined. The new test retains four of the five original dimensions. The EQEBI has 27 items, and its score reliability is higher than that of the original EQ and EBI. In Study 2, the EQEBI was applied to another sample to verify the psychometric properties of the obtained scores. The expected four unidimensional scales are confirmed. The scales are calibrated with the graded response model. Recommendations are offered for using the scales for epistemological beliefs assessment. (Contains 7 tables.)

Working memory and the general factor of intelligence (g) are highly related constructs. However, we still don't know why. Some models support the central role of simple short-term storage, whereas others appeal to executive functions like the control of attention. Nevertheless, the available empirical evidence does not suffice to get an answer, presumably because relevant measures are frequently considered in isolation. To overcome this problem, here we consider concurrently simple short-term storage, mental speed, updating, and the control of attention along with working memory and intelligence measures, across three separate studies. Several diverse measures are administered to a total of 661 participants. The findings are consistent with the view that simple short-term storage largely accounts for the relationship between working memory and intelligence. Mental speed, updating, and the control of attention are not consistently related to working memory, and they are not genuinely associated with intelligence once the short-term storage component is removed. (Contains 9 figures and 3 tables.)

We investigated sex effects and the effects of educational attainment (EA) on the covariance structure of the WAIS-III in a subsample of the Spanish standardization data. We fitted both first order common factor models and second order common factor models. The latter include general intelligence ("g") as a second order common factor. The results indicate that sex differences in means are due mainly to the WAIS-III factors Working Memory (WM) and Perceptual Organization (PO), rather than to the second order "g" factor. In treating EA as a predictor of the first common factors, we found that EA explained a significant, but clearly varying, amount of variance in the 4 first order common factors. Differences in explained variance over sex were very small. Treating EA as a predictor in the second order common factor model, we found that EA predicted "g" and first order factor Verbal Comprehension (VC). The relationships between PO, WM, and Perceptual Speed (PS) and EA are mediated by "g." In treating EA as a dependent variable, we found that VC and PS were the only significant predictors of EA in the first order common factor model. In the second order common factor model, the same result was obtained. While VC and PS were significant predictors of EA, the second order factor "g," and PO and WM were not. The results are of theoretical interest in the light of the Ankney-Rushton paradox, and the role of "g" in educational attainment.

There are several studies showing that working memory and intelligence are strongly related. However, working memory tasks require simultaneous processing and storage, so the causes of their relationship with intelligence are currently a matter of discussion. The present study examined the simultaneous relationships among short-term memory (STM), working memory (WM), and general intelligence ("g"). Two hundred and eight participants performed six verbal, quantitative, and spatial STM tasks, six verbal, quantitative, and spatial WM tasks, and eight tests measuring fluid, crystallized, spatial, and quantitative intelligence. Especial care is taken to avoid misrepresenting the relations among the constructs being studied because of specific task variance. Structural equation modelling (SEM) results revealed that (a) WM and g are (almost) isomorphic constructs, (b) the isomorphism vanishes when the storage component of WM is partialed out, and (c) STM and WM (with its storage component partialed out) predict "g."

Results for 3,340 participants taking a battery of cognitive tests and an analysis of the Spanish standardization of the Wechsler Adult Intelligence Scale-III support the differentiation of intelligence across the range of ability, with WAIS-III results more supportive of the differentiation theory. (SLD)

Investigated whether average Full Scale IQ (FSIQ) differences can be attributed to "g" using the Spanish standardization sample of the Wechsler Adult Intelligence Scale III (WAIS III) (n=703 females and 666 men). Results support the conclusion that WAIS III FSIQ does not directly or exclusively measure "g" across the full range of population distribution of intelligence. (SLD)

Used the Spanish standardization of the Wechsler Adult Intelligence Scale III (WAIS III) (n=1,369) to test the age dedifferentiation hypothesis. Results show no changes in the percentage of variance accounted for by "g" and four group factors when restriction of range is controlled. Discusses an age indifferentation hypothesis. (SLD)