Search

Despite strong evidence supporting the use of Velicer's minimum average partial (MAP) method to establish the dimensionality of continuous variables, little is known about its performance with categorical data. Seeking to fill this void, the current study takes an in-depth look at the performance of the MAP procedure in the presence of ordinal-level measurement. Using Monte Carlo methods, seven factors related to the data (sample size, factor loading, number of variables per factor, number of factors, factor correlation, number of response categories, and skewness) as well as two factors related to the MAP method (type of correlation matrix and power) were systematically manipulated. The results indicate that using polychoric correlations and the squared partial correlations leads to considerably more accurate estimations than using Pearson correlations and/or raising the partial correlations to the fourth power. Additionally, the MAP method is shown to be a biased estimator of dimensionality in two conditions: (a) for low factor loadings (0.40) and (b) for medium factor loadings (0.55) and a small number of variables per factor (less than or equal to 6). The applicability of this method with categorical variables is discussed in the context of these findings. (Contains 4 tables, 4 figures, and 1 note.)

In a typical study comparing the relative efficiency of two item selection rules in computerized adaptive testing, the common result is that they simultaneously differ in accuracy and security, making it difficult to reach a conclusion on which is the more appropriate rule. This study proposes a strategy to conduct a global comparison of two or more selection rules. A plot showing the performance of each selection rule for several maximum exposure rates is obtained and the whole plot is compared with other rule plots. The strategy was applied in a simulation study with fixed-length CATs for the comparison of six item selection rules: the point Fisher information, Fisher information weighted by likelihood, Kullback-Leibler weighted by likelihood, maximum information stratification with blocking, progressive and proportional methods. Our results show that there is no optimal rule for any overlap value or root mean square error (RMSE). The fact that a rule, for a given level of overlap, has lower RMSE than another does not imply that this pattern holds for another overlap rate. A fair comparison of the rules requires extensive manipulation of the maximum exposure rates. The best methods were the Kullback-Leibler weighted by likelihood, the proportional method, and the maximum information stratification method with blocking. (Contains 1 table and 3 figures.)

This article deals with some of the problems that have hindered the application of Samejima's and Thissen and Steinberg's multiple-choice models: (a) parameter estimation difficulties owing to the large number of parameters involved, (b) parameter identifiability problems in the Thissen and Steinberg model, and (c) their treatment of omitted responses. The authors propose a new multiple-choice model (Restricted Samejima Multiple-Choice Model) that is more explicit about the assumed omitting mechanisms and takes advantage of this knowledge to improve the estimation of parameters. The three above-mentioned models and the Nominal Response Model were fitted to a 3,224-subject sample that took a Written English Test. Fit plots, chi[superscript 2] statistics, and information-based fit indexes were obtained to assess the goodness of fit. Results show that the new model proposed fits as well as the other two multiple-choice models and better than the Nominal model. A parameter recovery simulation study was also carried out to confirm that its estimation features are indeed improved. Parameters are better recovered in the new model than in the other two multiple-choice models. (Contains 1 figure, 8 tables, and 1 note.)

This article proposes a new test (called the EQEBI) for the measurement of epistemological beliefs, integrating and extending the Epistemological Questionnaire (EQ) and the Epistemic Beliefs Inventory (EBI). In Study 1, the two tests were translated and applied to a Spanish-speaking sample. A detailed dimensionality exploration, by means of the monotone homogeneity model and confirmatory factor analysis, provides unexpected dimensionality results. Therefore, a new test is proposed, and the dimensionality and psychometric properties of the test's scores are determined. The new test retains four of the five original dimensions. The EQEBI has 27 items, and its score reliability is higher than that of the original EQ and EBI. In Study 2, the EQEBI was applied to another sample to verify the psychometric properties of the obtained scores. The expected four unidimensional scales are confirmed. The scales are calibrated with the graded response model. Recommendations are offered for using the scales for epistemological beliefs assessment. (Contains 7 tables.)

Compared easy and difficult versions of self-adapted tests (SAT) and computerized adapted tests. No significant differences were found among the tests for estimated ability or posttest state anxiety in studies with 187 Spanish high school students, although other significant differences were found. Discusses implications for interpreting test anxiety in the SAT condition. (SLD)

All previous reports on the phenomenon of flashbulb memories relate to hearing of shocking (or 'bad') news; in other words, that of negative affect. This study represents the first attempt to investigate whether those criteria used to define flashbulb memories would similarly apply to events of similar strength but of positive affect. 70 questionnaires were administered relating to 20 events over a 10-yr. period. No significant differences were found on the cardinal features of flashbulb memories for events of negative versus positive affect. This suggests that an hitherto untapped research area may be explored to clarify controversial issues within this construct such as whether a special mechanism exists in the formation of flashbulb memories.

Byline: Steven L. Wise, Vicente Ponsoda, Julio Olea A self-adapted test is a computerised test that allows examinees to choose the difficulty levels of the items they have to answer. This paper describes its basic principles and the main research results about its effects on examinees' performance and well-being. Despite some divergent results and compared to computerised adaptive testing, it appears safe to conclude that self-adapted testing decreases anxiety, reduces the correlation between anxiety and proficiency, increases testing time and sometimes increases examinee mean proficiency. Its advantages and liabilities are discussed, along with implications for measurement practice.

One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies. [ABSTRACT FROM AUTHOR]

Copyright of Revista de Psicologia del Trabajo y de Las Organizaciones is the property of Colegio Oficial de Psicologos de Madrid and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This study analyses the extent to which cheating occurs in a real selection setting. A two-stage, unproctored and proctored, test administration was considered. Test score inconsistencies were concluded by applying a verification test (Guo and Drasgow Z-test). An initial simulation study showed that the Z-test has adequate Type I error and power rates in the specific selection settings explored. A second study applied the Z-test statistic verification procedure to a sample of 954 employment candidates. Additional external evidence based on item time response to the verification items was gathered. The results revealed a good performance of the Z-test statistic and a relatively low, but non-negligible, number of suspected cheaters that showed higher distorted ability estimates. The study with real data provided additional information on the presence of suspected cheating in unproctored applications and the viability of using item response times as an additional evidence of cheating. In the verification test, suspected cheaters spent 5.78 seconds per item more than expected considering the item difficulty and their assumed ability in the unproctored stage. We found that the percentage of suspected cheaters in the empirical study could be estimated at 13.84%. In summary, the study provides evidence of the usefulness of the Z-test in the detection of cheating in a specific setting, in which a computerized adaptive test for assessing English grammar knowledge was used for personnel selection. [ABSTRACT FROM AUTHOR]

Forced-choice questionnaires have been proposed as a way to control some response biases associated with traditional questionnaire formats (e.g., Likert-type scales). Whereas classical scoring methods have issues of ipsativity, item response theory (IRT) methods have been claimed to accurately account for the latent trait structure of these instruments. In this article, the authors propose the multi-unidimensional pairwise preference two-parameter logistic (MUPP-2PL) model, a variant within Stark, Chernyshenko, and Drasgow’s MUPP framework for items that are assumed to fit a dominance model. They also introduce a Markov Chain Monte Carlo (MCMC) procedure for estimating the model’s parameters. The authors present the results of a simulation study, which shows appropriate goodness of recovery in all studied conditions. A comparison of the newly proposed model with a Brown and Maydeu’s Thurstonian IRT model led us to the conclusion that both models are theoretically very similar and that the Bayesian estimation procedure of the MUPP-2PL may provide a slightly better recovery of the latent space correlations and a more reliable assessment of the latent trait estimation errors. An application of the model to a real data set shows convergence between the two estimation procedures. However, there is also evidence that the MCMC may be advantageous regarding the item parameters and the latent trait correlations. [ABSTRACT FROM AUTHOR]

In order for practitioners to be able to use tests appropriately, they must have rigorous information on the quality of the tests. This is why the Spanish test review model (Prieto & Muñiz, 2000) has been applied for a number of years. The goal of this paper is to update and revise this model in order to incorporate the recommendations that have been provided since the original model was applied and to incorporate the latest psychometric and technological innovations. The original model was revised following a series of steps, and the revised proposal was reviewed by a number of experts. After incorporating their suggestions, we have arrived at the final version, which is described in this paper. With the application of the revised model, and the publication of the corresponding results, we hope to continue to improve the use of tests, and consequently, the professional practice of psychology. [ABSTRACT FROM AUTHOR]

For professionals to use tests adequately, they must have rigorous information about the quality of the tests. This is why during the last years, the Spanish test review model (Prieto & Muñiz, 2000) has been applied. The goal of this paper is to update and revise this model in order to incorporate the recommendations given after the original model was applied and, to incorporate the latest psychometric and technological innovations. The original model was revised following different steps, and the revised proposal was reviewed by a number of experts. After incorporating their suggestions, we arrived to the final version, which is described in this paper. With the application of the revised model, and the publication of the corresponding results, we expect to continue improving the use of tests, and consequently, the professional practice of Psychology. [ABSTRACT FROM AUTHOR]

Background: Multistage adaptive testing has recently emerged as an alternative to the computerized adaptive test. The current study details a new multistage test to assess fluid intelligence. Method: An item pool of progressive matrices with constructed response format was developed, and divided into six subtests. The subtests were applied to a sample of 724 college students and their psychometric properties were studied (i.e., reliability, dimensionality and validity evidence). The item pool was calibrated under the graded response model, and two multistage structures were developed, based on the automatic test assembly principles. Finally, the test information provided by each structure was compared in order to select the most appropriate one. Results: The item pool showed adequate psychometric properties. From the two compared multistage structures, the simplest structure (i.e., routing test and two modules in the next stages) were more informative across the latent trait continuum and were therefore kept. Discussion: Taken together, the results of the two studies support the application of the F1MT (Fluid Intelligence Multistage Test), a multistage test to assess fluid intelligence accurately and innovatively. [ABSTRACT FROM AUTHOR]

Artificial neural networks are sophisticated modelling and prediction tools capable of extracting complex, non-linear relationships between predictor (input) and predicted (output) variables. This study explores this capacity by modelling non-linearities in the hardiness-modulated burnout process with a neural network. Specifically, two multi-layer feed-forward artificial neural networks are concatenated in an attempt to model the composite non-linear burnout process. Sensitivity analysis, a Monte Carlo–based global simulation technique, is then utilised to examine the first-order effects of the predictor variables on the burnout sub-dimensions and consequences. Results show that (1) this concatenated artificial neural network approach is feasible to model the burnout process, (2) sensitivity analysis is a prolific method to study the relative importance of predictor variables and (3) the relationships among variables involved in the development of burnout and its consequences are to different degrees non-linear. Practitioner Summary: Many relationships among variables (e.g. stressors and strains) are not linear, yet researchers use linear methods such as Pearson correlation or linear regression to analyse these relationships. Artificial neural network analysis is an innovative method to analyse non-linear relationships and in combination with sensitivity analysis superior to linear methods. [ABSTRACT FROM AUTHOR]

Despite the globalization of HRM, there is a dearth of research on the potential use of contextualized selection instruments such as situational judgment tests (SJTs) in other countries than those where the selection instruments were originally developed. Therefore, two studies are conducted to examine the transportability of an integrity SJT that was originally developed in the United States to a Spanish context. Study 1 showed that most SJT scenarios (16 out of 19) that were developed in the United States were also considered realistic in a Spanish context. In Study 2, the item option endorsement patterns converged to the original scoring scheme, with the exception of two items. In addition, there were high correlations between the original US empirical scoring scheme and two empirical scoring schemes that were tailored to the Spanish context (i.e., mode consensus scoring and proportional consensus scoring). Finally, correlations between the SJT integrity scores and ratings on a self-report integrity measure did not differ significantly from each other according to the type of scoring key (original US scoring vs. Spanish scoring keys). Overall, these results shed light on potential issues and solutions related to the cross-cultural use of contextualized selection instruments such as SJTs. [ABSTRACT FROM AUTHOR]

This article explores how traditional scores obtained from different forced-choice (FC) formats relate to their true scores and item response theory (IRT) estimates. Three FC formats are considered from a block of items, and respondents are asked to (a) pick the item that describes them most (PICK), (b) choose the two items that describe them the most and the least (MOLE), or (c) rank all the items in the order of their descriptiveness of the respondents (RANK). The multi-unidimensional pairwise-preference (MUPP) model, which is extended to more than two items per block and different FC formats, is applied to obtain the responses to each item block. Traditional and IRT (i.e., expected a posteriori) scores are computed from each data set and compared. The aim is to clarify the conditions under which simpler traditional scoring procedures for FC formats may be used in place of the more appropriate IRT estimates for the purpose of inter-individual comparisons. Six independent variables are considered: response format, number of items per block, correlation between the dimensions, item discrimination level, and sign-heterogeneity and variability of item difficulty parameters. Results show that the RANK response format outperforms the other formats for both the IRT estimates and traditional scores, although it is only slightly better than the MOLE format. The highest correlations between true and traditional scores are found when the test has a large number of blocks, dimensions assessed are independent, items have high discrimination and highly dispersed location parameters, and the test contains blocks formed by positive and negative items. [ABSTRACT FROM AUTHOR]

Background. In order to carry out a rigorous psychological evaluation, three conditions must be met: the practitioners must have the appropriate qualifications, the tests must show good psychometric properties, and they must be used correctly. The aim of this paper is to present the recent guidelines developed by the International Test Commission on the use of tests in three areas: research, quality control, and security. Method. The guidelines developed by the International Test Commission will be analysed and discussed. Results. The new guidelines on the use of tests offer a whole range of theoretical and practical recommendations to guide the appropriate use of tests in research settings, in order to develop and implement effective quality control strategies, and to preserve the security of all of the data involved in the assessment process. Conclusions. The new guidelines developed by the International Test Commission will contribute to the correct use of tests in research settings, to an improvement in the quality control of testing, and to ensuring security in assessment processes. [ABSTRACT FROM AUTHOR]

Copyright of Papeles del Psicólogo is the property of Consejo General de Colegios Oficiales de Psicologos and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Psicológica is the property of University of Valencia, Department of Methodology of the Behavioral Sciences and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This article describes the results of the second evaluation of psychological tests published in Spain. The Committee on Testing of the Spanish Psychological Association agreed on assessing 12 tests, selected mainly for their novelty and wide use. Each test has been evaluated by two experts. As in the first evaluation (Muñiz, Fernández-Hermida, Fonseca-Pedrero, Campillo-Álvarez and Peña-Suárez, 2011), assessments were made by responding to the Questionnaire for the Assessment of Tests (Prieto and Muñiz, 2000), which adapts to the Spanish context the assessment model developed by the European Federation of Psychologists' Associations. Results are provided in both absolute and relative terms, as they are compared to those of the first evaluation. They refer to the quality of documentation and materials, the coverage of the validation studies, reliability, norms, etc. Reviewers were also asked about the suitability of the instrument and procedure used to conduct the assessment. Suggestions are provided that may be useful to improve next test evaluations. [ABSTRACT FROM AUTHOR]

Intimate partner violence (IPV) among Latinos is related to economic problems, acculturation stress, alcohol use, impulsivity, and trauma. Little is known of the role of abusive tactics to explain more severe acts of violence. This study, conducted with 73 Latina victims, aims to describe the role of emotional, physical, and sexual abuse on the escalation of violence, and find indicators to predict severe IPV and lethality. Analyses included bivariate correlations, multiple regressions, and analyses of variance (ANOVAs). Results indicate that emotional IPV is the gateway to more severe abuse (physical and sexual). Victims of sexual abuse are at higher risk. Victims' fear of a partner is a useful indicator to assess the severity of IPV and the risk of lethality. [ABSTRACT FROM AUTHOR]

The most commonly employed item selection rule in a computerized adaptive test (CAT) is that of selecting the item with the maximum Fisher information for the estimated trait level. This means a highly unbalanced distribution of item-exposure rates, a high overlap rate among examinees and, for item bank management, strong pressure to replace items with a high discrimination parameter in the bank. An alternative for mitigating these problems involves, at the beginning of the test, basing item selection mainly on randomness. As the test progresses, the weight of information in the selection increases. In the present work we study, for two selection rules, the progressive methods (Revuelta & Ponsoda, 1998) and the proportional method (Segall, 2004a), different functions that define the weight of the random component according to the position in the test of the item to be administered. The functions were tested in simulated item banks and in an operative bank.We found that both the progressive and the proportional methods tolerate a high weight of the random component with minimal or zero loss of accuracy, while bank security and maintenance are improved. [ABSTRACT FROM AUTHOR]

Investigates a self-adapted English vocabulary test in a Spanish high school population for psychological measurement. Item pool description; Descriptive statistics for the item parameter distributions; Description of a testing session; Comparison among test types.

Describes an easy to use computer program for computer-adaptive testing. Program checking through simulation; Data on the accuracy of ability estimation; Instructions on running the program.

This paper presents a new two-dimensional Multiple-Choice Model accounting for Omissions (MCMO). Based on Thissen and Steinberg multiple-choice models, the MCMO defines omitted responses as the result of the respondent not knowing the correct answer and deciding to omit rather than to guess given a latent propensity to omit. Firstly, using a Monte Carlo simulation, the accuracy of the parameters estimated from data with different sample sizes (500, 1,000, and 2,000 subjects), test lengths (20, 40, and 80 items) and percentages of omissions (5, 10, and 15%) were investigated. Later, the appropriateness of the MCMO to the Trends in International Mathematics and Science Study (TIMSS) Advanced 2015 mathematics and physics multiple-choice items was analyzed and compared with the Holman and Glas' Between-item Multi-dimensional IRT model (B-MIRT) and with the three-parameter logistic (3PL) model with omissions treated as incorrect responses. The results of the simulation study showed a good recovery of scale and position parameters. Pseudo-guessing parameters ( d ) were less accurate, but this inaccuracy did not seem to have an important effect on the estimation of abilities. The precision of the propensity to omit strongly depended on the ability values (the higher the ability, the worse the estimate of the propensity to omit). In the empirical study, the empirical reliability for ability estimates was high in both physics and mathematics. As in the simulation study, the estimates of the propensity to omit were less reliable and their precision varied with ability. Regarding the absolute item fit, the MCMO fitted the data better than the other models. Also, the MCMO offered significant increments in convergent validity between scores from multiple-choice and constructed-response items, with an increase of around 0.02 to 0.04 in R 2 in comparison with the two other methods. Finally, the high correlation between the country means of the propensity to omit in mathematics and physics suggests that (1) the propensity to omit is somehow affected by the country of residence of the examinees, and (2) the propensity to omit is independent of the test contents.