Search

The frequently neglected and often misunderstood relationship between classical test theory and item response theory is discussed for the unidimensional case with binary measures and no guessing. It is pointed out that popular item response models can be directly obtained from classical test theory-based models by accounting for the discrete nature of the observed items. Two distinct observational equivalence approaches are outlined that render the item response models from corresponding classical test theory-based models, and can each be used to obtain the former from the latter models. Similarly, classical test theory models can be furnished using the reverse application of either of those approaches from corresponding item response models. [ABSTRACT FROM AUTHOR]

A latent variable modeling approach to evaluate scale reliability under realistic conditions in empirical behavioral and social research is discussed. The method provides point and interval estimation of reliability of multicomponent measuring instruments when several assumptions are violated. These assumptions include missing data, correlated errors, nonnormality, lack of unidimensionality, and data not missing at random. The procedure can be readily used to aid scale construction and development efforts in applied settings, and is illustrated using data from an educational study. [ABSTRACT FROM PUBLISHER]

Purpose: To evaluate the conformity of the Arm Activity measure (ArmA) passive function sub-scale to the Rasch model.Methods: A consecutive cohort of patients (n = 92) undergoing rehabilitation, including upper limb rehabilitation and spasticity management, at two specialist rehabilitation units were included. Rasch analysis was used to examine scaling and conformity to the model. Responses were analysed using Rasch unidimensional measurement models (RUMM 2030). The following aspects were considered: overall model and individual item fit statistics and fit residuals, internal reliability, item response threshold ordering, item bias, local dependency and unidimensionality.Results: ArmA contains both active and passive function sub-scales, but in this analysis only the passive function sub-scale was considered. Four of the seven items in the ArmA passive function sub-scale initially had disordered thresholds. These items were rescored to four response options, which resulted in ordered thresholds for all items. Once the items with disordered thresholds had been rescored, item bias was not identified for age, global disability level or diagnosis, but with a small difference in difficulty between males and females for one item of the scale. Local dependency was not observed and the unidimensionality of the sub-scale was supported and good fit to the Rasch model was identified. The person separation index (PSI) was 0.95 indicating that the scale is able to reliably differentiate at least two groups of patients.Conclusions: The ArmA passive function sub-scale was shown in this evaluation to conform to the Rasch model once disordered thresholds had been addressed. Using the logit scores produced by the Rasch model it was possible to convert this back to the original scale range.Implications for RehabilitationThe ArmA passive function sub-scale was shown, in this evaluation, to conform to the Rasch model once disordered thresholds had been addressed and therefore to be a clinically applicable and potentially useful hierarchical measure.Using Rasch logit scores it has be possible to convert back to the original ordinal scale range and provide an indication of real change to enable evaluation of clinical outcome of importance to patients and clinicians. [ABSTRACT FROM AUTHOR]

Este estudo avaliou o ajuste do modelo Rasch quanto à unidimensionalidade das Matrizes Progressivas Coloridas de Raven. Foram estudadas 441 crianças de ambos os sexos, de 1ª a 4ª série do ensino fundamental. Para avaliar a possibilidade de violação, optou-se pela análise de componentes principais de resíduos, pois eles podem ser suficientes para violar a unidimensionalidade intrínseca dos itens. Foram selecionados os itens de maior saturação em relação à dimensão positiva e negativa do primeiro fator, em relação ao teste como um todo e a cada série em particular. Esses itens foram novamente analisados pelo modelo de Rasch, separadamente, e correlacionadas as pontuações do fator positivo com o negativo. Os resultados sugeriram que o instrumento como um todo e a série B possibilitam a interpretação da existência de outra dimensão além da extraída pelo modelo de Rasch e nas outras duas séries as correlações estiveram próximas do ponto de rejeição.This study assessed the adjustment of the Rasch model concerning unidimensionality of Raven's Colored Standard Progressive Matrices. Four hundred and forty-one (441) elementary school children, of both genders, attending first to fourth grade were investigated. Considering that residuals may be enough to violate the intrinsic unidimensionality of the items, residual principal component analysis was used to assess the possibility of this violation. Items with higher factorial loading relating to the positive and negative dimensions on the first factor were selected, in respect to the test as a whole and to each subset in particular. These items were re-analyzed with the Rasch model within each data group. Then the positive and negative factor measures were correlated. Results suggested the possibility of other dimensions besides those extracted by Rasch model based on the test and subset B, though the correlation coefficients of the other subsets were close to rejection.

This study assessed the adjustment of the Rasch model concerning unidimensionality of Raven's Colored Standard Progressive Matrices. Four hundred and forty-one (441) elementary school children, of both genders, attending first to fourth grade were investigated. Considering that residuals may be enough to violate the intrinsic unidimensionality of the items, residual principal component analysis was used to assess the possibility of this violation. Items with higher factorial loading relating to the positive and negative dimensions on the first factor were selected, in respect to the test as a whole and to each subset in particular. These items were re-analyzed with the Rasch model within each data group. Then the positive and negative factor measures were correlated. Results suggested the possibility of other dimensions besides those extracted by Rasch model based on the test and subset B, though the correlation coefficients of the other subsets were close to rejection.

To test the applicability of item response theory (IRT) to the Korean Nurses' Licensing Examination (KNLE), item analysis was performed after testing the unidimensionality and goodness-of-fit. The results were compared with those based on classical test theory. The results of the 330-item KNLE administered to 12,024 examinees in January 2004 were analyzed. Unidimensionality was tested using DETECT and the goodness-of-fit was tested using WINSTEPS for the Rasch model and Bilog-MG for the two-parameter logistic model. Item analysis and ability estimation were done using WINSTEPS. Using DETECT, Dmax ranged from 0.1 to 0.23 for each subject. The mean square value of the infit and outfit values of all items using WINSTEPS ranged from 0.1 to 1.5, except for one item in pediatric nursing, which scored 1.53. Of the 330 items, 218 (42.7%) were misfit using the two-parameter logistic model of Bilog-MG. The correlation coefficients between the difficulty parameter using the Rasch model and the difficulty index from classical test theory ranged from 0.9039 to 0.9699. The correlation between the ability parameter using the Rasch model and the total score from classical test theory ranged from 0.9776 to 0.9984. Therefore, the results of the KNLE fit unidimensionality and goodness-of-fit for the Rasch model. The KNLE should be a good sample for analysis according to the IRT Rasch model, so further research using IRT is possible.

In the lead article, Davenport, Davison, Liou, & Love demonstrate the relationship among homogeneity, internal consistency, and coefficient alpha, and also distinguish among them. These distinctions are important because too often coefficient alpha--a reliability coefficient--is interpreted as an index of homogeneity or internal consistency. We argue that factor analysis should be conducted before calculating internal consistency estimates of reliability. If factor analysis indicates the assumptions underlying coefficient alpha are met, then it can be reported as a reliability coefficient. However, to the extent that items are multidimensional, alternative internal consistency reliability coefficients should be computed based on the parameter estimates of the factor model. Assuming a bifactor model evidenced good fit, and the measure was designed to assess a single construct, omega hierarchical--the proportion of variance of the total scores due to the general factor--should be presented. Omega--the proportion of variance of the total scores due to all factors--also should be reported in that it represents a more traditional view of reliability, although it is computed within a factor analytic framework. By presenting both these coefficients and potentially other omega coefficients, the reliability results are less likely to be misinterpreted. [ABSTRACT FROM AUTHOR]

I discuss the contribution by Davenport, Davison, Liou, & Love (2015) in which they relate reliability represented by coefficient α to formal definitions of internal consistency and unidimensionality, both proposed by Cronbach (1951). I argue that coefficient α is a lower bound to reliability and that concepts of internal consistency and unidimensionality, however defined, belong to the realm of validity, viz. the issue of what the test measures. Internal consistency and unidimensionality may play a role in the construction of tests when the theory of the attribute for which the test is constructed implies that the items be internally consistent or unidimensional. I also offer examples of attributes that do not imply internal consistency or unidimensionality, thus limiting these concepts' usefulness in practical applications. [ABSTRACT FROM AUTHOR]

A shortened version of the German adaptation of the morningness–eveningness scale of Horne and Östberg is analysed within a large sample of 994 physicians with respect to dimensionality, reliability, gender differences and validity. The psychometric analysis – which incorporates a highly robust method to check for unidimensionality – shows discrepancies towards unidimensionality and highlights three misfitting items. In addition, hypothesis testing indicates the presence of differential item functioning (DIF) with respect to gender which could be caused by differences in response formats. Although, reliability estimates are satisfactory, an overall lack of adequate psychometric properties of the scale within the population of physicians has to be reported. We derive suggestions for improvement of the original morningness–eveningness questionnaire (MEQ)-scale and provide general comments on how to check for unidimensionality without imposing a restrictive response model. [ABSTRACT FROM AUTHOR]

An important gap in the field of second language vocabulary assessment concerns the lack of validated tests measuring aural vocabulary knowledge. The primary purpose of this study is to introduce and provide preliminary validity evidence for the Listening Vocabulary Levels Test (LVLT), which has been designed as a diagnostic tool to measure knowledge of the first five 1000-word frequency levels and the Academic Word List (AWL). Quantitative analyses based on the Rasch model utilized several aspects of Messick’s validation framework. The findings indicated that (1) the items showed sufficient spread of difficulty, (2) the majority of the items displayed good fit to the Rasch model, (3) items and persons generally performed as predicted by a priori hypotheses, (4) the LVLT correlated with Parts 1 and 2 of the TOEIC listening test at .54, (5) the items displayed a high degree of unidimensionality, (6) the items showed a strong degree of measurement invariance with disattenuated Pearson correlations of .97 and .98 for person measures estimated with different sets of items, and (7) carelessness and guessing exerted only minor influences on test scores. Follow-up interviews and qualitative analyses indicated that the LVLT measures the intended construct of aural vocabulary knowledge, the format is easily understood, and the test has high face validity. This test fills an important gap in the field of second language vocabulary assessment by providing teachers and researchers with a way to assess aural vocabulary knowledge. [ABSTRACT FROM AUTHOR]

This study compared three common vocabulary test formats, the Yes/No test, the Vocabulary Knowledge Scale (VKS), and the Vocabulary Levels Test (VLT), as measures of vocabulary difficulty. Vocabulary difficulty was defined as the item difficulty estimated through Item Response Theory (IRT) analysis. Three tests were given to 165 Japanese students, resulting in five measures of vocabulary knowledge and four measures of word difficulty. Analyses included item and score factor analysis, unidimensionality, local independence, and correlations. Results indicate that these are reliable tests. Tests of unidimensionality suggest these tests are essentially measuring one major latent trait, which can be interpreted as a factor for word knowledge. Strong correlations of the scores with each other provide evidence of concurrent validity, and for the interpretation of the scores as indicative of word knowledge. Correlations with other methods of estimating word difficulty, such as transformed frequency, length of word, or number of syllables, suggest that of these methods, the log of frequencies from very large corpora gives the best estimate of word difficulty. However, direct testing of vocabulary difficulty appears to, in the words of Kreuz (1987), “provide a better account of recognition latencies than do counts based on printed word frequency” (p. 159). [ABSTRACT FROM AUTHOR]

Many commonly used item response models make the unidimensionality assumption of a single latent trait underlying the response data. The validity of this assumption needs to be tested before these models can be applied. One option is to use Stout’s non-parametric hypothesis test of essential unidimensionality, which is operationalized in the DIMTEST procedure. Although generally successful, Type I error rates of this procedure are usually deflated in small samples and inflated in large samples, while power can be low in small samples. A possible cause for the unfavorable Type I error rates and power may be that estimates of the sampling distribution, bias, and standard error of the test statistic are not sufficiently accurate in finite samples. In this study, five alternative hypothesis testing procedures were formulated that replace the (asymptotically correct) approximations in the current DIMTEST procedure with computational alternatives. The performance of these procedures was investigated in two simulation studies. One of these alternative procedures, which uses a conditional covariance statistic directly in a bootstrap hypothesis test, exhibited better controlled Type I errors and higher power than the current DIMTEST procedure in most conditions. Averaged over all sample sizes and correlations between two underlying dimensions, power increased by 5 percentage points for simple structure and by 7 percentage points for approximate simple structure. [ABSTRACT FROM AUTHOR]

Many studies have addressed the issue of dimensionality in activities of daily living (ADL) and its hierarchy, with contradictory results. This paper aims to study the unidimensional structure and the hierarchy of ADL in the Spanish Disability Survey. We analysed a sample of 8381 people aged 65 and over with at least one disability and receiving personal help with one of them. The following Rasch model properties were studied in 14 items enquiring about basic and instrumental ADL: fit to the model, reliability, unidimensionality, local independency between items and differential item functioning (DIF) by gender, age and proxy. The unidimensionality of the scale was confirmed. The final analysis of eight items showed a satisfactory fit, good reliability, local independency, unidimensionality and no DIF by age. The disability linear measure showed significant differences by gender and age. The obtained disability scale is a simple and reliable measure, and it suggests a hierarchical order of ADL and predicts a schedule of functional impairment in older adults. Rating the disability of older community-dwelling people has a predictive value that policy makers might find useful in service planning. [ABSTRACT FROM AUTHOR]

Internet surveys have become a very popular research tool. Relatively little attention has, however, been devoted to the possible changes in psychometric properties when measurements are obtained with Internet surveys. The Marlowe–Crowne Social Desirability Scale (MCSDS) is the most widely used instrument for measuring the tendency to respond in a socially desirable way and is often used to validate other measures. The purpose of the current research is to evaluate the dimensionality and reliability of measurements obtained with the MCSDS and short forms of the scale in an Internet sample of the general public in Iceland. An e-mail invitation was sent to a sample of 1200 panel members drawn from the Social Science Research Institute (SSRI) probability based panel, of those 536 participants completed all items on the MCSDS. Reliability estimates were in line with results from previous studies ( α = .81 for the MCSDS data and α ranging from .59 to .75 for short forms). Using confirmatory factor analysis, a good fit was obtained for a one-factor model of measurements obtained with the MCSDS and its short forms (apart from significant chi square values in all cases but one), which generally supports the assumption of unidimensionality. [ABSTRACT FROM AUTHOR]

Study Design. Cross-sectional design. Objective. To explore (1) the factor structure of the Roland-Morris Disability Questionnaire (RMDQ), (2) whether there is a dominant factor, and (3) whether the potential factors are unique predictors of other aspects related to back pain. Summary of Background Data. The RMDQ is one of the most recommended back-specific questionnaires assessing disability. The RMDQ is scored as a unidimensional scale summarizing answers to all 24 questions (Yes/No) regarding daily life functioning. However, there are indications that the scale is multidimensional. Methods. Patients (n = 457; age, 18-60 yr) with 8 to 12 weeks of back pain filled in questionnaires assessing subjective health complaints, emotional distress, instrumental and emotion-focused coping, and fear voidance behavior at baseline. A total of 371 patients (81.7%) filled in the RMDQ. Exploratory factor analysis was used to examine the factor structure of RMDQ items. Multiple regression analyses were used to assess whether the derived factors predicted relevant problems in back pain differently. Results. Exploratory factor analysis showed indices of model fit for a 3-factor solution after removing 2 items because of low prevalence (19 and 24). Two items were removed because of cross-loadings and low loadings (2 and 22). No support for a dominant factor was found as the 3 factors were only moderately correlated ( r = 0.34-0.40), and the ratio between the first and second eigenvalue was 2.6, not supporting essential unidimensionality. "Symptoms" were the factor that most strongly predicted subjective health complaints, whereas "avoidance of activity and participation" predicted fear avoidance behavior, instrumental and emotional coping. "Limitation in daily activities" did not predict any of these variables. Conclusion. The main findings of our study are that the RMDQ consists of 3 independent factors, and not 1 dominant factor as suggested previously. We think the time is now ripe to start treating and scoring the RMDQ as a multidimensional scale. [ABSTRACT FROM AUTHOR]

Purpose: The Medical Outcome Study Short Form 36 (SF-36) is one of the most commonly used questionnaires for monitoring the Health-Related Quality of Life (HRQOL) of opioid-dependent patients. However, the unidimensionality and reliability of the SF-36 have not been verified in opioid-dependent patients. The aim of this study was to examine the unidimensionality and to improve the test reliability of the SF-36 for use in opioid-dependent patients. Methods: A total of 583 opioid-dependent patients were recruited in the study. Unidimensionality was examined by conducting unidimensional Rasch analysis. Item fit statistics and principle component analysis were used to check the item-model fit in each of the eight subscales of the SF-36. Reliability was evaluated by applying both unidimensional and multidimensional Rasch analyses. Results: After three misfitting items were excluded, the remaining items of each subscale in the SF-36 represented a single construct. The test reliabilities (0.80-0.87) yielded by the multidimensional approach were much higher than those (0.68-0.82) produced by the unidimensional approach. Conclusion: The remaining 32 items of the SF-36 are appropriate for evaluating the HRQOL in opioid-dependent patients in terms of unidimensionality. Additionally, the test scores produced by the multidimensional approach were more accurate than those obtained by the unidimensional approach. [ABSTRACT FROM AUTHOR]

The purpose of this study was to construct a new scale for measuring foreign language classroom anxiety (FLCA). It begun with the creation of an extended item pool generated by qualitative methods. Subsequent Rasch and semantic analyses led to the final 18-item Foreign Language Classroom Anxiety Inventory (FLCAI). In comparison with the Foreign Language Classroom Anxiety Scale (FLCAS), the FLCAI demonstrated more convincing evidence of unidimensionality and the optimal 5-point Likert scale functioned better. The FLCAI, while 55% the length of the FLCAS, thus more practical for classroom practitioners to administer and analyse, maintains its psychometric properties and covers a wider range on the construct continuum thus improving the degree of validity of the instrument. Finally, test anxiety was shown to be a component of FLCA. [ABSTRACT FROM AUTHOR]

Background and aim Thyroid diseases are prevalent and chronic. With treatment, quality of life is restored in most, but not all patients. Construct validity of the thyroid-related quality of life questionnaire, ThyPRO, has been established by multi-trait scaling, but not evaluated with more elaborate methods. The purpose of the present study was to evaluate dimensionality of the ThyPRO scales and to attempt to understand possible item misfit through structural equation modeling for categorical data. Methods The current 84-item version of ThyPRO consists of 13 scales, covering domains of physical (4 scales) and mental (2 scales) symptoms, function and well-being (3 scales) and participation/social function (4 scales). The data were collected from a cross-sectional sample of 907 thyroid patients. One-factor confirmatory models were fitted to each scale, and evaluated by model fit statistics (comparative fit index >0.95, root mean square error of approximation <0.08), magnitude of factor loadings, model residual correlations and modification indices (MI). Indications of multi-dimensionality were tested in bi-factor models. Possible item misfit was evaluated in a combined, investigational model. Results Each ThyPRO scale was adequately represented by a unidimensional model after minor revisions. Eleven items were identified in the unidimensional models as potentially misfitting and were investigated further by multidimensional modeling. Conclusion Elaborate psychometric modeling supported the construct validity of the ThyPRO. However, 11 potentially misfitting items and 18 items with local dependence to other items are candidates for removal in future item reduction processes. [ABSTRACT FROM AUTHOR]

Abstract: The objective of this re-analysis of the European Genome-Based Therapeutic Drugs for Depression Study (GENDEP) was to psychometrically test the unidimensionality of the full Montgomery Åsberg Depression Rating Scale (MADRS10) and the Hamilton Depression Scale (HAM-D17) versus their respective subscales (MADRS5 and HAM-D6) containing the core symptoms of depression severity. Rasch analysis was applied using RUMM 2030 software to assess the overall fit for unidimensionality. Neither the MADRS10 nor the HAM-D17 was found to fit the Rasch model for unidimensionality. The HAM-D6 (containing the items of depressed mood, guilt, work and interests, psychomotor retardation, psychic anxiety, and somatic general) as well as the analogue MADRS5 were tested for unidimensionality by use of the RUMM 2030 programme, and only the HAM-D6 was accepted. When testing for invariance across rating weeks or centres, the RUMM 2030 had to be supplemented with the Friedman two-way analysis of variance by ranks. The HAM-D6 but not the MADRS5 was accepted. It was therefore concluded that the HAM-D6 is a psychometrically valid outcome scale to measure change in clinical trials of antidepressants. [Copyright &y& Elsevier]

Purpose: Many malignancy scores have been developed without comprehensive statistical or measurement validation, and in particular without verification of the necessary property of unidimensionality. Here, we used Rasch analysis to assess unidimensionality and identify measurement biases of malignancy scores. Methods: The Weiss histopathological system (WHS), summing nine items of histopathological alteration, was used to evaluate 247 adrenocortical tumors. Rasch model analysis was implemented and compared to classical factor analytic methods to investigate the validity of item-score summation for both the original and modified WHS, to assess differential functioning of the WHS items across various factors related to patient and tumors, and to identify items or subtypes of tumors that could be considered for removal or exclusion from the WHS with the aims of improving measurement and relieving the burden on pathologists. Results: The WHS does not meet the necessary property of unidimensionality and is severely affected by differential item functioning in relation to size and weight of the tumor. Moreover, items are not well distributed along the spectrum of malignancy, most being located in the upper part and several at the same place. Conclusion: The WHS in its present form should be applied only to small or moderate size tumors, and better scoring systems could be developed by using more appropriately distributed items. Rasch analysis is a powerful method for developing, evaluating, refining and simplifying malignancy scores. [ABSTRACT FROM AUTHOR]

We propose a class of multidimensional Item Response Theory models for polytomously-scored items with ordinal response categories. This class extends an existing class of multidimensional models for dichotomously-scored items in which the latent abilities are represented by a random vector assumed to have a discrete distribution, with support points corresponding to different latent classes in the population. In the proposed approach, we allow for different parameterizations for the conditional distribution of the response variables given the latent traits, which depend on the type of link function and the constraints imposed on the item parameters. Moreover, we suggest a strategy for model selection that is based on a series of steps consisting of selecting specific features, such as the dimension of the model (number of latent traits), the number of latent classes, and the specific parameterization. In order to illustrate the proposed approach, we analyze a dataset from a study on anxiety and depression on a sample of oncological patients. [ABSTRACT FROM PUBLISHER]

Background: The Schizophrenia Coping Oral Health Profile and Index (SCOOHPI) scale studies the coping strategies of schizophrenic patients with regard to oral health. The structural validity of this scale is studied has been studied using factor analyses. In this article, we study the unidimensionality of the SCOOHPI scale to use it as an index., Methods: We studied the internal consistency of the items of the SCOOHPI scale. Then, we studied the construct validity. The unidimensionality of the SCOOHPI scale was studied by the partial credit model., Results: The data used in this study come from five hospitals, and the total number of individuals participating in this study is 96, of which 72% are men and 59% are smokers. The SCOOHPI scale has good internal consistency (α = 0.84). The validity of divergence was checked by the absence of correlation between the SCOOHPI scale and the GOHAI (General Oral Health Assessment Index) scale. The unidimensionality of the SCOOHPI scale with data smoothing was demonstrated by the partial credit model., Conclusion: In this study, we completed the study of the psychometric validation of the SCOOHPI. The SCOOHPI scale can then contribute to improving evaluation of the coping strategies of schizophrenic patients with regard to oral health.

Heavy reliance on Cronbach's alpha has been standard practice in many validation studies. However, there seem to be two misconceptions about the interpretation of alpha. First, alpha is mistakenly considered as an indication of unidimensionality and second, that the higher the value of alpha the better. The aim of this study is to clarify these misconceptions with the use of real data from the educational setting. Results showed that high alpha values can be obtained in multidimensional scales or tests given a sufficient number of items. Therefore, alpha cannot be an indication of unidimensionality. At the same time, after a certain point, higher values of alpha do not necessarily mean higher reliability and better quality scales or tests. In fact very high values of alpha could bean indication of lengthy scales, parallel items or a narrow coverage of the construct under consideration. Researchers are advised to apply caution when reporting alpha. [ABSTRACT FROM AUTHOR]

The Digit Span subtest from the Wechsler Scales is used to measure Freedom from Distractibility or Working Memory. Some published research suggests that Digit Span forward should be interpreted differently from Digit Span backward. The present study explored the dimensionality of the Wechsler Memory Scale–III Digit Span (forward and backward) items in a sample of heterogeneous neuroscience patients (n = 267) using confirmatory factor analysis (CFA) for dichotomous items. Results suggested that four correlated factors underlie Digit Span, reflecting easy and hard items in both forward and backward presentation orders. The model for Digit Span was then cross-validated in a seizure disorders sample (n = 223) by replication of the CFA and by examination of measurement invariance. Measurement invariance tests of the precise numerical generalization of trait estimation across groups. Results supported measurement invariance and it was concluded that forward and backward digit span scores should be interpreted as measures of the same cognitive ability. [ABSTRACT FROM PUBLISHER]