Search

This paper provides a literature review of assessment of fit of item response theory models. Various types of fit procedures for item response theory models are reviewed, with a focus on their advantages and disadvantages. Real data examples are used to demonstrate some of the fit procedures. Recommendations are provided for researchers and practitioners who are interested in assessing the fit of item response theory models. [ABSTRACT FROM AUTHOR]

Item Response Theory (IRT), over its nearly 100-year history, has become one of the most popular methodologies for modeling response patterns in measures in education, psychology and health. Due to its advantages, IRT is particularly popular in large-scale assessments. A pre-condition for the validity of the estimations obtained from IRT is that the data meet the model assumptions. The purpose of this study is to examine the testing of model assumptions in studies using IRT models. For this purpose, 107 studies in the National Thesis Center of the Council of Higher Education that use the IRT model on real data were examined. The studies were analyzed according to sample size, unidimensionality, local independence, overall model fit, item fit and non-speedness test criteria. According to the results, it was observed that the unidimensionality assumption was tested at a high level (89%) and Factor Analytic approaches were predominantly used. Local independence assumption was not tested in 36% of the studies, unidimensionality was cited as evidence in 40% of the studies and tested in 24% of the studies. Overall model fit was tested at a moderate level (51%) and Log-Likelihood and information criteria were used. Item fit and Non-Speedness testing were tested at a low level (26% and 9%). IRT assumptions should be considered as a whole and all assumptions should be tested from an evidence-based perspective. [ABSTRACT FROM AUTHOR]

Character measurement has become increasingly relevant and needed in recent years due to its significant impact on an individual’s success in various aspects of life. However, existing methods of character measurement such as self-assessment, peer assessment, and observation, still have many limitations. The Situational Judgement Test (SJT) is a behavioral measurement tool with the ability to minimise measurement bias. Therefore, this study aims to demonstrate the content and construct validity of the SJT-based PSP instrument and estimate its reliability. The research findings include an Aiken’s V index of 0.864, loading factor of all variables ≥ 0.5, Goodness of Fit statistics with χ2/df = 122.299/48, CFI = 0.938, TLI = 0.915, RMSEA = 0.066, and SRMR = 0.046, all within the good fit category. The empirical data of PSP align with the GRM model, with item fitness MNSQ method showing 59 items fit to the model. PSP has a Construct Reliability (CR) of 0.967 and an Average Variance Explained (AVE) of 0.830. These findings contribute theoretically and practically to the field of measurement, indicating that the SJT-based instrument is valid and reliable for measuring the PSP.

Background: Abdominal pain is a common and often debilitating issue for children and adolescents. In many cases, it is not caused by a specific somatic condition but rather emerges from a complex interplay of bio-psycho-social factors, leading to functional abdominal pain (FAP). Given the complex nature of FAP, understanding its origins and how to effectively manage this condition is crucial. Until now, however, no questionnaire exists that targets knowledge in this specific domain. To address this, the Abdominal Pain Knowledge Questionnaire (A-PKQ) was developed. Methods: Two versions were created (one for children and one for parents) and tested in four gastroenterology clinics and one specialized pain clinic in Germany between November 2021 and February 2024. Children between 8 and 17 years of age (N = 128) and their accompanying parents (N = 131) participated in the study. Rasch analysis was used to test the performance of both versions of the questionnaire. Results: The original questionnaires exhibited good model and item fit. Subsequently, both questionnaires were refined to improve usability, resulting in final versions containing 10 items each. These final versions also demonstrated good model and item fit, with items assessing a variety of relevant domains. Conclusion: The A-PKQ is an important contribution to improving assessment in clinical trials focused on pediatric functional abdominal pain. [ABSTRACT FROM AUTHOR]

Copyright of Psychological Science is the property of Psychological Science Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This article explores the occurrence and impact of local item dependence (LID) in a cloze test. LID refers to situations where responses to test items are influenced by other items. The researchers used the Rasch model to analyze a cloze test and identified three pairs of items that exhibited LID. Removing these items improved the test's fit and unidimensionality, but did not affect the overall ability of test takers. The study emphasizes the importance of detecting and addressing LID in language testing, while also suggesting the need for alternative approaches to handle LID without compromising the test's properties. [Extracted from the article]

Copyright of Humanities & Educational Sciences Journal is the property of Humanities & Educational Sciences Journal and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Viable methods for the identification of item misfit or Differential Item Functioning (DIF) are central to scale construction and sound measurement. Many approaches rely on the derivation of a limiting distribution under the assumption that a certain model fits the data perfectly. Typical DIF assumptions such as the monotonicity and population independence of item functions are present even in classical test theory but are more explicitly stated when using item response theory or other latent variable models for the assessment of item fit. The work presented here provides a robust approach for DIF detection that does not assume perfect model data fit, but rather uses Tukey's concept of contaminated distributions. The approach uses robust outlier detection to flag items for which adequate model data fit cannot be established. [ABSTRACT FROM AUTHOR]

Abstract When scaling psychological tests with methods of item response theory it is necessary to investigate to what extent the responses correspond to the model predictions. In addition to the statistical evaluation of item misfit, the question arises as to its practical significance. Although item removal is undesirable for several reasons, its practical consequences are rarely investigated and focus mostly on main survey data with pre-selected items. In this paper, we identify criteria to evaluate practical significance and discuss them with respect to various types of assessments and their particular purposes. We then demonstrate the practical consequences of item misfit using two data examples from the German PISA 2018 field trial study: one with cognitive data and one with non-cognitive/metacognitive data. For the former, we scale the data under the GPCM with and without the inclusion of misfitting items, and investigate how this influences the trait distribution and the allocation to reading competency levels. For non-cognitive/metacognitive data, we explore the effect of excluding misfitting items on estimated gender differences. Our results indicate minor practical consequences for person allocation and no changes in the estimated gender-difference effects.

Abstract Background Leisure satisfaction has been one of primary variables to explain an individual’s choice of leisure and recreational activities’ participation. The Leisure Satisfaction Scale (LSS)-short form has been widely utilized to measure leisure and recreation participants’ satisfaction levels. However, limited research has been studied on the LSS-short form that would provide sufficient evidence to use it to measure individual leisure satisfaction levels. Thus, the purpose of the study was to determine whether the LSS-short form would be appropriate to measure individuals’ leisure satisfaction levels. Method The convenience sampling was used in this study from the south-central United States. The LSS-short form questionnaire was administered to 436 individuals after removing 20 surveys due to incomplete questions. The WINSTEPS computer program was utilized to analyze the Rating scale fit; Item fit; Differential Item Functioning (DIF); and Person-Item map by utilizing Rasch rating scale model. Results The results indicated that the five-point Likert-type LSS-short form was appropriate to utilize. Two of 24 LSS-short form items had overfit or misfit and were eliminated. DIF indicated that all remained 22 items were suitable to measure leisure satisfaction levels. Overall, 22 item were finally selected for the reconstructed version of the LSS-short form. In addition, Person-Item map showed that ability and item difficulty were fit matched. Conclusions As the importance of leisure has been increased, the newly reconstructed LSS-short form would be recommended to evaluate individual leisure satisfaction levels in future studies. Furthermore, leisure and recreation professionals can provide and develop effective leisure activities or programs by measuring individual’s leisure satisfaction level with the new version of LSS-short form.

Item-level fit analysis not only serves as a complementary check to global fit analysis, it is also essential in scale development because the fit results will guide item revision and/or deletion (Liu & Maydeu-Olivares, 2014). During data collection, missing response data may likely happen due to various reasons. Chi-square-based item fit indices (e.g., Yen's Q 1 , McKinley and Mill's G 2 , Orlando and Thissen's S-X 2 and S-G 2 ) are the most widely used statistics to assess item-level fit. However, the role of total scores with complete data used in S-X 2 and S-G 2 is different from that with incomplete data. As a result, S-X 2 and S-G 2 cannot handle incomplete data directly. To this end, we propose several modified versions of S-X 2 and S-G 2 to evaluate item-level fit when response data are incomplete, named as M impute -X 2 and M impute -G 2 , of which the subscript " impute " denotes different imputation methods. Instead of using observed total scores for grouping, the new indices rely on imputed total scores by either a single imputation method or three multiple imputation methods (i.e., two-way with normally distributed errors, corrected item-mean substitution with normally distributed errors and response function imputation). The new indices are equivalent to S-X 2 and S-G 2 when response data are complete. Their performances are evaluated and compared via simulation studies; the manipulated factors include test length, sources of misfit, misfit proportion, and missing proportion. The results from simulation studies are consistent with those of Orlando and Thissen (2000, 2003), and different indices are recommended under different conditions. [ABSTRACT FROM AUTHOR]

When scaling psychological tests with methods of item response theory it is necessary to investigate to what extent the responses correspond to the model predictions. In addition to the statistical evaluation of item misfit, the question arises as to its practical significance. Although item removal is undesirable for several reasons, its practical consequences are rarely investigated and focus mostly on main survey data with pre-selected items. In this paper, we identify criteria to evaluate practical significance and discuss them with respect to various types of assessments and their particular purposes. We then demonstrate the practical consequences of item misfit using two data examples from the German PISA 2018 field trial study: one with cognitive data and one with non-cognitive/metacognitive data. For the former, we scale the data under the GPCM with and without the inclusion of misfitting items, and investigate how this influences the trait distribution and the allocation to reading competency levels. For non-cognitive/metacognitive data, we explore the effect of excluding misfitting items on estimated gender differences. Our results indicate minor practical consequences for person allocation and no changes in the estimated gender-difference effects. [ABSTRACT FROM AUTHOR]

Background: Leisure satisfaction has been one of primary variables to explain an individual's choice of leisure and recreational activities' participation. The Leisure Satisfaction Scale (LSS)-short form has been widely utilized to measure leisure and recreation participants' satisfaction levels. However, limited research has been studied on the LSS-short form that would provide sufficient evidence to use it to measure individual leisure satisfaction levels. Thus, the purpose of the study was to determine whether the LSS-short form would be appropriate to measure individuals' leisure satisfaction levels. Method: The convenience sampling was used in this study from the south-central United States. The LSS-short form questionnaire was administered to 436 individuals after removing 20 surveys due to incomplete questions. The WINSTEPS computer program was utilized to analyze the Rating scale fit; Item fit; Differential Item Functioning (DIF); and Person-Item map by utilizing Rasch rating scale model. Results: The results indicated that the five-point Likert-type LSS-short form was appropriate to utilize. Two of 24 LSS-short form items had overfit or misfit and were eliminated. DIF indicated that all remained 22 items were suitable to measure leisure satisfaction levels. Overall, 22 item were finally selected for the reconstructed version of the LSS-short form. In addition, Person-Item map showed that ability and item difficulty were fit matched. Conclusions: As the importance of leisure has been increased, the newly reconstructed LSS-short form would be recommended to evaluate individual leisure satisfaction levels in future studies. Furthermore, leisure and recreation professionals can provide and develop effective leisure activities or programs by measuring individual's leisure satisfaction level with the new version of LSS-short form. [ABSTRACT FROM AUTHOR]

In this article, statistical properties of the root mean square deviation (RMSD) item fit statistic in item response models are studied. It is shown that RMSD estimates will indicate even misfit for items whose parametric assumption of the item response function is correct (i.e., fitting items) if some item response functions in the test are misspecified. Moreover, it is demonstrated that the RMSD values of misfitting and fitting items depend on the proportion of misfitting items. We propose three alternative bias-corrected RMSD estimators that reduce the bias for fitting items. However, these alternative estimators provide slightly negatively biased estimates for misfitting items compared to the originally proposed RMSD statistic. In the numerical experiments, we study the case of a misspecified one-parameter logistic item response model and the behavior of the RMSD statistic if differential item functioning occurs. [ABSTRACT FROM AUTHOR]

The generalized S − X 2 –test is a test of item fit for items with polytomous responses format. The test is based on a comparison of the observed and expected number of responses in strata defined by the test score. In this article, we make four contributions. We demonstrate that the performance of the generalized S − X 2 –test depends on how sparse cells are pooled. We propose alternative implementations of the test within the framework of limited information testing. We derive the distribution of the S − X 2 –residuals that can be used for post hoc analyses. We suggest a diagnostic plot that visualizes the form of the misfit. The performance of the alternative implementations is investigated in a simulation study. The simulation study suggests that the alternative implementations are capable of controlling the Type-I error rate well and have high power. An empirical application concludes this article. [ABSTRACT FROM AUTHOR]

Purpose: In Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions. Methods: We conducted two simulation studies in the context of dichotomously scored item responses. We manipulated the type of assumption violation, the proportion of violating items, sample size, and quality. False positive rates and power to detect assumption violations were our main outcome variables. Furthermore, we used the Crit coefficient in a Mokken scale analysis to a set of responses to the General Health Questionnaire (GHQ-12), a self-administered questionnaire for assessing current mental health. Results: We found that the false positive rates of Crit were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Overall, in small samples Crit lacked the power to detect misfit, and in larger samples power differed considerably depending on the type of violation and proportion of misfitting items. Furthermore, we also found in our empirical example that even in large samples the Crit index may fail to detect assumption violations. Discussion: Even in large samples, the Crit coefficient showed limited usefulness for detecting moderate and severe violations of monotonicity. Our findings are relevant to researchers and practitioners who use Mokken scaling for scale and questionnaire construction and revision. [ABSTRACT FROM AUTHOR]

When scaling data using item response theory, valid statements based on the measurement model are only permissible if the model fits the data. Most item fit statistics used to assess the fit between observed item responses and the item responses predicted by the measurement model show significant weaknesses, such as the dependence of fit statistics on sample size and number of items. In order to assess the size of misfit and to thus use the fit statistic as an effect size, dependencies on properties of the data set are undesirable. The present study describes a new approach and empirically tests it for consistency. We developed an estimator of the distance between the predicted item response functions (IRFs) and the true IRFs by semiparametric adaptation of IRFs. For the semiparametric adaptation, the approach of extended basis functions due to Ramsay and Silverman (2005) is used. The IRF is defined as the sum of a linear term and a more flexible term constructed via basis function expansions. The group lasso method is applied as a regularization of the flexible term, and determines whether all parameters of the basis functions are fixed at zero or freely estimated. Thus, the method serves as a selection criterion for items that should be adjusted semiparametrically. The distance between the predicted and semiparametrically adjusted IRF of misfitting items can then be determined by describing the fitting items by the parametric form of the IRF and the misfitting items by the semiparametric approach. In a simulation study, we demonstrated that the proposed method delivers satisfactory results in large samples (i.e., N ≥ 1,000). [ABSTRACT FROM AUTHOR]

S − χ 2 is a popular item fit index that is available in commercial software packages such as flex MIRT. However, no research has systematically examined the performance of S − χ 2 for detecting item misfit within the context of the multidimensional graded response model (MGRM). The primary goal of this study was to evaluate the performance of S − χ 2 under two practical misfit scenarios: first, all items are misfitting due to model misspecification, and second, a small subset of items violate the underlying assumptions of the MGRM. Simulation studies showed that caution should be exercised when reporting item fit results of polytomous items using S − χ 2 within the context of the MGRM, because of its inflated false positive rates (FPRs), especially with a small sample size and a long test. S − χ 2 performed well when detecting overall model misfit as well as item misfit for a small subset of items when the ordinality assumption was violated. However, under a number of conditions of model misspecification or items violating the homogeneous discrimination assumption, even though true positive rates (TPRs) of S − χ 2 were high when a small sample size was coupled with a long test, the inflated FPRs were generally directly related to increasing TPRs. There was also a suggestion that performance of S − χ 2 was affected by the magnitude of misfit within an item. There was no evidence that FPRs for fitting items were exacerbated by the presence of a small percentage of misfitting items among them. [ABSTRACT FROM AUTHOR]

Testing whether items fit the assumptions of an item response theory model is an important step in evaluating a test. In the literature, numerous item fit statistics exist, many of which show severe limitations. The current study investigates the root mean squared deviation (RMSD) item fit statistic, which is used for evaluating item fit in various large-scale assessment studies. The three research questions of this study are (1) whether the empirical RMSD is an unbiased estimator of the population RMSD; (2) if this is not the case, whether this bias can be corrected; and (3) whether the test statistic provides an adequate significance test to detect misfitting items. Using simulation studies, it was found that the empirical RMSD is not an unbiased estimator of the population RMSD, and nonparametric bootstrapping falls short of entirely eliminating this bias. Using parametric bootstrapping, however, the RMSD can be used as a test statistic that outperforms the other approaches—infit and outfit, S − X 2—with respect to both Type I error rate and power. The empirical application showed that parametric bootstrapping of the RMSD results in rather conservative item fit decisions, which suggests more lenient cut-off criteria. [ABSTRACT FROM AUTHOR]

In this manuscript, the applicability of the Hausman test to the evaluation of item response models is investigated. The Hausman test is a general test of model fit. The test assesses whether for a model in question the parameter estimates of two different estimators coincide. The test can be implemented for item response models by comparing the parameter estimates of the marginal maximum likelihood estimator with the corresponding parameter estimates of a limited information estimator. For a correctly specified item response model, the difference of the two estimates is normally distributed around zero. The Hausman test can be used for the evaluation of item fit and global model fit. The performance of the test is evaluated in a simulation study. The simulation study suggests that the implemented versions of the test adhere to the nominal Type-I error rate well in samples of 1000 test takers and more. The test is also capable to detect misspecified item characteristic functions, but lacks power to detect violations of the conditional independence assumption.

In this manuscript, the applicability of the Hausman test to the evaluation of item response models is investigated. The Hausman test is a general test of model fit. The test assesses whether for a model in question the parameter estimates of two different estimators coincide. The test can be implemented for item response models by comparing the parameter estimates of the marginal maximum likelihood estimator with the corresponding parameter estimates of a limited information estimator. For a correctly specified item response model, the difference of the two estimates is normally distributed around zero. The Hausman test can be used for the evaluation of item fit and global model fit. The performance of the test is evaluated in a simulation study. The simulation study suggests that the implemented versions of the test adhere to the nominal Type-I error rate well in samples of 1000 test takers and more. The test is also capable to detect misspecified item characteristic functions, but lacks power to detect violations of the conditional independence assumption. [ABSTRACT FROM AUTHOR]

Open ended and multiple choice questions are commonly placed on the same tests; however, there is a discussion on the effects of using different item types on the test and item statistics. This study aims to compare model and item fit statistics in a mixed format test where multiple choice and constructed response items are used together. In this 25-item fourth grade science test administered to 2351 students in 35 schools in Turkey, items are calibrated separately and concurrently utilizing different IRT models. An important aspect of this study is that the effect of the calibration method on model and item fit is investigated on real data. Firstly, while the 1-, 2-, and 3-Parameter Logistic models are utilized to calibrate the binary coded items, the Graded Response Model and the Generalized Partial Credit Model are used to calibrate the open-ended ones. Then, combinations of dichotomous and polytomous models are employed concurrently. The results based on model comparisons revealed that the combination of the 3PL and the Graded Response Model produced the best fit statistics. [ABSTRACT FROM AUTHOR]

Abstract Latent regression models are used for score-reporting purposes in large-scale educational survey assessments such as the National Assessment of Educational Progress (NAEP) and Trends in International Mathematics and Science Study (TIMSS). One component of these models is based on item response theory. While there exists some research on assessment of fit of item response theory models in the context of large-scale assessments, there is a scope of further research on the topic. We suggest two types of residuals to assess the fit of item response theory models in the context of large-scale assessments. The Type I error rates and power of the residuals are computed from simulated data. The residuals are computed using data from four NAEP assessments. Misfit was found for all data sets for both types of residuals, but the practical significance of the misfit was minimal.

For assessing the fit of item response theory models, it has been suggested to apply overall goodness-of-fit tests as well as tests for individual items and item pairs. Although numerous goodness-of-fit tests have been proposed in the literature for the Rasch model, their relative power against several model violations has not been investigated so far. This study compares four of these tests, which are all available in R software: T10, T11, M2, and the LR test. Results on the Type I error rate and the sensitivity to violations of different assumptions of the Rasch model (unidimensionality, local independence on the level of item pairs, equal item discrimination, zero as a lower asymptote for the item characteristic curves, invariance of the item parameters) are reported. The results indicate that the T11 test is comparatively most powerful against violations of the assumption of parallel item characteristic curves, which includes the presence of unequal item discriminations and a non-zero lower asymptote. Against the remaining model violations, which can be summarized as local dependence, M2 is found to be most powerful. T10 and LR are found to be sensitive against violations of the assumption of parallel item characteristic curves, but are insensitive against local dependence.

Testing item-level fit is important in scale development to guide item revision/deletion. Many item-level fit indices have been proposed in literature, yet none of them were directly applicable to an important family of models, namely, the higher order item response theory (HO-IRT) models. In this study, chi-square-based fit indices (i.e., Yen's Q1, McKinley and Mill's G2, Orlando and Thissen's S-X2, and S-G2) were extended to HO-IRT models. Their performances are evaluated via simulation studies in terms of false positive rates and correct detection rates. The manipulated factors include test structure (i.e., test length and number of dimensions), sample size, level of correlations among dimensions, and the proportion of misfitting items. For misfitting items, the sources of misfit, including the misfitting item response functions, and misspecifying factor structures were also manipulated. The results from simulation studies demonstrate that the S-G2 is promising for higher order items. [ABSTRACT FROM AUTHOR]

In this study it was aimed to evaluate the effects of various factors such as sample sizes, percentage of misfit items in the test and item quality (item discrimination) on item and model fit in case of misspecification of Q matrix. Data were generated in accordance with DINA model. Q matrix was specified for 4 attributes and 15 items. While data were generated, sample sizes as 1000, 2000, 4000, s and g parameters as low and high discrimination index were manipulated. Three different misspecified Q matrix (overspecified, underspecified and mixed) was developed considering the percentage of misfit items (%20 and %40). In the study, S-X² was used as item fit statistics. Furthermore absolute (abs(fcor), max(X²)) and relative (-2 log-likelihood, Akaike's information criterion (AIC) and Bayesian information criterion (BIC)) model fit statistics were used. Investigating the results obtained from this simulation study, it was concluded that S-X² was sufficient to detect misfit items. When the percentage of misfit items in the test was much or Q matrix was both underspecified and overspecified, the correct detection of both abs(fcor) and max(X²) statistics was approximately 1 or 1. In addition, the correct detection rates of both statistics was high under other conditions, too. AIC and BIC were successful to detect model misfit in the cases where the Q matrix underspecified, whereas, they were failed detect model misfit for other cases. It can be said that the performance of BIC was mostly better than other relative model fit statistics to detect model misfit. [ABSTRACT FROM AUTHOR]

The psychometric properties of the 2 versions of the Junior Metacognitive Awareness Inventory were examined with Singapore student samples. Other than 2 misfitting items and an underutilized response scale, Rasch analysis demonstrated that the instruments have good measurement precision, and no differential item functioning was detected across gender or ethnic groups. [ABSTRACT FROM AUTHOR]

Testing item fit is an important step when calibrating and analyzing item response theory (IRT)-based tests, as model fit is a necessary prerequisite for drawing valid inferences from estimated parameters. In the literature, numerous item fit statistics exist, sometimes resulting in contradictory conclusions regarding which items should be excluded from the test. Recently, researchers argue to shift the focus from statistical item fit analyses to evaluating practical consequences of item misfit. This article introduces a method to quantify potential bias of relationship estimates (e.g., correlation coefficients) due to misfitting items. The potential deviation informs about whether item misfit is practically significant for outcomes of substantial analyses. The method is demonstrated using data from an educational test. [ABSTRACT FROM AUTHOR]

Background: The Multiple Sclerosis Walking Scale-12 (MSWS-12) measures walking ability from the patients’ perspective. We examined the quality of the MSWS-12 using an item response theory model, the graded response model (GRM). Methods: A total of 625 unique Dutch multiple sclerosis (MS) patients were included. After testing for unidimensionality, monotonicity, and absence of local dependence, a GRM was fit and item characteristics were assessed. Differential item functioning (DIF) for the variables gender, age, duration of MS, type of MS and severity of MS, reliability, total test information, and standard error of the trait level (θ) were investigated. Results: Confirmatory factor analysis showed a unidimensional structure of the 12 items of the scale, explaining 88% of the variance. Item 2 did not fit into the GRM model. Reliability was 0.93. Items 8 and 9 (of the 11 and 12 item version respectively) showed DIF on the variable severity, based on the Expanded Disability Status Scale (EDSS). However, the EDSS is strongly related to the content of both items. Conclusion: Our results confirm the good quality of the MSWS-12. The trait level (θ) scores and item parameters of both the 12- and 11-item versions were highly comparable, although we do not suggest to change the content of the MSWS-12. [ABSTRACT FROM AUTHOR]

Goodness of fit for item response theory (IRT) models in a frequentist and Bayesian framework are evaluated. The assumptions that are targeted are differential item functioning (DIF), local independence (LI), and the form of the item characteristics curve (ICC) in the one-, two-, and three parameter logistic models. It is shown that a Lagrange multiplier (LM) test, which is a frequentist based approach, can be defined in such a way that the statistics are based on the residuals, that is, differences between observations and their expectations under the model. In a Bayesian framework, identical residuals are used in posterior predictive checks. In a Bayesian framework, it proves convenient to use normal ogive representation of IRT models. For comparability of the two frameworks, the LM statistics are adapted from the usual logistic representation to normal ogive representation. Power and Type I error rates are evaluated using a number of simulation studies. Results show that Type I error rates are conservative in the Bayesian framework and that there is more power for the fit indices in a frequentist framework. An empirical data example is presented to show how the frameworks compare in practice. [ABSTRACT FROM AUTHOR]

Latent regression models are used for score-reporting purposes in large-scale educational survey assessments such as the National Assessment of Educational Progress (NAEP) and Trends in International Mathematics and Science Study (TIMSS). One component of these models is based on item response theory. While there exists some research on assessment of fit of item response theory models in the context of large-scale assessments, there is a scope of further research on the topic. We suggest two types of residuals to assess the fit of item response theory models in the context of large-scale assessments. The Type I error rates and power of the residuals are computed from simulated data. The residuals are computed using data from four NAEP assessments. Misfit was found for all data sets for both types of residuals, but the practical significance of the misfit was minimal. [ABSTRACT FROM AUTHOR]

Purpose: We developed the Arm Function in Multiple Sclerosis Questionnaire (AMSQ) to measure arm and hand function in MS, based on existing scales. We aimed at developing a unidimensional scale containing enough items to be used as an itembank. In this study, we investigated reliability and differential item functioning of the Dutch version.Method: Patients were recruited from two MS Centers and a Dutch website for MS patients. We performed item factor analysis on the polychoric correlation matrix, using multiple fit-indices to investigate model fit. The graded response model, an item response theory model, was used to investigate item goodness-of-fit, reliability of the estimated trait levels (θ), differential item functioning, and total information. Differential item functioning was investigated for type of MS, gender, administration version, and test length.Results: Factor analysis results suggested one factor. All items showedp-values of the item goodness-of-fit statistic above 0.0016. The reliability was 0.95, and no items showed differential item functioning on any of the investigated variables.Conclusion: AMSQ is a unidimensional 31-item questionnaire for measuring arm function in MS. Because of a well fit in a graded response model, it is suitable for further development as a computer adaptive test.Implications for RehabilitationA new questionnaire for arm and hand function recommended in people with multiple sclerosis (AMSQ).Scale characteristics make the questionnaire suitable for use in clinical practice and research.Good reliability.Further development as a computer adaptive test to reduce burden of (repetitive) testing in patients is feasible. [ABSTRACT FROM PUBLISHER]