Search

This paper provides a literature review of assessment of fit of item response theory models. Various types of fit procedures for item response theory models are reviewed, with a focus on their advantages and disadvantages. Real data examples are used to demonstrate some of the fit procedures. Recommendations are provided for researchers and practitioners who are interested in assessing the fit of item response theory models., (© 2025 The British Psychological Society.)

Background: Abdominal pain is a common and often debilitating issue for children and adolescents. In many cases, it is not caused by a specific somatic condition but rather emerges from a complex interplay of bio-psycho-social factors, leading to functional abdominal pain (FAP). Given the complex nature of FAP, understanding its origins and how to effectively manage this condition is crucial. Until now, however, no questionnaire exists that targets knowledge in this specific domain. To address this, the Abdominal Pain Knowledge Questionnaire (A-PKQ) was developed., Methods: Two versions were created (one for children and one for parents) and tested in four gastroenterology clinics and one specialized pain clinic in Germany between November 2021 and February 2024. Children between 8 and 17 years of age ( N = 128) and their accompanying parents ( N = 131) participated in the study. Rasch analysis was used to test the performance of both versions of the questionnaire., Results: The original questionnaires exhibited good model and item fit. Subsequently, both questionnaires were refined to improve usability, resulting in final versions containing 10 items each. These final versions also demonstrated good model and item fit, with items assessing a variety of relevant domains., Conclusion: The A-PKQ is an important contribution to improving assessment in clinical trials focused on pediatric functional abdominal pain.

Viable methods for the identification of item misfit or Differential Item Functioning (DIF) are central to scale construction and sound measurement. Many approaches rely on the derivation of a limiting distribution under the assumption that a certain model fits the data perfectly. Typical DIF assumptions such as the monotonicity and population independence of item functions are present even in classical test theory but are more explicitly stated when using item response theory or other latent variable models for the assessment of item fit. The work presented here provides a robust approach for DIF detection that does not assume perfect model data fit, but rather uses Tukey's concept of contaminated distributions. The approach uses robust outlier detection to flag items for which adequate model data fit cannot be established., Competing Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article., (© The Author(s) 2022.)

Item-level fit analysis not only serves as a complementary check to global fit analysis, it is also essential in scale development because the fit results will guide item revision and/or deletion (Liu & Maydeu-Olivares, 2014). During data collection, missing response data may likely happen due to various reasons. Chi-square-based item fit indices (e.g., Yen's Q 1 , McKinley and Mill's G 2 , Orlando and Thissen's S-X 2 and S-G 2 ) are the most widely used statistics to assess item-level fit. However, the role of total scores with complete data used in S-X 2 and S-G 2 is different from that with incomplete data. As a result, S-X 2 and S-G 2 cannot handle incomplete data directly. To this end, we propose several modified versions of S-X 2 and S-G 2 to evaluate item-level fit when response data are incomplete, named as M impute -X 2 and M impute -G 2 , of which the subscript " impute " denotes different imputation methods. Instead of using observed total scores for grouping, the new indices rely on imputed total scores by either a single imputation method or three multiple imputation methods (i.e., two-way with normally distributed errors, corrected item-mean substitution with normally distributed errors and response function imputation). The new indices are equivalent to S-X 2 and S-G 2 when response data are complete. Their performances are evaluated and compared via simulation studies; the manipulated factors include test length, sources of misfit, misfit proportion, and missing proportion. The results from simulation studies are consistent with those of Orlando and Thissen (2000, 2003), and different indices are recommended under different conditions., Competing Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article., (© The Author(s) 2022.)

Background: Leisure satisfaction has been one of primary variables to explain an individual's choice of leisure and recreational activities' participation. The Leisure Satisfaction Scale (LSS)-short form has been widely utilized to measure leisure and recreation participants' satisfaction levels. However, limited research has been studied on the LSS-short form that would provide sufficient evidence to use it to measure individual leisure satisfaction levels. Thus, the purpose of the study was to determine whether the LSS-short form would be appropriate to measure individuals' leisure satisfaction levels., Method: The convenience sampling was used in this study from the south-central United States. The LSS-short form questionnaire was administered to 436 individuals after removing 20 surveys due to incomplete questions. The WINSTEPS computer program was utilized to analyze the Rating scale fit; Item fit; Differential Item Functioning (DIF); and Person-Item map by utilizing Rasch rating scale model., Results: The results indicated that the five-point Likert-type LSS-short form was appropriate to utilize. Two of 24 LSS-short form items had overfit or misfit and were eliminated. DIF indicated that all remained 22 items were suitable to measure leisure satisfaction levels. Overall, 22 item were finally selected for the reconstructed version of the LSS-short form. In addition, Person-Item map showed that ability and item difficulty were fit matched., Conclusions: As the importance of leisure has been increased, the newly reconstructed LSS-short form would be recommended to evaluate individual leisure satisfaction levels in future studies. Furthermore, leisure and recreation professionals can provide and develop effective leisure activities or programs by measuring individual's leisure satisfaction level with the new version of LSS-short form., (© 2022. The Author(s).)

Purpose: In Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions., Methods: We conducted two simulation studies in the context of dichotomously scored item responses. We manipulated the type of assumption violation, the proportion of violating items, sample size, and quality. False positive rates and power to detect assumption violations were our main outcome variables. Furthermore, we used the Crit coefficient in a Mokken scale analysis to a set of responses to the General Health Questionnaire (GHQ-12), a self-administered questionnaire for assessing current mental health., Results: We found that the false positive rates of Crit were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Overall, in small samples Crit lacked the power to detect misfit, and in larger samples power differed considerably depending on the type of violation and proportion of misfitting items. Furthermore, we also found in our empirical example that even in large samples the Crit index may fail to detect assumption violations., Discussion: Even in large samples, the Crit coefficient showed limited usefulness for detecting moderate and severe violations of monotonicity. Our findings are relevant to researchers and practitioners who use Mokken scaling for scale and questionnaire construction and revision., (© 2021. The Author(s).)

When scaling data using item response theory, valid statements based on the measurement model are only permissible if the model fits the data. Most item fit statistics used to assess the fit between observed item responses and the item responses predicted by the measurement model show significant weaknesses, such as the dependence of fit statistics on sample size and number of items. In order to assess the size of misfit and to thus use the fit statistic as an effect size, dependencies on properties of the data set are undesirable. The present study describes a new approach and empirically tests it for consistency. We developed an estimator of the distance between the predicted item response functions (IRFs) and the true IRFs by semiparametric adaptation of IRFs. For the semiparametric adaptation, the approach of extended basis functions due to Ramsay and Silverman (2005) is used. The IRF is defined as the sum of a linear term and a more flexible term constructed via basis function expansions. The group lasso method is applied as a regularization of the flexible term, and determines whether all parameters of the basis functions are fixed at zero or freely estimated. Thus, the method serves as a selection criterion for items that should be adjusted semiparametrically. The distance between the predicted and semiparametrically adjusted IRF of misfitting items can then be determined by describing the fitting items by the parametric form of the IRF and the misfitting items by the semiparametric approach. In a simulation study, we demonstrated that the proposed method delivers satisfactory results in large samples (i.e., N ≥ 1,000)., (© 2020 The Authors. British Journal of Mathematical and Statistical Psychology published by John Wiley & Sons Ltd on behalf of British Psychological Society.)

S - χ 2 is a popular item fit index that is available in commercial software packages such as flex MIRT. However, no research has systematically examined the performance of S - χ 2 for detecting item misfit within the context of the multidimensional graded response model (MGRM). The primary goal of this study was to evaluate the performance of S - χ 2 under two practical misfit scenarios: first, all items are misfitting due to model misspecification, and second, a small subset of items violate the underlying assumptions of the MGRM. Simulation studies showed that caution should be exercised when reporting item fit results of polytomous items using S - χ 2 within the context of the MGRM, because of its inflated false positive rates (FPRs), especially with a small sample size and a long test. S - χ 2 performed well when detecting overall model misfit as well as item misfit for a small subset of items when the ordinality assumption was violated. However, under a number of conditions of model misspecification or items violating the homogeneous discrimination assumption, even though true positive rates (TPRs) of S - χ 2 were high when a small sample size was coupled with a long test, the inflated FPRs were generally directly related to increasing TPRs. There was also a suggestion that performance of S - χ 2 was affected by the magnitude of misfit within an item. There was no evidence that FPRs for fitting items were exacerbated by the presence of a small percentage of misfitting items among them., Competing Interests: Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article., (© The Author(s) 2020.)

In this manuscript, the applicability of the Hausman test to the evaluation of item response models is investigated. The Hausman test is a general test of model fit. The test assesses whether for a model in question the parameter estimates of two different estimators coincide. The test can be implemented for item response models by comparing the parameter estimates of the marginal maximum likelihood estimator with the corresponding parameter estimates of a limited information estimator. For a correctly specified item response model, the difference of the two estimates is normally distributed around zero. The Hausman test can be used for the evaluation of item fit and global model fit. The performance of the test is evaluated in a simulation study. The simulation study suggests that the implemented versions of the test adhere to the nominal Type-I error rate well in samples of 1000 test takers and more. The test is also capable to detect misspecified item characteristic functions, but lacks power to detect violations of the conditional independence assumption., (Copyright © 2020 Ranger and Much.)

For assessing the fit of item response theory models, it has been suggested to apply overall goodness-of-fit tests as well as tests for individual items and item pairs. Although numerous goodness-of-fit tests have been proposed in the literature for the Rasch model, their relative power against several model violations has not been investigated so far. This study compares four of these tests, which are all available in R software: T 10 , T 11 , M 2 , and the LR test. Results on the Type I error rate and the sensitivity to violations of different assumptions of the Rasch model (unidimensionality, local independence on the level of item pairs, equal item discrimination, zero as a lower asymptote for the item characteristic curves, invariance of the item parameters) are reported. The results indicate that the T 11 test is comparatively most powerful against violations of the assumption of parallel item characteristic curves, which includes the presence of unequal item discriminations and a non-zero lower asymptote. Against the remaining model violations, which can be summarized as local dependence, M 2 is found to be most powerful. T 10 and LR are found to be sensitive against violations of the assumption of parallel item characteristic curves, but are insensitive against local dependence.

Testing item-level fit is important in scale development to guide item revision/deletion. Many item-level fit indices have been proposed in literature, yet none of them were directly applicable to an important family of models, namely, the higher order item response theory (HO-IRT) models. In this study, chi-square-based fit indices (i.e., Yen's Q 1 , McKinley and Mill's G 2 , Orlando and Thissen's S-X 2 , and S-G 2 ) were extended to HO-IRT models. Their performances are evaluated via simulation studies in terms of false positive rates and correct detection rates. The manipulated factors include test structure (i.e., test length and number of dimensions), sample size, level of correlations among dimensions, and the proportion of misfitting items. For misfitting items, the sources of misfit, including the misfitting item response functions, and misspecifying factor structures were also manipulated. The results from simulation studies demonstrate that the S-G 2 is promising for higher order items., Competing Interests: Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Testing item fit is an important step when calibrating and analyzing item response theory (IRT)-based tests, as model fit is a necessary prerequisite for drawing valid inferences from estimated parameters. In the literature, numerous item fit statistics exist, sometimes resulting in contradictory conclusions regarding which items should be excluded from the test. Recently, researchers argue to shift the focus from statistical item fit analyses to evaluating practical consequences of item misfit. This article introduces a method to quantify potential bias of relationship estimates (e.g., correlation coefficients) due to misfitting items. The potential deviation informs about whether item misfit is practically significant for outcomes of substantial analyses. The method is demonstrated using data from an educational test., Competing Interests: Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Background: The Multiple Sclerosis Walking Scale-12 (MSWS-12) measures walking ability from the patients' perspective. We examined the quality of the MSWS-12 using an item response theory model, the graded response model (GRM)., Methods: A total of 625 unique Dutch multiple sclerosis (MS) patients were included. After testing for unidimensionality, monotonicity, and absence of local dependence, a GRM was fit and item characteristics were assessed. Differential item functioning (DIF) for the variables gender, age, duration of MS, type of MS and severity of MS, reliability, total test information, and standard error of the trait level (θ) were investigated., Results: Confirmatory factor analysis showed a unidimensional structure of the 12 items of the scale, explaining 88% of the variance. Item 2 did not fit into the GRM model. Reliability was 0.93. Items 8 and 9 (of the 11 and 12 item version respectively) showed DIF on the variable severity, based on the Expanded Disability Status Scale (EDSS). However, the EDSS is strongly related to the content of both items., Conclusion: Our results confirm the good quality of the MSWS-12. The trait level (θ) scores and item parameters of both the 12- and 11-item versions were highly comparable, although we do not suggest to change the content of the MSWS-12., (© The Author(s), 2016.)

Purpose: We developed the Arm Function in Multiple Sclerosis Questionnaire (AMSQ) to measure arm and hand function in MS, based on existing scales. We aimed at developing a unidimensional scale containing enough items to be used as an itembank. In this study, we investigated reliability and differential item functioning of the Dutch version., Method: Patients were recruited from two MS Centers and a Dutch website for MS patients. We performed item factor analysis on the polychoric correlation matrix, using multiple fit-indices to investigate model fit. The graded response model, an item response theory model, was used to investigate item goodness-of-fit, reliability of the estimated trait levels (θ), differential item functioning, and total information. Differential item functioning was investigated for type of MS, gender, administration version, and test length., Results: Factor analysis results suggested one factor. All items showed p-values of the item goodness-of-fit statistic above 0.0016. The reliability was 0.95, and no items showed differential item functioning on any of the investigated variables., Conclusion: AMSQ is a unidimensional 31-item questionnaire for measuring arm function in MS. Because of a well fit in a graded response model, it is suitable for further development as a computer adaptive test. Implications for Rehabilitation A new questionnaire for arm and hand function recommended in people with multiple sclerosis (AMSQ). Scale characteristics make the questionnaire suitable for use in clinical practice and research. Good reliability. Further development as a computer adaptive test to reduce burden of (repetitive) testing in patients is feasible.