Search

Background: The Huntington's Disease (HD) Everyday Functioning (Hi-DEF) is a new patient-reported outcome (PRO) instrument designed to measure the impact of cognitive impairment on daily functioning in the early stages of HD. Objective: To assess the measurement properties and finalize item content of the Hi-DEF. Methods: A cross-sectional, observational psychometric validation study was conducted among individuals with early stages of HD at 9 US centers of excellence. Rasch Measurement Theory (RMT) analysis of the initial draft version of the Hi-DEF (47 items) and subscales (i.e., 'Home', 'At work', 'Driving', and 'Communication') was conducted to examine measurement properties including sample-to-scale targeting, suitability of response scale (ordering of response thresholds), scale cohesiveness (item fit), local independence, and person fit. Results: 151 participants (mean age 47 years (SD 12), 59% female) were included. Seven items were removed based on dependency and item fit. The remaining 40-item version of the Hi-DEF demonstrated good measurement properties. Across the four subscales, targeting ranged from 49–70% (72% full scale), reliability ascertained by person separation index ranged from 0.53–0.87 (0.92 full scale), response scales were ordered for 25–100% of items (75% full scale), 0–12% items displayed misfit (2% full scale), and 0–1% (2% full scale) item pairs displayed dependency. Conclusions: Our study supports the psychometric integrity of the Hi-DEF as a reliable and valid new PRO instrument designed to assess the impact of cognitive impairment on daily functioning in the early stages of HD. Future work will evaluate the external validity and utility in clinical trial applications. [ABSTRACT FROM AUTHOR]

Evaluating scale comparability in international large-scale assessments depends on measurement invariance (MI). The root mean square deviation (RMSD) is a standard method for establishing MI in several programs, such as the Programme for International Student Assessment and the Programme for the International Assessment of Adult Competencies. Previous research showed that the RMSD was unable to detect departures from MI when the latent trait distribution was far from item difficulty. In this study, we developed three alternative approaches to the original RMSD: equal, item information, and b -norm weighted RMSDs. Specifically, we considered the item-centered normalized weight distributions to compute the item characteristic curve difference in the RMSD procedure more efficiently. We further compared all methods' performance via a simulation study and the item information and b -norm weighted RMSDs showed the most promising results. An empirical example is demonstrated, and implications for researchers are discussed. [ABSTRACT FROM AUTHOR]

The S- X 2 statistic (Orlando & Thissen, 2000) is popular among researchers and practitioners who are interested in the assessment of item fit. However, the statistic suffers from the Chernoff–Lehmann problem (Chernoff & Lehmann, 1954) and hence does not have a known asymptotic null distribution. This paper suggests a modified version of the S- X 2 statistic that is based on the modified Rao–Robson χ 2 statistic (Rao & Robson, 1974). A simulation study and a real data analyses demonstrate that the use of the modified statistic instead of the S- X 2 statistic would lead to fewer items being flagged for misfit. [ABSTRACT FROM AUTHOR]

Item-level fit analysis not only serves as a complementary check to global fit analysis, it is also essential in scale development because the fit results will guide item revision and/or deletion (Liu & Maydeu-Olivares, 2014). During data collection, missing response data may likely happen due to various reasons. Chi-square-based item fit indices (e.g., Yen's Q 1 , McKinley and Mill's G 2 , Orlando and Thissen's S-X 2 and S-G 2 ) are the most widely used statistics to assess item-level fit. However, the role of total scores with complete data used in S-X 2 and S-G 2 is different from that with incomplete data. As a result, S-X 2 and S-G 2 cannot handle incomplete data directly. To this end, we propose several modified versions of S-X 2 and S-G 2 to evaluate item-level fit when response data are incomplete, named as M impute -X 2 and M impute -G 2 , of which the subscript " impute " denotes different imputation methods. Instead of using observed total scores for grouping, the new indices rely on imputed total scores by either a single imputation method or three multiple imputation methods (i.e., two-way with normally distributed errors, corrected item-mean substitution with normally distributed errors and response function imputation). The new indices are equivalent to S-X 2 and S-G 2 when response data are complete. Their performances are evaluated and compared via simulation studies; the manipulated factors include test length, sources of misfit, misfit proportion, and missing proportion. The results from simulation studies are consistent with those of Orlando and Thissen (2000, 2003), and different indices are recommended under different conditions. [ABSTRACT FROM AUTHOR]

A novel approach to item-fit analysis based on an asymptotic test is proposed. The new test statistic, χ w 2 , compares pseudo-observed and expected item mean scores over a set of ability bins. The item mean scores are computed as weighted means with weights based on test-takers' a posteriori density of ability within the bin. This article explores the properties of χ w 2 in case of dichotomously scored items for unidimensional IRT models. Monte Carlo experiments were conducted to analyze the performance of χ w 2 . Type I error of χ w 2 was acceptably close to the nominal level and it had greater power than Orlando and Thissen's S - x 2 . Under some conditions, power of χ w 2 also exceeded the one reported for the computationally more demanding Stone's χ 2 ∗ . [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this research is to address the need for a robust system to accurately determine a cutoff score by using the Angoff method and leveraging the Rasch infit and outfit statistics of item response theory by detecting and removing misfitting items in a test. Design/methodology/approach: Researchers in educational evaluation support the use of grades as a complete measure to evaluate students' overall academic performance. In higher education, determining cutoff scores is crucial for assessing academic progress and achievement. Higher education institutions worldwide set predefined minimal standards of success, sometimes referred to as cutoff thresholds. These thresholds serve as benchmarks for assessing performance quality and determining eligibility for academic credit or advancement. Findings: The proposed approach combines the Angoff method with Rasch item fit statistics to precisely determine the cutoff score by excluding items that do not fit well (misfitting items) in a test. This methodology holds the potential to enhance the fairness and accuracy of grading practices, ensuring that assessments align with educational objectives and provide meaningful feedback to students. Originality/value: The value of this research extends to various fields where cutoff scores are used for decision-making purposes. By using Rasch item fit statistics and the Angoff method, practitioners and researchers can enhance the quality and validity of their assessments. [ABSTRACT FROM AUTHOR]

The generalized S − X 2 –test is a test of item fit for items with polytomous responses format. The test is based on a comparison of the observed and expected number of responses in strata defined by the test score. In this article, we make four contributions. We demonstrate that the performance of the generalized S − X 2 –test depends on how sparse cells are pooled. We propose alternative implementations of the test within the framework of limited information testing. We derive the distribution of the S − X 2 –residuals that can be used for post hoc analyses. We suggest a diagnostic plot that visualizes the form of the misfit. The performance of the alternative implementations is investigated in a simulation study. The simulation study suggests that the alternative implementations are capable of controlling the Type-I error rate well and have high power. An empirical application concludes this article. [ABSTRACT FROM AUTHOR]

In Programme for International Student Assessment (PISA), item response theory (IRT) scaling is used to examine the psychometric properties of items and scales and to provide comparable test scores across participating countries and over time. To balance the comparability of IRT item parameter estimations across countries with the best possible model fit, a partial invariance approach is used in PISA. In this approach, international or common item parameters are estimated for the majority of items, while unique or country‐specific item parameters are allowed for item‐country combinations where a misfit to the common parameters can be identified. The goal of the current study is to establish item fit statistic thresholds for identifying such misfits. We investigated the impact of various thresholds on scale and score estimation. To evaluate the impact of various item fit thresholds, we systematically examined the number of unique item parameters and country performance distributions and compared the overall model fit statistics using data from PISA 2015 and 2018. Results showed that RMSD =.10 provides the best fitting model while still establishing stable parameter estimations and sufficient comparability across groups. The applications and implications of the results are discussed. [ABSTRACT FROM AUTHOR]

In this article, statistical properties of the root mean square deviation (RMSD) item fit statistic in item response models are studied. It is shown that RMSD estimates will indicate even misfit for items whose parametric assumption of the item response function is correct (i.e., fitting items) if some item response functions in the test are misspecified. Moreover, it is demonstrated that the RMSD values of misfitting and fitting items depend on the proportion of misfitting items. We propose three alternative bias-corrected RMSD estimators that reduce the bias for fitting items. However, these alternative estimators provide slightly negatively biased estimates for misfitting items compared to the originally proposed RMSD statistic. In the numerical experiments, we study the case of a misspecified one-parameter logistic item response model and the behavior of the RMSD statistic if differential item functioning occurs. [ABSTRACT FROM AUTHOR]

Orlando and Thissen (2000) introduced the S−X2$S-X^{2}$ item‐fit index for testing goodness‐of‐fit with dichotomous item response theory (IRT) models. This study considers and evaluates an alternative approach for computing S−X2$S-X^{2}$ values and other factors associated with collapsing tables of observed and expected numbers (OE tables), which can affect flagging items. Results suggest that collapsing OE tables requires careful consideration of a trade‐off between power and empirical type I error rate. Concurrent collapsing of score categories would be preferred over separate collapsing for its procedural simplicity, minimal effect of choice of a minimum cell value on empirical type I error rates, and reasonable type I error rates even for the most sparse condition in the study. For separate collapsing, a smaller minimum cell value is recommended as OE tables possess more sparseness (e.g., longer test lengths and smaller sample sizes) if inflated type I error rates are more of a concern in detecting items for misfit based on the S−X2$S-X^{2}$ index. If it is more important to identify misfit items, the study results recommend using a larger minimum cell value for collapsing. [ABSTRACT FROM AUTHOR]

Objectives: The aim of this study was to analyze the Chinese version of the Nociception Coma Scale–Revised in patients with prolonged disorders of consciousness within the framework of Rasch modeling, including investigating the invariance of total scores across different etiologies of disorders of consciousness. Design: Prospective psychometric study. Participants: Patients with prolonged disorders of consciousness from the Rehabilitation and Neurology units in hospital. Interventions: None. Main Outcome Measure: The Nociception Coma Scale–Revised was undertaken by trained raters and the Coma Recovery Scale-Revised was used to assess patients' consciousness. The psychometric properties within the Rasch model including item-person targeting, reliability and separation, item fit, unidimensionality, and differential item functioning were assessed. Results: 84 patients with prolonged disorders of consciousness (mean age 53 years; mean injury 5 months; 42 with Minimally Conscious State and 42 with Unresponsive Wakefulness Syndrome) of 252 observations were enrolled in the study. Through the procedure of repeated assessment and differential item function, a lower item bias Rasch set was purified. The Rasch model assumptions were examined and met, with item reliability and validity meeting the recommended threshold. Conclusions: The Chinese version of the Nociception Coma Scale–Revised demonstrated unidimensionality, good reliability and separation, and good item fit, but dissatisfied person fit and item-person targeting. The verbal subscale showed a notable discrepancy between person responses and the difficulty of the items, suggesting limited clinical significance. [ABSTRACT FROM AUTHOR]

We used simulation techniques to assess the item-level and familywise Type I error control and power of an IRT item-fit statistic, the S-X2. Previous research indicated that the S-X2 has good Type I error control and decent power, but no previous research examined familywise Type I error control. We varied percentage of misfitting items, sample size, and test length, and computed familywise Type I error with no correction, a Bonferroni correction, and a Benjamini-Hochberg correction. The S-X2 controlled item-level and familywise Type I errors when corrections were applied to conditions with no misfitting items. In the presence of misfitting items, the S-X2 exhibited inflated item-level and familywise false hit rates in many conditions, even with familywise Type I error corrections. Lastly, power was low and negatively impacted when either of the familywise Type I error corrections was applied. We suggest using the S-X2 with no familywise Type I error control in conjunction with other methods of assessing item fit (e.g., visual analysis). [ABSTRACT FROM AUTHOR]

Testing whether items fit the assumptions of an item response theory model is an important step in evaluating a test. In the literature, numerous item fit statistics exist, many of which show severe limitations. The current study investigates the root mean squared deviation (RMSD) item fit statistic, which is used for evaluating item fit in various large-scale assessment studies. The three research questions of this study are (1) whether the empirical RMSD is an unbiased estimator of the population RMSD; (2) if this is not the case, whether this bias can be corrected; and (3) whether the test statistic provides an adequate significance test to detect misfitting items. Using simulation studies, it was found that the empirical RMSD is not an unbiased estimator of the population RMSD, and nonparametric bootstrapping falls short of entirely eliminating this bias. Using parametric bootstrapping, however, the RMSD can be used as a test statistic that outperforms the other approaches—infit and outfit, S − X 2—with respect to both Type I error rate and power. The empirical application showed that parametric bootstrapping of the RMSD results in rather conservative item fit decisions, which suggests more lenient cut-off criteria. [ABSTRACT FROM AUTHOR]

The effect of item parameters (discrimination, difficulty, and level of guessing) on the item-fit statistic was investigated using simulated dichotomous data. Nine tests were simulated using 1,O00 persons, 50 items, three levels of item discrimination, three levels of item difficulty, and three levels of guessing. The item fit was estimated using two fit statistics: the likelihood ratio statistic (χ²B), and the standardized residuals (SRs). All the item parameters were simulated to be normally distributed. Results showed that the levels of item discrimination and guessing affected the item-fit values. As the level of item discrimination or guessing increased, item-fit values increased and more items misfit the model The level of item difficulty did not affect the item-fit statistic. [ABSTRACT FROM AUTHOR]

Dodeen (2004) studied the correlation between the item parameters of the three-parameter logistic model and two item fit statistics, and found some linear relationships (e.g., a positive correlation between item discrimination parameters and item fit statistics) that have the potential for influencing the work of practitioners who employ item response theory. This article examines the same type of linear relationships as studied by Dodeen. However, this article adds to the literature by employing item fit statistics not considered by Dodeen, which have been recently suggested and whose Type I error rates have been demonstrated to be generally close to the nominal level. Detailed simulations show that if one uses certain of the recently suggested item fit statistics, there is no need to worry about any linear relationships between the item parameters and item fit statistics. [ABSTRACT FROM AUTHOR]

A Q-matrix, which reflects how attributes are measured for each item, is necessary when applying a cognitive diagnosis model to an assessment. In most cases, the Q-matrix is constructed by experts in the field and may be subjective and incorrect. One efficient method to refine the Q-matrix is to employ a suitable statistic that is calculated using response data. However, this approach is limited by its need to estimate all items in the Q-matrix even if only some are incorrect. To address this challenge, this study proposes an item fit statistic root mean square error approximation (RMSEA) for validating a Q-matrix with the deterministic inputs, noisy, "and" (DINA) model. Using a search algorithm, two simulation studies were performed to evaluate the effectiveness and efficiency of the proposed method at recovering Q-matrices. Results showed that using RMSEA can help define attributes in a Q-matrix. A comparison with the existing Delta method and residual sum of squares (RSS) method revealed that the proposed method had higher mean recovery rates and can be used to identify and correct Q-matrix misspecifications. When no error exists in the Q-matrix, the proposed method does not modify the correct Q-matrix. [ABSTRACT FROM AUTHOR]

Background: To provide both preventive and rehabilitative conditions in a workplace, one must understand how employees experience work demands. Such an understanding can be obtained from each individual with valid and quality-assured questionnaires. The Work Environment Impact Questionnaire (WEIQ) is a new questionnaire for measuring employees' self-perceived work ability in relation to their specific workplace environment. The purpose of this study was to assess the measurement properties in terms of construct validity of the WEIQ. Methods: A cross-sectional survey study was conducted with 288 respondents from three different workplaces involving assisted living personnel, vocational rehabilitation personnel and personnel at a research institute. The measurement properties of the WEIQ were assessed according to Rasch Measurement Theory (RMT), including assessment of item-to-sample targeting, threshold ordering, item fit statistics, unidimensionality and reliability. Results: Item fit, i.e., fit residuals, item characteristic curves (ICC) and chi square values, were all satisfactory, and no disordered thresholds were present after collapsing the lowest response categories. However, issues with local dependent (LD) item correlations was present in 7.6% cases, four items showed statistically significant differential item functioning (DIF), where 11% of the respondents had person fit residuals outside the recommended range of ± 2.5 and the t-test for unidimensionality did not meet the criterion of 5%. Scale-to-sample targeting and reliability (0.92) were good. LD could be resolved with testlets and at the same time maintaining fit and improving dimensionality, but then the reliability decreased to 0.82. Conclusions: This study provides an initial validation of the WEIQ to be used for assessing employees' self-perceived work ability. Most measurement properties were acceptable, but further exploration of LD, DIF and unidimensionality in additional work settings and with larger sample sizes is warranted. Trial registration: Not applicable. [ABSTRACT FROM AUTHOR]

Introduction: The National Eye Institute Visual Function Questionnaire (NEI VFQ) is a common patient-reported outcome measure (PROM) in uveitis trials. Its psychometric properties using state-of-the-art scoring based on Rasch models, a latent trait model that improves accuracy of PROMs assessment, has not yet been investigated. Methods: The study participants were recruited online from uveitis patient organizations, where individuals self-reported their uveitis diagnosis and visual acuity level. These participants then completed the NEI VFQ-25. The visual function (VF) and socioemotional (SE) subscales were psychometrically analysed in terms of item fit, targeting, internal consistency, dimensionality, and differential item functioning (DIF), using Rasch models. Criterion validity was examined based on associations between NEI VFQ person measures and recent visual acuity (VA) levels. Results: Ninety-nine participants recruited online from uveitis patient organizations (68 women, 31 men; mean age 50 ± 15 years; 46.5% self-reported receiving systematic therapy for uveitis, 0.6% NEI VFQ-25 missing data) were included. The mean difficulty of items was lower than the average person ability. None of the items demonstrated misfit to an extent that would induce noise into the measurement. The consistency metrics person reliability and person separation index of the subscales were 0.85 and 2.34 (NEI VFQ-VF), 0.86 and 2.52 (NEI VFQ-SE), respectively. There was no evidence of multidimensionality and none of the items showed DIF by gender. The differences between item and person measures were 1.44 (NEI VFQ-VF) and 1.03 (NEI VFQ-SE). NEI VFQ-25 person measures were significantly lower in participants with visual impairment (all p values ≤ 0.007). Conclusion: Rasch model-based scoring of the re-engineered NEI VFQ-25 demonstrates acceptable internal consistency, item fit and construct validity for assessing two key domains of quality of life in individuals self-reporting uveitis. The PROM was targeted at a higher level of difficulty than present in our heterogeneous sample. [ABSTRACT FROM AUTHOR]

Copyright of Psychological Science is the property of Psychological Science Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Questionnaires for the assessment of attitudes and other psychological traits are crucial in educational and psychological research, and item response theory (IRT) has become a viable tool for scaling such data. Many international large-scale assessments aim at comparing these constructs across countries, and the invariance of measures across countries is thus required. In its most recent cycle, the Programme for International Student Assessment (PISA 2015) implemented an innovative approach for testing the invariance of IRT-scaled constructs in the context questionnaires administered to students, parents, school principals, and teachers. On the basis of a concurrent calibration with equal item parameters across all groups (i.e., languages within countries), a group-specific item-fit statistic (root mean square deviance [RMSD]) was used as a measure for the invariance of item parameters for individual groups. The present simulation study examines the statistic's distribution under different types and extents of (non)invariance in polytomous items. Responses to five 4-point Likert-type items were generated under the generalized partial credit model (GPCM) for 1,000 simulees in 50 groups each. For one of the five items, either location or discrimination parameters were drawn from a normal distribution. In addition to the type of noninvariance, the extent of noninvariance was varied by manipulating the variation of these distributions. The results indicate that the RMSD statistic is better at detecting noninvariance related to between-group differences in item location than in item discrimination. The study's findings may be used as a starting point to sensitivity analysis aiming to define cutoff values for determining (non)invariance. [ABSTRACT FROM AUTHOR]

Research related to the fit evaluation at the item level involving cognitive diagnosis models (CDMs) has been scarce. According to the parsimony principle, balancing goodness of fit against model complexity is necessary. General CDMs require a larger sample size to be estimated reliably, and can lead to worse attribute classification accuracy than the appropriate reduced models when the sample size is small and the item quality is poor, which is typically the case in many empirical applications. The main purpose of this study was to systematically examine the statistical properties of four inferential item-fit statistics: S-[sup X2], the likelihood ratio (LR) test, the Wald (W) test, and the Lagrange multiplier (LM) test. To evaluate the performance of the statistics, a comprehensive set of factors, namely, sample size, correlational structure, test length, item quality, and generating model, is systematically manipulated using Monte Carlo methods. Results show that the S-[sup X2] statistic has unacceptable power. Type I error and power comparisons favor LR and W tests over the LM test. However, all the statistics are highly affected by the item quality. With a few exceptions, their performance is only acceptable when the item quality is high. In some cases, this effect can be ameliorated by an increase in sample size and test length. This implies that using the above statistics to assess item fit in practical settings when the item quality is low remains a challenge. [ABSTRACT FROM AUTHOR]

Mokken Scale Analysis (MSA) is a nonparametric approach that offers exploratory tools for understanding the nature of item responses while emphasizing invariance requirements. MSA is often discussed as it relates to Rasch measurement theory, which also emphasizes invariance, but uses parametric models. Researchers who have compared and combined MSA and Rasch approaches have generally concluded that these approaches offer complementary perspectives. However, researchers have not considered how the two approaches may provide insight related to examinee subgroups. Using a simulation and illustrative real data analysis, we propose an exploratory approach to integrating MSA and Rasch techniques to understand item fit as it relates to differences in item characteristics between examinee subgroups. Our results suggest that integrating complete-sample and subgroup-level MSA and Rasch model item fit analyses can provide insight into differences in item quality between subgroups. In some cases, misfit at the complete sample level corresponds to subgroup-related differences. On the other hand, fit at the complete sample level does not guarantee consistent measurement properties within subgroups. When subgroups are important for scale interpretation and use, our results suggest that conducting separate analyses within groups and considering their results with respect to those from the full sample may lend useful insight. [ABSTRACT FROM AUTHOR]

Background: Non-communicable diseases (NCDs) have a considerable impact on an individual's social life. Life-long medication, fear of death, dietary restrictions, hospital admissions, dependency on caregivers, and strained interpersonal relationships play a role. The available questionnaires measure only individual aspects of social impact. The aim of the study was to develop and validate a comprehensive questionnaire to assess the social impact of NCDs. Material and Methods: A cross-sectional study was conducted among patients and caregivers seeking care at a tertiary care hospital for diabetes, hypertension, chronic obstructive pulmonary disease (COPD), stroke, coronary artery disease (CAD), and cancer. Thirty patients each with the above diseases were administered Cousineau Self-Perceived Burden Scale to assess perceived burden among patients, Caregiver Burden Scale to assess caregiver burden, EuroQol-5D-3L to assess the quality of life, and face validated questions of sociodemographic details, cost of healthcare, and stress due to disease. Rasch analysis was performed to identify items that lie within the traditional bounds of mean-square item fit. The Delphi method with 10 subject experts was used to identify the questions that would be included in the final tool. Results: The data consists of 150 observations, with 77 categorical questions subdivided into various domains. From this, a total of 30 questions were extracted by Rasch analysis. The new questionnaire had good internal consistency (Cronbach's alpha: 0.969, 95% confidence interval: 0.96–0.98). The questionnaire was finalized after reaching a consensus among subject experts. Conclusion: The social impact questionnaire developed consisted of 30 questions, has good internal consistency, and can be used to measure the social impact of NCDs. [ABSTRACT FROM AUTHOR]

Copyright of Revista Iberoamericana de Diagncstico y Evaluacicn Psicolcgica is the property of Asociacion Iberoamericana de Diagnostico y Evaluacion Psicologica and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Background: This study aimed to develop an instrument for assessing physical functioning among adults aged 50 years or older living in the community. Methods: Based on a review of various national health surveys and cohort studies, a 144-item bank was constructed for assessing physical functioning. Focus group interviews were conducted among adults aged 50 years or older to investigate their level of understanding of 60 selected items, followed by a pretest of the items on a nationally representative sample (n=508). The final 25-item questionnaire was tested on an independent sample (n=259) for validity and reliability based on classical test and item response theories. Predictive validity at the 6-month follow-up was tested in a separate sample (n=263). Results: The newly developed Life Functioning (LF) scale assessed the dimensions of functional limitations, disabilities, and social activities. The scale satisfied a one-dimensionality assumption with good item fit and demonstrated criterion validity, construct validity, high internal consistency (Cronbach's alpha=0.93), and test-retest reliability (intra-class correlation coefficient=0.84; 95% confidence interval, 0.76–0.89). The LF scale comprised 25 items with a total score ranging from 0 to 100. Higher scores indicated higher levels of functioning. The LF score was significantly associated with the Physical Functioning score at 6 months. Conclusion: The LF scale was developed to assess the physical functioning of people in their late midlife or older. Future studies should test the instrument on a national sample and evaluate its application in diverse population subgroups. [ABSTRACT FROM AUTHOR]

Assessing item fit for unidimensional item response theory models for dichotomous items has always been an issue of enormous interest, but there exists no unanimously agreed item fit diagnostic for these models, and hence there is room for further investigation of the area. This paper employs the posterior predictive model-checking method, a popular Bayesian model-checking tool, to examine item fit for the above-mentioned models. An item fit plot, comparing the observed and predicted proportion-correct scores of examinees with different raw scores, is suggested. This paper also suggests how to obtain posterior predictive p-values (which are natural Bayesian p-values) for the item fit statistics of Orlando and Thissen that summarize numerically the information in the above-mentioned item fit plots. A number of simulation studies and a real data application demonstrate the effectiveness of the suggested item fit diagnostics. The suggested techniques seem to have adequate power and reasonable Type I error rate, and psychometricians will find them promising. [ABSTRACT FROM AUTHOR]

Parents and preschool teachers play a key role in shaping children's dietary behaviors. Knowledge of nutrition and healthy dietary choices is a key component to improve dietary habits and reduce the prevalence of obesity and associated co-morbidities. Using valid and reliable instruments is necessary for accurate assessment of knowledge to tailor interventions and measure effectiveness specific to the population of interest. The objectives of this paper are to (1) identify potential gaps in the baseline nutrition knowledge among parents and teachers using a previously validated questionnaire prior to a preschool obesity prevention intervention; and (2) assess the instrument's reliability and construct validity for a low socioeconomic status population using a post hoc Rasch analysis. Participants included 177 parents and 75 teachers who participated in a Head Start intervention study. Knowledge scores, instrument reliability, and item fit and difficulty were assessed using a Rasch analysis; t-tests were used to determine differences in scores between parents and teachers. Parents answered 38% of questions correctly while teachers correctly answered 46% of the questions. Adequate item fit and reliability were indicated for Sections 1 and 2 of the Nutrition Knowledge Questionnaire (NKQ). Section 3 demonstrated less adequate reliability. The items were found to adequately and reliably define the unidimensional measures of the three components of knowledge represented in this instrument, providing evidence of construct validity. However, Rasch measures indicated the NKQ overall was difficult for participants. Recommendations for improving the instrument for nutrition education/intervention and research practice areas related to obesity and obesity-related conditions are addressed. [ABSTRACT FROM AUTHOR]

Background: Abdominal pain is a common and often debilitating issue for children and adolescents. In many cases, it is not caused by a specific somatic condition but rather emerges from a complex interplay of bio-psycho-social factors, leading to functional abdominal pain (FAP). Given the complex nature of FAP, understanding its origins and how to effectively manage this condition is crucial. Until now, however, no questionnaire exists that targets knowledge in this specific domain. To address this, the Abdominal Pain Knowledge Questionnaire (A-PKQ) was developed. Methods: Two versions were created (one for children and one for parents) and tested in four gastroenterology clinics and one specialized pain clinic in Germany between November 2021 and February 2024. Children between 8 and 17 years of age (N = 128) and their accompanying parents (N = 131) participated in the study. Rasch analysis was used to test the performance of both versions of the questionnaire. Results: The original questionnaires exhibited good model and item fit. Subsequently, both questionnaires were refined to improve usability, resulting in final versions containing 10 items each. These final versions also demonstrated good model and item fit, with items assessing a variety of relevant domains. Conclusion: The A-PKQ is an important contribution to improving assessment in clinical trials focused on pediatric functional abdominal pain. [ABSTRACT FROM AUTHOR]

This study employed Rasch analyses to validate a novel measure of sport experience: the Sport Experience Measure: Children and Youth (SEM:CY). Analyses were applied to self-reported data of n = 503 young people (age 9–18 years, M = 12.91, 50% female) in Canada who were engaging in sport during the previous 12 months. The revised measure, consisting of 24 items on a 3-point response scale, demonstrated good fit statistics (e.g., item fit residual: M = −0.50, SD = 0.94 and person fit residual: M = −0.62, SD = 2.33), an ability to reliably discriminate between levels of sport experience, and an absence of differential item functioning for various groups (males and females, older and younger individuals, solo and team sports, and those playing at various competitive levels, including recreation). The SEM:CY is a succinct tool that can serve as a valuable means to gauge the quality of an individual's sport experience, which can facilitate positive youth development and sustain participation across the life span. [ABSTRACT FROM AUTHOR]

Residual analysis (e.g. Hambleton & Swaminathan, Item response theory: principles and applications, Kluwer Academic, Boston, ; Hambleton, Swaminathan, & Rogers, Fundamentals of item response theory, Sage, Newbury Park, ) is a popular method to assess fit of item response theory (IRT) models. We suggest a form of residual analysis that may be applied to assess item fit for unidimensional IRT models. The residual analysis consists of a comparison of the maximum-likelihood estimate of the item characteristic curve with an alternative ratio estimate of the item characteristic curve. The large sample distribution of the residual is proved to be standardized normal when the IRT model fits the data. We compare the performance of our suggested residual to the standardized residual of Hambleton et al. (Fundamentals of item response theory, Sage, Newbury Park, ) in a detailed simulation study. We then calculate our suggested residuals using data from an operational test. The residuals appear to be useful in assessing the item fit for unidimensional IRT models. [ABSTRACT FROM AUTHOR]

Collaborative problem-solving (CPS) skills are recognised as an essential aspect of 21st century skills and STEM education. In the STEM framework, students' CPS skills in the learning process need to be appropriately monitored, yet assessment tools for students' CPS skills are currently not widely developed in chemistry. This study aimed to develop and validate the assessment tool for measuring high school students' collaborative problem-solving skills in chemistry (CPS-C). Fifty-two students participated in the assessment, which included tasks on Coke Titration, Fruit Battery, and Soap Making. The data were analyzed using Multidimensional Item Response Theory (MIRT) models and the Generalized Partial Credit Model (GPCM). The results indicated that the CPS-C tool showed good internal consistency and item fit. Additionally, ten students were interviewed, and the interview content was analyzed using MAXQDA, emphasizing the importance of providing clear and concise instructions, reducing the number of tasks, and offering meaningful and relevant options. Moreover, this study also validated the reliability of cognitive ability tests, mental load and effort tests, and interest and motivation tests. The results showed satisfactory reliability for all tests except for the mental load and effort tests, which had a lower reliability coefficient of 0.68. Despite some limitations, the CPS-C tool showed great potential for effectively assessing CPS skills. Further research is needed to validate the tool's effectiveness and reliability across larger, more diverse samples and different cultural contexts. [ABSTRACT FROM AUTHOR]

Item Response Theory (IRT), over its nearly 100-year history, has become one of the most popular methodologies for modeling response patterns in measures in education, psychology and health. Due to its advantages, IRT is particularly popular in large-scale assessments. A pre-condition for the validity of the estimations obtained from IRT is that the data meet the model assumptions. The purpose of this study is to examine the testing of model assumptions in studies using IRT models. For this purpose, 107 studies in the National Thesis Center of the Council of Higher Education that use the IRT model on real data were examined. The studies were analyzed according to sample size, unidimensionality, local independence, overall model fit, item fit and non-speedness test criteria. According to the results, it was observed that the unidimensionality assumption was tested at a high level (89%) and Factor Analytic approaches were predominantly used. Local independence assumption was not tested in 36% of the studies, unidimensionality was cited as evidence in 40% of the studies and tested in 24% of the studies. Overall model fit was tested at a moderate level (51%) and Log-Likelihood and information criteria were used. Item fit and Non-Speedness testing were tested at a low level (26% and 9%). IRT assumptions should be considered as a whole and all assumptions should be tested from an evidence-based perspective. [ABSTRACT FROM AUTHOR]

Background: Patient‐reported outcome measures (PROMs) for hair loss focus mainly on Alopecia Areata. We created a PROM (i.e., HAIR‐Q) that is applicable to any hair loss condition. The HAIR‐Q measures satisfaction with hair. Patients/Methods: Concept elicitation interviews were conducted and analyzed to develop a draft scale. Content validity was established through multiple rounds of patient and expert input. Psychometric properties of the scale were examined in an online sample (i.e., Prolific) using Rasch measurement theory (RMT) analysis. Test–retest reliability and tests of construct validation were examined. Results: Content validity of a 22‐item draft scale was established with input from 11 patients, 12 experts and an online Prolific sample of 59 people who had a variety of hair loss treatments. In the RMT analysis (n = 390), 8 items were dropped. Data for the 14‐item scale fit the Rasch model (χ2 = 89.85, df = 70, p = 0.06). All 14 items had ordered thresholds and good item fit. Reliability was high with person separation index and Cronbach alpha values ≥0.91, and intraclass correlation coefficient of 0.94 based on a sample of 97 participants. Higher (better) scores on the scale were associated with having more hair, looking younger than ones' age, satisfaction with hair overall, being less bothered by hair loss, and for those who had a hair loss treatment in the past year, being more satisfied with their hair now than before treatment (p < 0.001). Conclusion: The HAIR‐Q evidenced reliability and validity and can be used in research and to inform clinical care to measure satisfaction with hair from the patient perspective. [ABSTRACT FROM AUTHOR]

Background: The FACE‐Q Aesthetics is used extensively to measure patient reported outcomes for minimally invasive and surgical facial aesthetic treatments. We recently developed a new FACE‐Q scale to assess satisfaction with the appearance of the temples. Aim: The aim of this study was to field test the FACE‐Q Satisfaction with Temples scale to examine its psychometric properties. Methods: The FACE‐Q Satisfaction with Temples scale was administered to 171 adults (22 years or older) seeking minimally invasive treatment to improve temple hollowing as part of a clinical trial. The severity of temple hollowing was established through the clinician‐reported Allergan Temple Hollowing scale (clinician‐rated). The psychometric properties of the FACE‐Q Satisfaction with Temples scale were established by testing Rasch Measurement Theory (RMT) assumptions and model fit; unidimensionality by principal component analysis; and construct validity by hypothesis testing. Results: The study sample consisted of 171 adults (mean age 54.7 ± 9.9, range, 25–82 years). RMT analysis provided evidence for the scientific soundness of a 12‐item Satisfaction with Temples scale. The data fit the Rasch model (χ2 = 20.47, df = 24, p = 0.67), all items had ordered thresholds, and good item fit. Scale reliability was high, with Person Separation Index and Cronbach alpha values with and without extremes ≥0.93. Principal component analysis revealed a single component with high factor coefficients. Construct validity was established as scores for the Satisfaction with Temples and Face Overall scales were correlated (r = 0.623, p < 0.001). Conclusion: The FACE‐Q Satisfaction with Temples scale is a reliable and valid measure that can be used in clinical practice and research to measure outcomes following treatment for temple hollowing. [ABSTRACT FROM AUTHOR]

Background: This study aimed to translate, culturally adapt, and psychometrically validate the PROMIS Anxiety Short Form 8a item bank into Arabic for the general population of Saudi Arabia. Methods: The PROMIS Anxiety Short Form was translated according to the FACIT group method. Second, psychometric validation was conducted on a convenience sample of 322 participants (mean age, 26 ± 10.4 years; predominantly female) from the general population in Saudi Arabia. Rasch analysis (Winsteps® version 5.6.1) was used to examine category functioning, item fit, the person separation index, item difficulty, unidimensionality, and local dependency. Results: Translation and cultural adaptation demonstrated that most of the items were culturally suitable and conveyed the same underlying concepts as the original scale. The five response categories of the scale satisfied the category functioning criteria, and all items fit the underlying construct, with the exception of one item that demonstrated a misfit. The item difficulty demonstrated poor targeting for the sample population; however, the person separation index and reliability were good (2.67 and 0.88, respectively) and no local dependency was noted. Conclusions: The Arabic translation of PROMIS-A SF8a demonstrated good structural validity and psychometrics, making it a valuable tool for screening anxiety in Arabic-speaking populations. The application of this outcome measure shows promise for healthcare professionals and patients alike, as it contributes to the provision of high-quality care and formulation of appropriate treatment plans. [ABSTRACT FROM AUTHOR]

Copyright of Journal of National Education / Millî Eğitim Dergisi is the property of Milli Egitim Bakanligi and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This paper presents Norwegian adaptations of the KoMus and KOPRA-M assessments designed to evaluate music-related competencies. Our research delves into the assessments’ alignment with the curriculum and investigates the psychometric properties of the Norwegian versions using a sample of Norwegian fifth graders (KoMus: N  =  374, KOPRA-M: N  =  370). Furthermore, it provides practical illustrations of how these item response theory (IRT)–based assessments can be employed, including competency-level examples. The Norwegian short versions of the KoMus and KOPRA-M tests demonstrated robust psychometric characteristics (especially in terms of reliability, model, and item fit), making them promising instruments for the deeper exploration of musical competency within primary and lower secondary school contexts. [ABSTRACT FROM AUTHOR]

A valid, reliable, and practical instrument is needed to evaluate ESL textbooks in Malaysian community colleges. Thus, a questionnaire guided by the Litz Theory (2005) was designed to evaluate ESL textbooks in Malaysian community colleges. This paper assessed the validation of the questionnaire using the Rasch Model analysis of construct validity and reliability. The questionnaire investigates community college students' perceptions of the ESL textbooks they use in their classrooms. The sample of this study comprised 123 community college students in Malaysia. The Rasch Model produced construct validity and reliability measurements using the dimensionality, item fit, and item polarity parameters and the person and item separation analysis. The results of the Rasch Model Analysis reveal that the construct validity in the study shows the uniformity of the instruments in the good category. This indicates that the items used in this study are related to the material's content. Hence, this questionnaire is valid and reliable to evaluate ESL textbooks in Malaysian community colleges. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Social Sciences Research / Sosyal Bilimler Arastirmalari Dergisi is the property of ODU Journal of Social Sciences Research and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The utility of Orlando and Thissen's (, ) S-X fit index was extended to the model-fit analysis of the graded response model (GRM). The performance of a modified S-X in assessing item-fit of the GRM was investigated in light of empirical Type I error rates and power with a simulation study having various conditions typically encountered in applied testing situations. The results show that the Type I error rates were controlled adequately around the nominal alpha by S-X. The power of the S-X statistic was much lower when the source of misfit was multidimensionality than when it was due to discrepancy from the true GRM curves. Once the data size increased sufficiently, however, appropriate power was obtained regardless of the source of the item-misfit. In summary, the generalized S-X appears to be a promising index for investigating item fit for polytomous items in educational and psychological assessments. [ABSTRACT FROM AUTHOR]

This research examines the utility of the s - χ² statistic proposed by Orlando and Thissen (2000) in evaluating item fit for multidimensional item response models. Monte Carlo simulation was conducted to investigate both the Type I error and statistical power of this fit statistic in analyzing two kinds of multidimensional test structures: approximate simple structure and complex structure. Overall, results show that this statistic is capable of evaluating item fit in the application of multidimensional item response models. It is important to identify the structure of multidimensional tests before this fit statistic is applied. For tests with an approximate simple structure, the sampling distribution can be approximated by a standard chi-square distribution. But for tests with a complex structure, that approximation is more complicated. As regards power in detecting the model nonfitting items, the performance of this statistic in multidimensional tests is comparable to that in unidimensional tests. [ABSTRACT FROM AUTHOR]

Empirical information regarding performance of model-fit procedures has been a persistent need in measurement practice. Statistical procedures for evaluating item fit were applied to real test examples that consist of both dichotomously and polytomously scored items. The item fit statistics used in this study included the PARSCALE's, Orlando and Thissen's (2000)and, and Stone's (2000)and. The results of this study indicated that the fit of an individual item was affected by the choice of model-fit analyses. The performance of fit indices appeared to vary depending on item response theory (IRT) model mixtures used for calibration, sample size, and test length. In terms of consistency among the fit indices, the statistics based on the same approach (e.g.,and) showed considerably higher agreement in detecting misfitting items than the statistics based on different approaches (e.g.,and).Consistent and inconsistent findings compared to previous research are discussed along with practical implications. [ABSTRACT FROM AUTHOR]

In this paper, the power rates and distributional properties of the Outfit, Infit, Lz, ECI2z and ECI4z statistics when they are used in tests with biased or differential item functioning (DIF) were explored. In this study, different conditions of sample size, sample size ratio focal and reference group, impact between groups, DIF effect size, and percentage of DIF items were manipulated. In addition, examinee responses were generated to simulate uniform DIF. Results suggest that item fit statistics generally detected medium percents of DIF in large samples (1000/500 or 1000/1000) only when DIF effect size was relatively high and when the mean of focal and reference group was different. Moreover, when groups had equal mean, low correct identification rates were found in the five item-fit indices. In general, the results showed adequate control of false positive rates. These findings lead to the conclusion that all indices used in this study are partially adequate fit measures for detecting biased items, mainly when impact between groups is present and sample size is large. [ABSTRACT FROM AUTHOR]

The current study examined the Type I error rates and power of several item response theory (IRT) item fit indices used in conjunction with the graded response model (GRM). Specifically, S — χ2, χ2*, and adjusted χ 2 degrees of freedom ratios (χ2/dfs) were examined. Model misfit was introduced by manipulating item parameters and by using a different IRT model to generate item data. Results indicated lower than expected Type I error rates for S — χ2 and χ 2*. Adjusted χ2/dfs resulted in large Type I error rates when used with cross validation and very low Type I error rates when used without cross validation. χ2* and adjusted χ 2/dfs without cross validation were the most powerful overall. [ABSTRACT FROM AUTHOR]

In this study we examined procedures for assessing model-data fit of item response theory (IRT) models for mixed format data. The model fit indices used in this study include PARSCALE's , Orlando and Thissen's and , and Stone's and . To investigate the relative performance of the fit statistics at the item level, we conducted two simulation studies: Type I error and power studies. We evaluated the performance of the item fit indices for various conditions of test length, sample size, and IRT models. Among the competing measures, the summed score-based indices and were found to be the sensible and efficient choice for assessing model fit for mixed format data. These indices performed well, particularly with short tests. The pseudo-observed score indices, and , showed inflated Type I error rates in some simulation conditions. Consistent with the findings of current literature, the PARSCALE's index was rarely useful, although it provided reasonable results for long tests. [ABSTRACT FROM AUTHOR]

Orlando and Thissen's S-X2 item fit index has performed better than traditional item fit statistics such as Yen's Q1 and McKinley and Mill's G2 for dichotomous item response theory (IRT) models. This study extends the utility of S-X2 to polytomous IRT models, including the generalized partial credit model, partial credit model, and rating scale model. The performance of the generalized S-X2 in assessing item model fit was studied in terms of empirical Type I error rates and power and compared to G2. The results suggest that the generalized S-X2 is promising for polytomous items in educational and psychological testing programs. [ABSTRACT FROM AUTHOR]

Orlando and Thissen (2000) developed an item fit statistic for binary item response theory (IRT) models known as S - X². This article generalizes their statistic to polytomous unfolding models. Four alternative formulations of S - X² are developed for the generalized graded unfolding model (GGUM). The GGUM is a unidimensional IRT model for unfolding polytomous responses. It yields single-peaked, nonmonotonic item characteristic curves that predict a higher item score to the extent that an individual is located close to an item on the underlying latent continuum. A simulation was performed to examine the characteristics of these new item fit indices under the GGUM, as well as a traditional likelihood ratio χ² test (G²). All variants of S - X² exhibited reasonable Type I error rates, but that for G² was more erratic. The new indices exhibited variable power to detect misfit. Two new item fit tests are recommended for use based on simulation results. [ABSTRACT FROM AUTHOR]