Search

The performance of various classical test theory (CTT) item discrimination estimators has been compared in the literature using both empirical and simulated data, resulting in mixed results regarding the preference of some discrimination estimators over others. This study analyzes the performance of various item discrimination estimators in CTT: point-biserial correlation, point-biserial correlation with item excluded from the test total score, biserial correlation, phi coefficient splitting total score using the median, and discrimination index. For this study, data were generated from unidimensional logistic item response theory (IRT) models with one and two parameters. The factors considered in the study were test length, intervals for item difficulty and item discrimination parameters, as well as the composition of one or two groups of the examinees with specific ability distribution parameters. Results indicate that the biserial coefficient was most highly correlated with the IRT discrimination parameter across different simulation conditions. The degree of comparability among estimators and estimator invariance varied across conditions.

Self-regulation has become a widely discussed subject in education as it facilitates learners' ability to master their own learning. The purpose of the present study is to examine the psychometric properties of self-regulation in second language learning via Rasch measurement. A total of 528 high-school students from an East Asian country participated in the study, and the Self-Regulating Capacity in English Language Learning (SRClang) Scale was used as a self-report instrument. The results suggest that self-regulation in second language learning as measured by the SRClang Scale does not appear to be unidimensional. Based on item-model fit statistics, several items in the SRClang Scale may need to be revised or removed. Nonetheless, the response categories in the SRClang Scale function well as intended, and the scale can reliably separate items and persons based on their difficulty levels and ability trait levels, respectively. Overall, the SRClang Scale serves as an appropriate measure of self-regulatory capacity in second language learning.

Cognitive diagnosis models (CDMs) continue to generate interest among researchers and practitioners because they can provide diagnostic information relevant to classroom instruction and student learning. However, its modeling component has outpaced its complementary component-test construction. Thus, most applications of cognitive diagnosis modeling involve retrofitting of CDMs to assessments constructed using classical test theory (CTT) or item response theory (IRT). This study explores the relationship between item statistics used in the CTT, IRT, and CDM frameworks using such an assessment, specifically a large-scale mathematics assessment. Furthermore, by highlighting differences between tests with varying levels of diagnosticity using a measure of item discrimination from a CDM approach, this study empirically uncovers some important CTT and IRT item characteristics. These results can be used to formulate practical guidelines in using IRT- or CTT-constructed assessments for cognitive diagnosis purposes.

The purpose of this study was to investigate psychometric properties of the Children's Depression Inventory within a nonclinical and longitudinal sample (8th and 12th grades). Using the Rasch rating scale, most items represented one dimension. There was adequate separation among items and no overlap between ranges of item difficulties with latent trait parameters. (Contains 4 tables and 3 figures.)

The present study examined the technical adequacy of curriculum-based measures (CBMs) of early numeracy. Six 1-min early mathematics tasks were administered to 137 kindergarten and first-grade students, along with an omnibus test of early mathematics. The CBM measures included Count Out Loud, Quantity Discrimination, Number Identification, Missing Number, Next Number, and Number Facts. Alpha and split-half reliabilities were assessed, and construct validity and criterion validity (both concurrent and predictive) were examined. In addition, an item analysis was conducted for each measure using classical test theory and item response theory approaches, specifically using a Rasch model. Results indicated that the measures produced moderately strong reliability coefficients, satisfactory item difficulty and discrimination, evidence to support both concurrent and predictive validity with the standardized early mathematics test, and the tasks appeared to be measuring the construct of early numeracy proficiency. (Contains 6 tables.)

In this study, psychometric properties of a Korean-adapted version of the Experiences in Close Relationships--Revised scale were investigated using confirmatory factor analysis (CFA) in conjunction with item response theory. In CFA, neither the supposed two-factor model of Anxiety and Avoidance nor two single-factor models for each factor showed a reasonable fit. In addition, the results from the CFA suggested that the negative-worded and positive-worded items form distinct factors. In the item response theory analysis, the item response matrices of the Korean version did not meet unidimensionality and local independence assumptions. The hierarchical nature of the multidimensionality in the data and residual association among items were dealt with by using a bifactor model. Subsequent exploratory bifactor analysis and full-information item bifactor analysis of graded response data provided support for the bifactor structure of the data and other unique features of the Korean version. The differences between the Experiences in Close Relationships--Revised and the Korean version and their probable causes and implications are discussed. (Contains 6 tables and 2 figures.)

Studies of international mathematics achievement such as the Trends in Mathematics and Science Study (TIMSS) have employed classical test theory and item response theory to rank individuals within a latent ability continuum. Although these approaches have provided insights into comparisons between countries, they have yet to examine how specific attribute mastery affects student performance and how they can provide information for curricular instruction. In the 2007 administration of TIMSS, two benchmark participants--Massachusetts and Minnesota--were tested following the same procedural methods, providing an opportunity for comparison within and across the United States. Overall comparison of their performance showed Massachusetts and Minnesota to significantly outperform the United States. However, this article shows that there is a greater wealth of fine-grained information that can be translated directly for classroom application at the attribute level when a cognitive diagnostic model (CDM) such as the deterministic, inputs, noisy, "and" gate (Junker & Sijtsma, 2001) model is used. Results showed a significant disparity between proportions of correctly answering and mastering skills required to solve an item. Advantages of CDMs are discussed as well as a CDM-based method to filter distractor response categories that can aid instructors to diagnose a student's attribute mastery. (Contains 10 tables and 8 figures.)

Cognitive diagnosis models (CDMs), as alternative approaches to unidimensional item response models, have received increasing attention in recent years. CDMs are developed for the purpose of identifying the mastery or nonmastery of multiple fine-grained attributes or skills required for solving problems in a domain. For CDMs to receive wider use, researchers and practitioners need to understand the basic properties of these models. The article focuses on one CDM, the deterministic inputs, noisy "and" gate (DINA) model, and the invariance property of its parameters. Using simulated data involving different attribute distributions, the article demonstrates that the DINA model parameters are absolutely invariant when the model perfectly fits the data. An additional example involving different ability groups illustrates how noise in real data can contribute to the lack of invariance in these parameters. Some practical implications of these findings are discussed.

Self-deception has become a construct of great interest in individual differences research because it has been associated with levels of resilience and mental health. The Balanced Inventory of Desirable Responding (BIDR) is a self-report measure used for quantifying self-deception. In this study we used Rasch modeling to examine the properties of the self-deception scale of the BIDR in terms of dimensionality, use of response category, sample appropriateness, and reliability. A total of 315 university students (ages 18-21) were administered the self-deception scale of the BIDR. Seven-category and 2-category scoring methods were compared, as approved by the developers of the scale. Overall, the 7-category model was the best fit for the data and the sample. We concluded that the scale has the best reliability using a 7-category model with Item 13 deleted. Because of low person measure separation and reliability, the appropriateness of use of this instrument in undergraduate populations is questioned; the use of the measure in populations with larger ranges of self-deception is not recommended. (Contains 2 figures and 4 tables.)

The purpose of this study was to assess the model fit of a 2PL through comparison with the nonparametric item characteristic curve (ICC) estimation procedures. Results indicate that three nonparametric procedures implemented produced ICCs that are similar to that of the 2PL for items simulated to fit the 2PL. However for misfitting items, especially nonmonotone items, the greatest difference is between the 2PL and kernel smoothing procedures. In general, the differences between ICCs from the nonparametric procedures and the 2PL are reduced as both sample size and test length increase. The false positive rate of the test for model fit is promising for nonparametric ICC estimation methods. Power to detect misfitting items simulated with 4PL is low. Power to detect nonmonotone items is generally much higher. Power is best for kernel smoothing but also good for isotonic regression in the medium to large sample sizes and longer test length conditions. Power for the smoothed isotonic regression is uniformly low. (Contains 4 tables.)

Clinical and health policy research frequently involves health status measurement using generic or disease specific instruments. These instruments are generally developed to arrive at several scales, each measuring a distinct domain of health quality of life (HQOL). Clinical settings are starting to explore how to integrate patient perspectives of HQOL outcomes into patient care. However, the length of many HQOL instruments poses a challenge in terms of patient burden, as well as clinic flow time. The most popular paradigm for scale construction utilizes classical test theory methodology and can lead to excessive and redundant items in an effort to bolster reliability measurements such as Cronbach's alpha above levels of accepted reliability. This paper presents techniques for utilizing item response theory to arrive at single item scales that are diagnostically informative and short enough to have clinical utility. A danger of such dramatic scale reduction is that validity might be compromised. This danger is addressed in terms of criterion related validity and sensitivity to clinical changes over a 36 months period. The reduction methods are illustrated using selected scales from the Arthritis Impact Measurement Scales 2 (AIMS2) with data obtained from the study "Pharmaceutical Care Outcomes: The Patient Role (PCOPR)."

This study compares the performance of three nonparametric item characteristic curve (ICC) estimation procedures: isotonic regression, smoothed isotonic regression, and kernel smoothing. Smoothed isotonic regression, employed along with an appropriate kernel function, provides better estimates and also satisfies the assumption of strict monotonicity. As the number of items and the sample size increase, the kernel smoothing and smoothed isotonic regression ICC estimation procedures yield similar results across all conditions. (Contains 2 tables and 2 figures.)

Large-scale educational testing data often contain vast amounts of variables associated with information pertaining to test takers, schools, or access to educational resources—information that can help explain relationships between test taker performance and their learning environment. This study examines approaches to incorporate latent and observed explanatory variables as predictors for cognitive diagnostic models (CDMs). Methods to specify and simultaneously estimate observed and latent variables (estimated using item response theory) as predictors affecting attribute mastery were examined. Real-world data analyses were conducted to demonstrate the application using large-scale international testing data. Simulation studies were conducted to examine the recovery and classification for simultaneously estimating multiple latent (using dichotomous and polytomous items as indicators for the latent construct) and observed predictors for varying sample sizes and number of attributes. Results showed stable parameter recovery and consistency in attribute classification. Implications for latent predictors and attribute specifications are discussed. [ABSTRACT FROM AUTHOR]

When students solve problems, their proficiency in a particular subject may influence how well they perform in a similar, but different area of study. For example, studies have shown that science ability may have an effect on the mastery of mathematics skills, which in turn may affect how examinees respond to mathematics items. From this view, it becomes natural to examine the relationship of performance on a particular area of study to the mastery of attributes on a related subject. To examine such an influence, this study proposes a covariate extension to the deterministic input noisy “and” gate (DINA) model by applying a latent class regression framework. The DINA model has been selected for the study as it is known for its parsimony, easy interpretation, and potential extension of the covariate framework to more complex cognitive diagnostic models. In this approach, covariates can be specified to affect items or attributes. Real-world data analysis using the fourth-grade Trends in International Mathematics and Science Study (TIMSS) data showed significant relationships between science ability and attributes in mathematics. Simulation study results showed stable recovery of parameters and latent classes for varying sample sizes. These findings suggest further applications of covariates in a cognitive diagnostic modeling framework that can aid the understanding of how various factors influence mastery of fine-grained attributes. [ABSTRACT FROM AUTHOR]