Search

Multiple-choice (MC) items are widely used in educational tests. Distractor analysis, an important procedure for checking the utility of response options within an MC item, can be readily implemented in the framework of item response theory (IRT). Although random guessing is a popular behavior of test-takers when answering MC items, none of the existing IRT models for distractor analysis have considered the influence of random guessing in this process. In this article, we propose a new IRT model to distinguish the influence of random guessing from response option functioning. A brief simulation study was conducted to examine the parameter recovery of the proposed model. To demonstrate its effectiveness, the new model was applied to the mathematics tests of the Hong Kong Diploma of Secondary Education Examination (HKDSE) from 2015 to 2019. The results of empirical analyses suggest that the complexity of item contents is a key factor in inducing students' random guessing. The implications and applications of the new model to other testing situations are also discussed.

Educational tests often have a cluster of items linked by a common stimulus (testlet). In such a design, the dependencies caused between items are called testlet effects. In particular, the directional testlet effect (DTE) refers to a recursive influence whereby responses to earlier items can positively or negatively affect the scores on later items. This study aims to introduce an innovative measurement model to describe DTEs among multiple polytomouslyscored open‐ended items. Through simulations, we found that (1) item and DTE parameters can be accurately recovered in Latent GOLD®, (2) ignoring positive (or negative) DTEs by fitting a standard item response theory model can result in the overestimation (or underestimation) of test reliability, (3) collapsing multiple items of a testlet into a super item is still effective in eliminating DTEs, (4) the popular multidimensional strategy of adding nuisance factors to describe item dependencies fails to account for DTE adequately, and (5) fitting the proposed model for DTE to testlet data involving nuisance factors will observe positive DTEs but will not have a better fit. Moreover, using the proposed model, we demonstrated the coexistence of positive and negative DTEs in a real history exam. [ABSTRACT FROM AUTHOR]