認知診断モデルに基づく自動項目生成システムの妥当性検証 --一次方程式の認知診断テストにおける事例研究--

Cognitive diagnostic assessments (CDAs) require a large number of items to measure the target attributes with high precision. An automatic item generation (AIG) system would help to reduce the cost and effort of item writing in CDAs. This study aimed to develop a valid AIG system for CDAs in linear equations of mathematics by designing an AIG system and examining two aspects of a generated CDA in cognitive diagnostic modeling: (a) the M-matrix, which specifies the set of attributes required by each item model and (b) the item discrimination index, which is computed from item parameters in the deterministic input, noisy-and-gate (DINA) model. First, we compared an original M-matrix to two alternative M-matrices by using information criteria, posterior predictive model checks, and item discrimination indices. Second, we examined the magnitude and variability of the item discrimination indices for every item model. No substantially large differences were found among the results from all of the M-matrices. The discrimination indices tended to be high in items that measured major attributes, and the variabilities of the indices were small within each item model, except for a few item models. These findings indicate the validity of our M-matrix and AIG system. Furthermore, they suggest ways to improve the AIG system. Research limitations and how future studies can help to enhance the AIG system are discussed. [ABSTRACT FROM AUTHOR]