Search

Analyzing students' growth remains an important topic in educational research. Most recently, Diagnostic Classification Models (DCMs) have been used to track skill acquisition in a longitudinal fashion, with the purpose to provide an estimate of students' learning trajectories in terms of the change of fine-grained skills overtime. Response time (RT), the amount of time the test taker spends considering and answering each item, has been extensively studied and used in testing environments as a useful source of information to reflect individual response behavior and item characteristics. In this study, we consider using RTs in a learning environment to model students' learning progress. This could provide additional diagnostic information on students' fluency of applying the mastered skills. We propose a framework to model changes in RTs with a higher-order hidden Markov DCM. The proposed models are evaluated through a computer-based learning system that is designed to improve students' spatial skills. Results indicate that the proposed model can demonstrate both within and between group differences in learning through the predicted growth of latent speed on different items.

Cognitive diagnostic models (CDMs) model the probability of correctly answering an item as a function of an examinee's attribute mastery pattern. Because estimation of the mastery pattern involves more than a continuous measure of ability, reliability concepts introduced by classical test theory and item response theory do not apply. The cognitive diagnostic index (CDI) measures an item's overall discrimination power, which indicates an item's usefulness in examinee attribute pattern estimation. Because of its relationship with correct classification rates, the CDI was shown to be instrumental in cognitively diagnostic test assembly. This article generalizes the CDI to attribute-level discrimination indices for an item. Two different attribute-level discrimination indices are defined; their relationship with correct classification rates is explored using Monte Carlo simulations. There are strong relationships between the defined attribute indices and correct classification rates. Thus, one important potential application of these indices is test assembly from a CDM-calibrated item bank. (Contains 5 tables, 2 figures, and 3 notes.)

Nonparametric item response models have been developed as alternatives to the relatively inflexible parametric item response models. An open question is whether it is possible and practical to administer computerized adaptive testing with nonparametric models. This paper explores the possibility of computerized adaptive testing when using nonparametric item response models. A central issue is that the derivatives of item characteristic curves may not be estimated well, which eliminates the availability of the standard maximum Fisher information criterion. As alternatives, procedures based on Shannon entropy and Kullback-Leibler information are proposed. For a long test, these procedures, which do not require the derivatives of the item characteristic curves, become equivalent to the maximum Fisher information criterion. A simulation study is conducted to study the behavior of these two procedures, compared with random item selection. The study shows that the procedures based on Shannon entropy and Kullback-Leibler information perform similarly in terms of root mean square error, and perform much better than random item selection. The study also shows that item exposure rates need to be addressed for these methods to be practical.

Although cognitive diagnostic models (CDMs) can be useful in the analysis and interpretation of existing tests, little has been developed to specify how one might construct a good test using aspects of the CDMs. This article discusses the derivation of a general CDM index based on Kullback-Leibler information that will serve as a measure of how informative an item is for the classification of examinees. The effectiveness of the index is examined for items calibrated using the deterministic input noisy "and" gate model (DINA) and the reparameterized unified model (RUM) by implementing a simple heuristic to construct a test from an item bank. When compared to randomly constructed tests from the same item bank, the heuristic shows significant improvement in classification rates.

Objective. Patients who have cystic fibrosis (CF) and experience delayed diagnosis by traditional methods have greater nutritional insult compared with peers diagnosed via neonatal screening. The objective of this study was to evaluate cognitive function in children with CF and the influence of both early diagnosis through neonatal screening and the potential effect of early malnutrition. Methods. Cognitive assessment data were obtained for 89 CF patients (aged 7.3-17 years) during routine clinic visits. Patients had been enrolled in either the screened (N = 42) or traditional diagnosis (control) group (N = 47) of the Wisconsin CF Neonatal Screening Project. The Test of Cognitive Skills, Second Edition was administered to generate the Cognitive Skills Index (CSI) and cognitive factor scores (Verbal, Nonverbal, and Memory). Results. Cognitive scores in the overall study population were similar to normative data (CSI mean [standard deviation]: 102.5 [16.6]; 95% confidence interval: 99.1-105.9). The mean (standard deviation) CSI scores for the screened and control groups were 104.4 (14.4) and 99.8 (18.5), respectively. Significantly lower cognitive scores correlated with indicators of malnutrition and unfavorable family factors such as single parents, lower socioeconomic status, and less parental education. Our analyses revealed lower cognitive scores in patients with low plasma [alpha]-tocopherol ([alpha]-T) levels at diagnosis. In addition, patients in the control group who also had vitamin E deficiency at diagnosis ([alpha]-T < 300 [micro]g/dl) showed significantly lower CSI scores in comparison with [alpha]-T sufficient control subjects and both deficient and sufficient [alpha]-T subsets of screened patients. Conclusion. Results suggest that prevention of prolonged malnutrition by early diagnosis and nutritional therapy, particularly minimizing the duration of vitamin E deficiency, is associated with better cognitive functioning in children with CF. Pediatrics 2004;113:1549-1558; cognitive function; cystic fibrosis; malnutrition; neonatal screening; vitamin E deficiency., ABBREVIATIONS. CF, cystic fibrosis; SES, socioeconomic status; [alpha]-T, [alpha]-tocopherol; TCS/2, Test of Cognitive Skills, Second Edition; CSI, Cognitive Skills Index; MI, meconium ileus; PS, pancreatic sufficient; PI, pancreatic insufficient. It [...]