On Reporting IRT Ability Scores When the Test Is Not Unidimensional.

The purpose of this study was to examine the effect of test dimensionality on the stability of examinee ability estimates and item response theory (IRT) based score reports. A simulation procedure based on W. F. Stout's Essential Unidimensionality was used to generate test data with one dominant trait for the whole test and three minor traits specific to three subsets of items. The dimensionality of the data was controlled by varying the relative strengths of the specific traits. The errors in the ability estimation, which were examined both at test level and at subtest level, were compared among different degrees of test dimensionality. The correlation between the dominant trait and the minor traits was varied to three levels. When major and minor traits were not correlated, the standard errors in the ability estimates increased with increase in the strength of the minor traits. When the major and minor traits were correlated, on the other hand, the errors in the ability estimates slightly decreased as the strength of the minor traits was increased. (Contains 2 figures, 5 tables, and 12 references.) (Author/SLD)