Setting Standards on a Computerized-Adaptive Placement Examination. Laboratory or Psychometric and Evaluative Research Report No. 378.

Setting standards on educational tests is extremely challenging. The psychometric literature is replete with methods and guidelines for setting standards on educational tests; however, little attention has been paid to the process of setting standards on computerized adaptive tests (CATs). This lack of attention is unfortunate because CATs are becoming more widely used, and setting standards on these tests is typically more difficult than setting standards on nonadaptive (linear) tests. This paper discusses some of the issues to be addressed when setting standards on CATs, presents the results of a standard setting study conducted on a computerized adaptive placement test, and discusses the implications of the findings for future research and practice in this area. Thirteen mathematics experts participated in the standard-setting study using ACCUPLACER (College Board) scores. The results of the study suggest that standards can be set on CATs using subsets of items from a CAT item pool, and that methods designed to gather test-centered standard setting data more quickly than traditional methods show promise for setting standards on CATs. (Contains 2 figures, 5 tables, and 14 references.) (Author/SLD)