Detecting Items That Have Been Memorized in the Computerized Adaptive Testing Environment.

Because scores on high-stakes tests influence many decisions, tests need to be secure. Decisions based on scores affected by preknowledge of items are unacceptable. New methods are needed to detect the new cheating strategies used for computer-administered tests because item pools are typically used over time, providing the potential opportunity for test takers to share items with future test takers. Because of the serious ramifications of accusing someone as being a user of item preknowledge (or "cheater"), it may be more useful for operational computer-administered test developers to focus on item security rather than the behavior of individual test takers. This research explores the development and use of a fit index to detect items that have been memorized so that these items may be removed from the item pool, while leaving secure items in the pool. The results from this initial simulation for the developed Bayesian posterior log odds ratio index are promising. It is hoped that this work and future work will enable testing programs to determine more effectively how long to leave an item pool (or specific items) in the field. (Contains four tables, five figures, and five references.) (Author/SLD)