Setting the Response Time Threshold Parameter to Differentiate Solution Behavior from Rapid-Guessing Behavior

In this study four methods were compared for setting a response time threshold that differentiates rapid-guessing behavior from solution behavior when examinees are obliged to complete a low-stakes test. The four methods examined were: (1) a fixed threshold for all test items; (2) thresholds based on item surface features such as the amount of reading required; (3) thresholds based on visually inspecting response time distributions; (4) thresholds statistically generated based on a two-state mixture model (Schnipke & Scrams, 1997; 2002). To compare the sets of threshold parameters, we used the method designed by Wise and Kong (2005) to assess the reliability and validity of response time effort scores, which were generated on the basis of the specified threshold values. The results showed only minor differences among the four threshold identification methods. Recommendations were given regarding the uses of the methods. (Contains tables and 2 figures.)