Numerical Answer Options: Logical or Random Order?

Writers of mathematics test items, especially those who write for standardized tests, are often advised to arrange the answer options in logical order, usually ascending or descending numerical order. In this study, 32 mathematics items were selected for inclusion in four experimental pretest units, each consisting of 16 items. Two versions presented options in logically descending or ascending order, and two presented them randomly. Each test was administered as part of the American College Test (ACT) pretest procedures to approximately 300 examinees. A review of the speededness rate indicates that 99 percent of the examinees completed the test, leading to the conclusion that not only the difficulty, but the speededness was unaffected by the random ordering of the numerical options. Regarding the higher biserials in performance on the randomly ordered options, the higher-ability examinees (as determined by their performance on the ACT Assessment Mathematics Test) scored proportionately better than did the lower-ability group. It is evident that, while random ordering of numerical options is not an obstacle for upper-ability examinees, it may be one for lower-ability examinees. Given the difference in biserials, test writers might do well to avoid introducing confusion or an irrelevant source of error by continuing to arrange options in some logical sequence. Two tables present study findings. (SLD)