Mode Effects in College Admissions Testing and Differential Speededness as a Possible Explanation

As testing programs transition from paper to online testing, they must study mode comparability to support the exchangeability of scores from different testing modes. To that end, a series of three mode comparability studies was conducted during the 2019-2020 academic year with examinees randomly assigned to take the ACT college admissions exam on paper or online. Unlike many prior mode comparability studies, examinees testing online earned higher scores on average than examinees testing on paper, especially on the English, reading, and writing tests. Much of the observed mode effects were accounted for by items near the ends of the tests. Subsequent examination of item omit rates, differential item functioning by mode, and change-point analyses supported the notion that the observed mode effects were related to differential speededness. Specifically, examinees testing online were slightly less speeded, and were therefore able to perform better on items near the ends of the tests. Practical implications for linking across modes, test design, and test administration conditions are discussed.