Computer-Based and Paper-and-Pencil Tests: A Study in Calculus for STEM Majors

Computer-based testing is an expanding use of technology offering advantages to teachers and students. We studied Calculus II classes for science, technology, engineering, and mathematics majors using different testing modes. Three sections with 324 students employed: paper-and-pencil testing, computer-based testing, and both. Computer tests gave immediate feedback and allowed multiple submissions and pooling. Paper-and-pencil tests (PPTs) required work and explanation allowing inspection of high cognitive demand tasks. Each test mode used the strength of its method. Students were given the same lecture by the same instructor on the same day and the same homework assignments and due dates. The design is quasi-experimental, but students were not aware of the testing mode at registration. Two basic questions examined were as follows: (a) Do paper-and-pencil and computer-based tests (CBTs) measure knowledge and skill in science, technology, engineering, and mathematics Calculus II in a consistent manner? (b) How does the knowledge and skill gained by students in a fully computer-based Calculus II class compare to students in a class requiring pencil-and-paper tests and hence some paper-and-pencil work. These results indicate that CBTs are as consistent with PPTs as CBTs are with themselves. Results are also consistent with classes using PPTs having slightly better outcomes than fully computer-based classes using only computer assessments.