Paper Or IDE?

Exams are an integral assessment component in CS1 courses. However, little research exists concerning the impact of the exam format on student performance. To address this deficit, we conducted a year-long study to investigate student performance across two test formats, paper and IDE, with the goal of identifying any differences attributable to the format. Data was collected from the second and third exams over the course of two semesters, with five sections per semester. Each exam, whether paper or IDE, was constructed from a common list of concepts to ensure comparability across formats and sections. To assess the impact of exam format, we considered three points of concern. The first factor, grade received, provides a direct indication of student performance. Next, we examined the type of error committed by students (either syntax or logic) to determine influence of exam format (e.g., Does an IDE reduce the occurrence of syntax errors?). Finally, we considered exam completion rates to address concerns such as if the additional feedback might be overwhelming students in an already stressful environment (exam-taking) leading to lower completion rates.