Variation in Chemistry Instructors' Evaluations of Student Written Responses and Its Impact on Grading

Evaluation of student written work during summative assessments is an important task for instructors at all educational levels. Nevertheless, few research studies exist that provide insights into how different instructors approach this task. In this study, we characterised variation in chemistry instructors' approaches to the evaluation and grading of different student responses to conceptual questions in general chemistry summative assessments, and analysed the correlation of such variations with assigned grades. Our results revealed differences in the approaches followed by instructors along various dimensions, such as their focus of attention (e.g., knowledge vs. reasoning), the construction and use of a rubric to evaluate student work, and the extent to which instructors engaged in building inferences about student understanding. Most instructors, however, were consistent in the approaches they individually followed when analyzing different student responses. Differences in approaches along some dimensions correlated to assigned grades, but relevant dimensions differed for novice and experienced instructors (e.g., adopting an inferential "versus" a literal stance had a significant correlation with the grades assigned by experienced instructors but not by novice ones). The results of our study provide insights into strategies for the professional development of college instructors such as asking instructors to critically reflect on how the decisions they make along multiple dimensions during the grading process can affect the outcome.