Search

Technological developments are impacting upon UK assessment practices in many ways. For qualification awarding bodies, a key example of such impact is the ongoing shift towards examiners marking digitally scanned copies of examination scripts on screen rather than the original paper documents. This digitisation process has obvious benefits, affording flexible information management and distribution. Digitisation can also lead readers to interact differently with texts, with readers' physical engagement with digital texts differing from traditional paper-based texts. More specifically, it appears that reader navigation and annotation behaviours are particularly influenced by a digital mode shift. A consequence of this is that it is possible that the mental workload involved in reading on screen is greater than when reading on paper. Moreover, it is theorised that this additional load can have an adverse affect on reader comprehension building. These findings prompt questions about whether the mode of marking might unduly influence marking outcomes when assessors read longer essay responses in a digital environment. In light of this concern, this article explores how marking mode influences examiner essay marking behaviours, which in turn might influence their comprehension building and their marking outcomes. This study gathers data from 12 experienced examiners working within a large UK-based awarding body to investigate how mode influences their navigation and annotation behaviours when marking extended essays on paper and on screen. These comparative data are then used to explore a model which suggests that manual marking behaviours, such as navigation and annotation, can influence examiners' comprehension whilst marking. The implications for future developments in digital essay-marking software are then discussed. (Contains 4 figures and 5 tables.)

In the UK and elsewhere, large-scale educational assessment agencies are shifting the mode of school examination marking towards having examiners mark examination scripts on screen rather than on paper. This shift has prompted questions about whether the mode of marking might influence examiner marking accuracy, particularly in relation to extended essay responses. The study reported in this article investigated this issue empirically as part of a wider research project which looked broadly at the influence of marking mode on 12 examiners' marking outcomes and processes when assessing samples of extended essays. No association was found between marking mode and the magnitude of marking accuracy, and a practically and statistically negligible association was found between marking mode and the direction of marking accuracy. Given the mode-related influences on examiners' navigating and annotating behaviours that were also evident in the data, the article discusses the potential relationships between the findings. (Contains 9 tables, 2 figures and 3 notes.)