Providing Fine-Grained Feedback within an On-Line Learning System--Identifying the Workers from the Lurkers and the Shirkers

This paper describes a mechanism developed by the authors to gather student feedback from formative revision Multiple Choice Questionnaires (MCQs) within an on-line learning system. The MCQs provided first year Computer Science students with instant formative feedback, while data was also gathered about student responses, such as the percentage opting for each answer and the time taken to answer the question. We measured how students were using our on-line learning system; whether they were in fact "workers" who provided answers to the MCQs, were "lurkers" who did not provide answers but asked for solutions or "shirkers", who did not access the site at all! The data indicates that the time taken to answer a harder question was less than that of an easier question suggesting that the workers turned into lurkers strategically when they thought they could not answer successfully. It was not however clear whether the lurker suddenly finding an easier question would change back into a worker. Future work to encourage the shirkers to participate is also discussed.