1. Obtaining Rubric Weights for Assessments by More than One Lecturer Using a Pairwise Learning Model
- Author
-
International Working Group on Educational Data Mining, Quevedo, J. R., and Montanes, E.
- Abstract
Specifying the criteria of a rubric to assess an activity, establishing the different quality levels of proficiency of development and defining weights for every criterion is not as easy as one a priori might think. Besides, the complexity of these tasks increases when they involve more than one lecturer. Reaching an agreement about the criteria and the levels of proficiency might be easier taking into account the abilities students must achieve according to the purpose of the subject. However, the disagreement about the weights of every criterion in an assessment rubric might easily appear. This paper focuses on the automatic weight adjustment for the criteria of a rubric. This fitting can be considered as a global perception that the whole group of lecturers have about the accuracy of solving an activity. Firstly, each lecturer makes a proposal of weights and then, from a set of pairs of students he/she globally expresses who of each pair has solved better the activity for which the rubric was designed. Secondly, an approach based on the pairwise learning is proposed in this work to obtain adequate weights for the criteria of a rubric. The system commits fewer errors than the lecturers and make them improve and reconsider some aspects of the rubric. (Contains 3 figures and 5 tables.) [Funding was provided by the Ministerio de Ciencia e Innovacion (MICINN). For the complete proceedings, "Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009)," see ED539041.]
- Published
- 2009