Search

Higher education is faced with the question of how large numbers of students can be supported to learn complex skills without increasing teachers' supervision time proportionally and while preserving, or preferably improving, quality. Practicing skills only once does not work. Students need repetitive practice, feedback, and structured support to master a skill. They need to gain insight into what went well and what could be improved, so that they can further direct their attention while practicing. However, teachers cannot provide feedback after every practice session of students because their time is scarce. To solve this problem, an online formative assessment method for interactive and practice-oriented skills' training, Pe(e)rfectly Skilled, was developed. The Pe(e)rfectly Skilled method provides structured support for self-regulation, goal setting, feedback, and reflection. This method affords practicing skills repetitively, both individually and collaboratively, at students' own time, pace, and location. In this article, theoretical and practical underpinnings underlying the Pe(e)rfectly Skilled method are described.

Pitch-presentatie over Viewbrics. Viewbrics [ http://www.viewbrics.nl/ ] onderzoekt hoe videovoorbeelden in combinatie met rubrieken gebruikt kunnen worden om vaardigheden te beoordelen. Om complexe algemene vaardigheden onder de knie te krijgen is het belangrijk dat je een concreet en samenhangend beeld hebt van die vaardigheid. Viewbrics beschrijft de verschillende beheersingsniveaus van drie complexe vaardigheden: presenteren, samenwerken en informatiewijsheid. Viewbrics doet dat in tekst en verrijkt die tekstuele rubrics met videovoorbeelden. Zo kunnen leerlingen en leerkrachten hun prestaties vergelijken met de experts en de kunst ‘afkijken’ bij iemand die het al beheerst. Het project werkt samen met een kernteam met leerkrachten en leerlingen uit het voortgezet onderwijs, onderzoekers en experts.

To learn complex generic (21st century) skills students need to form a mental model of a skill. If students know their current and strived-for mastery level, they can better determine subsequent learning activities towards skills mastery. Rubrics support students in self-assessment of their skills as they provide textual descriptions of skills’ mastery levels with performance indicators for all constituent subskills. However, text-based rubrics have a limited capacity to support the formation of mental models of a complex skill. Video-enhanced rubrics (VER) with video modeling examples have the potential to improve and enrich mental model formation. VER can also be used to improve feedback quality a student receives from teachers, experts and peers while practicing a skill. Both affordances of VER can thus improve students’ performance.In the Viewbrics-project we therefore develop, through design-based research, a methodology for the formative assessment of complex skills with Video-Enhanced Rubrics (VER), precipitated in an online tool. We implemented this methodology in the online tool for three skills: collaborating, presenting and becoming information literate. This paper describes the features of the first prototype of the online tool and the results of a stakeholder evaluation of its perceived usefulness and usability, by means of a questionnaire and card-sorting exercise, with 7 teachers and 21 students of two secondary schools.The evaluation of this first prototype showed that both teachers and students evaluated the online tool and formative assessment methodology as handy, usable, helpful and feasible for learning complex skills, although some recommendations were made to further improve the design of the tool.

Learners aiming to master a complex skill may benefit from the combina-tion of abstract information found in a text-based analytical rubric and con-crete information provided by a video modeling example. In this paper, we address the design dilemmas of combining video modeling examples and rubrics into a Video Enhanced Rubric. We propose a model to address these design dilemma's and develop our first prototype based on this model. We review the first prototype through a two-stage international expert valida-tion session. In the first stage, 20 experts are asked to design a user inter-face for the Video Enhanced Rubric. In the second stage, 20 experts are asked to perform an expert appraisal of our first prototype. The preliminary results of the expert validation session are subsequently analyzed using Sauli, Cattaneo and van der Meij’s Framework for Developing Instructional Hypervideo to detect common design suggestions. Following the results of the expert validation, we developed a second prototype of the Video En-hanced Rubric. With the design guidelines of a Video Enhanced Rubric, we aim to improve the formative assessment and mastery of complex skills by fostering learner’s mental model development and the quality (consistency, concreteness) of both given as well as received feedback. On a more general note, we expect the design dilemmas addressed in this paper to inform re-searchers who aim to apply theoretical multimedia design guidelines to formative assessment practices with rubrics.

Rubrieken zijn in het onderwijs een veelgebruikt instrument om de beheersing van complexe vaardigheden zowel formatief als summatief te evalueren. Eén van de ontwerpaspecten van een rubriek is het definiëren en onderscheiden van de verschillende beheersingsniveau’s van een deelvaardigheid. Een ontwerpprobleem hierbij is het bepalen welke dimensies voor de kwaliteit van gedragscriteria bijdragen aan de complexiteit van een deel vaardigheid en deze te vertalen naar een consistente schaal met alle beheersingsniveau’s. Om voor docenten en onderwijsontwerpers zicht te krijgen op welke dimensies er op dit moment in de praktijk gebruikt worden voor de uitwerking van beheersingsniveau’s evalueerden we ruim 600 rubrieken voor het beoordelen van ‘samenwerken’. Uit 13 strikt geselecteerde rubrieken haalden we 6 categorieën met daarbinnen 13 dimensies voor het definiëren van schalen voor beheersingsniveau’s.

Rubrieken zijn in het onderwijs een veelgebruikt instrument om de beheersing van complexe vaardigheden zowel formatief als summatief te evalueren. Eén van de ontwerpaspecten van een rubriek is het definiëren en onderscheiden van de verschillende beheersingsniveau’s van een deelvaardigheid. Een ontwerpprobleem hierbij is het bepalen welke dimensies voor de kwaliteit van gedragscriteria bijdragen aan de complexiteit van een deel vaardigheid en deze te vertalen naar een consistente schaal met alle beheersingsniveau’s. Om voor docenten en onderwijsontwerpers zicht te krijgen op welke dimensies er op dit moment in de praktijk gebruikt worden voor de uitwerking van beheersingsniveau’s evalueerden we ruim 600 rubrieken voor het beoordelen van ‘samenwerken’. Uit 13 strikt geselecteerde rubrieken haalden we 6 categorieën met daarbinnen 13 dimensies voor het definiëren van schalen voor beheersingsniveau’s.

Many schools use analytic rubrics to (formatively) assess complex, generic or transversal (21st century) skills, such as collaborating and presenting. In rubrics, performance indicators on different levels of mastering a skill (e.g., novice, practiced, advanced, talented) are described. However, the dimensions used to describe the different mastery levels vary within and across rubrics and are in many cases not consistent, concise and often trivial, thereby hampering the quality of rubrics used to learn and assess complex skills. In this study we reviewed 600 rubrics available in three international databases (Rubistar, For All Rubrics, i-rubrics) and analyzed the dimensions found within 12 strictly selected rubrics that are currently used to distinguish mastery levels and describe performance indicators for the skill 'collaboration' at secondary schools. These dimensions were subsequently defined and categorized. This resulted in 13 different dimensions, clustered in 6 categories, feasible for defining skills’ mastery levels in rubrics. The identified dimensions can specifically support both teachers and researchers to construct, review and investigate performance indicators for each mastery level of a complex skill. On a more general level, they can support analysis of the overall quality of analytic rubrics to (formatively) assess complex skills.