Search

Higher education is faced with the question of how large numbers of students can be supported to learn complex skills without increasing teachers' supervision time proportionally and while preserving, or preferably improving, quality. Practicing skills only once does not work. Students need repetitive practice, feedback, and structured support to master a skill. They need to gain insight into what went well and what could be improved, so that they can further direct their attention while practicing. However, teachers cannot provide feedback after every practice session of students because their time is scarce. To solve this problem, an online formative assessment method for interactive and practice-oriented skills' training, Pe(e)rfectly Skilled, was developed. The Pe(e)rfectly Skilled method provides structured support for self-regulation, goal setting, feedback, and reflection. This method affords practicing skills repetitively, both individually and collaboratively, at students' own time, pace, and location. In this article, theoretical and practical underpinnings underlying the Pe(e)rfectly Skilled method are described.

Pitch-presentatie over Viewbrics. Viewbrics [ http://www.viewbrics.nl/ ] onderzoekt hoe videovoorbeelden in combinatie met rubrieken gebruikt kunnen worden om vaardigheden te beoordelen. Om complexe algemene vaardigheden onder de knie te krijgen is het belangrijk dat je een concreet en samenhangend beeld hebt van die vaardigheid. Viewbrics beschrijft de verschillende beheersingsniveaus van drie complexe vaardigheden: presenteren, samenwerken en informatiewijsheid. Viewbrics doet dat in tekst en verrijkt die tekstuele rubrics met videovoorbeelden. Zo kunnen leerlingen en leerkrachten hun prestaties vergelijken met de experts en de kunst ‘afkijken’ bij iemand die het al beheerst. Het project werkt samen met een kernteam met leerkrachten en leerlingen uit het voortgezet onderwijs, onderzoekers en experts.

To learn complex generic (21st century) skills students need to form a mental model of a skill. If students know their current and strived-for mastery level, they can better determine subsequent learning activities towards skills mastery. Rubrics support students in self-assessment of their skills as they provide textual descriptions of skills’ mastery levels with performance indicators for all constituent subskills. However, text-based rubrics have a limited capacity to support the formation of mental models of a complex skill. Video-enhanced rubrics (VER) with video modeling examples have the potential to improve and enrich mental model formation. VER can also be used to improve feedback quality a student receives from teachers, experts and peers while practicing a skill. Both affordances of VER can thus improve students’ performance.In the Viewbrics-project we therefore develop, through design-based research, a methodology for the formative assessment of complex skills with Video-Enhanced Rubrics (VER), precipitated in an online tool. We implemented this methodology in the online tool for three skills: collaborating, presenting and becoming information literate. This paper describes the features of the first prototype of the online tool and the results of a stakeholder evaluation of its perceived usefulness and usability, by means of a questionnaire and card-sorting exercise, with 7 teachers and 21 students of two secondary schools.The evaluation of this first prototype showed that both teachers and students evaluated the online tool and formative assessment methodology as handy, usable, helpful and feasible for learning complex skills, although some recommendations were made to further improve the design of the tool.

Rubrieken zijn in het onderwijs een veelgebruikt instrument om de beheersing van complexe vaardigheden zowel formatief als summatief te evalueren. Eén van de ontwerpaspecten van een rubriek is het definiëren en onderscheiden van de verschillende beheersingsniveau’s van een deelvaardigheid. Een ontwerpprobleem hierbij is het bepalen welke dimensies voor de kwaliteit van gedragscriteria bijdragen aan de complexiteit van een deel vaardigheid en deze te vertalen naar een consistente schaal met alle beheersingsniveau’s. Om voor docenten en onderwijsontwerpers zicht te krijgen op welke dimensies er op dit moment in de praktijk gebruikt worden voor de uitwerking van beheersingsniveau’s evalueerden we ruim 600 rubrieken voor het beoordelen van ‘samenwerken’. Uit 13 strikt geselecteerde rubrieken haalden we 6 categorieën met daarbinnen 13 dimensies voor het definiëren van schalen voor beheersingsniveau’s.

Rubrieken zijn in het onderwijs een veelgebruikt instrument om de beheersing van complexe vaardigheden zowel formatief als summatief te evalueren. Eén van de ontwerpaspecten van een rubriek is het definiëren en onderscheiden van de verschillende beheersingsniveau’s van een deelvaardigheid. Een ontwerpprobleem hierbij is het bepalen welke dimensies voor de kwaliteit van gedragscriteria bijdragen aan de complexiteit van een deel vaardigheid en deze te vertalen naar een consistente schaal met alle beheersingsniveau’s. Om voor docenten en onderwijsontwerpers zicht te krijgen op welke dimensies er op dit moment in de praktijk gebruikt worden voor de uitwerking van beheersingsniveau’s evalueerden we ruim 600 rubrieken voor het beoordelen van ‘samenwerken’. Uit 13 strikt geselecteerde rubrieken haalden we 6 categorieën met daarbinnen 13 dimensies voor het definiëren van schalen voor beheersingsniveau’s.

Many schools use analytic rubrics to (formatively) assess complex, generic or transversal (21st century) skills, such as collaborating and presenting. In rubrics, performance indicators on different levels of mastering a skill (e.g., novice, practiced, advanced, talented) are described. However, the dimensions used to describe the different mastery levels vary within and across rubrics and are in many cases not consistent, concise and often trivial, thereby hampering the quality of rubrics used to learn and assess complex skills. In this study we reviewed 600 rubrics available in three international databases (Rubistar, For All Rubrics, i-rubrics) and analyzed the dimensions found within 12 strictly selected rubrics that are currently used to distinguish mastery levels and describe performance indicators for the skill 'collaboration' at secondary schools. These dimensions were subsequently defined and categorized. This resulted in 13 different dimensions, clustered in 6 categories, feasible for defining skills’ mastery levels in rubrics. The identified dimensions can specifically support both teachers and researchers to construct, review and investigate performance indicators for each mastery level of a complex skill. On a more general level, they can support analysis of the overall quality of analytic rubrics to (formatively) assess complex skills.

This document is framed within the European Lifelong Learning project “Proposing modern e‐assessment approaches and tools to young and experienced in‐service teachers” (PREATY). The main purpose of this document is to identify the most relevant educational and technical considerations that may lead the decisions made in other related work packages. This document: - examines the key skills and competences that are needed to be assessed in a Technology Enhanced Environment; - reviews modern assessment approaches and tools for assessing key skills and competences; - reviews teacher training policies in the national and regional contexts of the partners who will be in charge of organizing teacher workshops in their countries; - proposes guidelines for teacher training on modern assessment approaches and tools; From this work we can conclude that: - some skills and competences are not frequently addressed by the schooling system. There is a need to increase reflection and awareness about how these skills and competences can be assessed; - some of the reviewed e‐assessment approaches and tools, like e‐Portfolio, rubrics and some integrated tools have been applied in secondary school contexts, just a few have been applied in primary education; - some approaches like learning analytics are still far from their adoption by teachers at any of the two levels; - a training approach for primary schools could be to familiarize teachers with not only the tools but also the formative e‐assessment approaches; - introducing formative e‐assessment in the schools requires both a change in the mentality of teachers (and other stakeholders) and tools that are better adapted to their needs and contexts. The PREATY project aims to contribute to both lines of work.

The scope of this deliverable is to present the approach adopted by the PREATY project (PRoposing modern E-assessment Approaches and Tools to Young and experienced in-service teachers) in Workpackage 3, and to report the main results of the teacher training activities as well as the school pilot cases studies carried out. This deliverable will be structured in three main parts. Part I will provide an overview of the organization of training seminars organized in each country. Part II will be focused on the pilot case studies that will happen in each country and the lessons learned. Part III will contain the global results.

Schools are in the process of reorienting their curricula to the development of 21st century skills and competences; however, their assessment methods have not yet been updated. The PREATY project aims to make teachers in primary and secondary schools aware of the use and benefits of modern e-assessment methods and tools. This paper presents the preliminary work developed in the project.