Search

High-quality elaborative peer feedback is a blessing for both learners and teachers. However, learners can experience difficulties in giving high-quality feedback on complex skills using textual analytic rubrics. High-quality elaborative feedback can be strengthened by adding video-modeling examples with embedded self-explanation prompts, turning textual analytic rubrics (TR) into so-called 'video-enhanced analytic rubrics' (VER). This study contrasts two experimental conditions (TR, n = 54; VERs, n = 49) with their version of the anonymized online tool (used to collect the given feedback in 'Tips for improvement and Tops identifying strengths'). Peer feedback quality (concreteness and consistency) was evaluated using Natural Language Processing. As expected, the video-enhanced rubrics condition resulted in a higher quantity of words used and a lower amount of naive wording compared to the textual rubric condition. Contrary to our assumptions, it did not lower the amount of non-constructive wording nor improved the amount of behavioral and process-related feedback. Possibly, the transition from providing more feedback to delivering more accurate behavioral and process-related feedback has not yet been made in the time set for the study.

Higher education is faced with the question of how large numbers of students can be supported to learn complex skills without increasing teachers' supervision time proportionally and while preserving, or preferably improving, quality. Practicing skills only once does not work. Students need repetitive practice, feedback, and structured support to master a skill. They need to gain insight into what went well and what could be improved, so that they can further direct their attention while practicing. However, teachers cannot provide feedback after every practice session of students because their time is scarce. To solve this problem, an online formative assessment method for interactive and practice-oriented skills' training, Pe(e)rfectly Skilled, was developed. The Pe(e)rfectly Skilled method provides structured support for self-regulation, goal setting, feedback, and reflection. This method affords practicing skills repetitively, both individually and collaboratively, at students' own time, pace, and location. In this article, theoretical and practical underpinnings underlying the Pe(e) rfectly Skilled method are described.

The development of multidisciplinary education requires people to communicate, learn, and design beyond the boundaries of their own domains. In this research, an education design framework is developed to facilitate and support university teachers in multidisciplinary educational design. In addition, it serves as an aid to potentially transform domain-specific action-oriented knowledge into domain-integrated action-oriented knowledge by supporting knowledge co-construction across domain boundaries. The educational design framework, grounded in seamless and hybrid learning paradigms and theory on wicked problem solving, is being developed in a design-oriented educational research. This resulted in a multidisciplinary educational design game, which aims to facilitate cross-border communication, knowledge co-construction, and educational design processes during multidisciplinary educational design and improve the quality of the resulting multidisciplinary educational design.

Novice teachers are often discouraged by the problems they encounter in their daily professional practices and they (still) feel unable to cope with. This is also reflected in high drop-out rates in the early stages of teachers' careers. In this paper a theory-informed methodology to support novice teachers' individual and collective professional development at the workplace is proposed. This methodology, called MirrorMe@work, strives to reduce teachers professional loneliness, increase their confidence and reduce stress, through technology-enhanced support of self- and co-regulation, (collective) reflection and knowledge co-creation processes, informed by (shared) memorable moments and critical incidents that teachers experience in their practices. Future and further research is needed to determine whether the MirrorMe@work method supports novice teachers as intended and whether this indeed reduces novice teachers' drop-out rates.

Acquiring complex oral presentation skills is cognitively demanding for students and demands intensive teacher guidance. The aim of this study was twofold: (a) to identify and apply design guidelines in developing an effective formative assessment method for oral presentation skills during classroom practice, and (b) to develop and compare two analytic rubric formats as part of that assessment method. Participants were first-year secondary school students in the Netherlands (n = 158) that acquired oral presentation skills with the support of either a formative assessment method with analytic rubrics offered through a dedicated online tool (experimental groups), or a method using more conventional (rating scales) rubrics (control group). One experimental group was provided text-based and the other was provided video-enhanced rubrics. No prior research is known about analytic video-enhanced rubrics, but, based on research on complex skill development and multimedia learning, we expected this format to best capture the (non-verbal aspects of) oral presentation performance. Significant positive differences on oral presentation performance were found between the experimental groups and the control group. However, no significant differences were found between both experimental groups. This study shows that a well-designed formative assessment method, using analytic rubric formats, outperforms formative assessment using more conventional rubric formats. It also shows that higher costs of developing video-enhanced analytic rubrics cannot be justified by significant more performance gains. Future studies should address the generalizability of such formative assessment methods for other contexts, and for complex skills other than oral presentation, and should lead to more profound understanding of video-enhanced rubrics.

Many schools use analytic rubrics to (formatively) assess complex, generic or transversal (21st century) skills, such as collaborating and presenting. In rubrics, performance indicators on different levels of mastering a skill (e.g., novice, practiced, advanced, talented) are described. However, the dimensions used to describe the different mastery levels vary within and across rubrics and are in many cases not consistent, concise and often trivial, thereby hampering the quality of rubrics used to learn and assess complex skills. In this study we reviewed 600 rubrics available in three international databases (Rubistar, For All Rubrics, i-rubrics) and analyzed the dimensions found within 12 strictly selected rubrics that are currently used to distinguish mastery levels and describe performance indicators for the skill 'collaboration' at secondary schools. These dimensions were subsequently defined and categorized. This resulted in 13 different dimensions, clustered in 6 categories, feasible for defining skills' mastery levels in rubrics. The identified dimensions can specifically support both teachers and researchers to construct, review and investigate performance indicators for each mastery level of a complex skill. On a more general level, they can support analysis of the overall quality of analytic rubrics to (formatively) assess complex skills.

Learners in the process of developing complex skills need a rich mental model of what such skills entail. Textual analytics rubrics (TR) are a widely used instrument to support formative assessment of complex skills, supporting feedback, reflection, and thus mental model development of complex skills. However, the textual nature of a rubric limits the possibility to deliver contextual and dynamic (time-and behaviour-oriented) information. In the Viewbrics online tool, we developed a version supporting the delivery of contextual and dynamic information by adding video-modelling examples with embedded self-explanation prompts to textual analytics rubrics. We called this combination as video-enhanced rubrics (VERs). Our current study investigates whether the Viewbrics online tool supports complex skills development and whether either textual- or video-enhanced rubrics best support complex skills' mastery. The study was a three-group (VERS n = 49, TR n = 54, control n = 50) within-subjects design. Learners' performance of complex skills was measured through expert, peer, and self-assessment using the Viewbrics online tool. A multilevel regression analysis shows learners in the TR and VERS conditions consistently outperforming the control condition to varying degrees across skills. However, no differences have been found between the two experimental conditions. Positive results across different complex skills indicate the Viewbrics online tool can be used to support the development of a wide range of complex skills in secondary education.

For investigating a comprehensive PLC framework, instruments are needed that capture the multi-layered PLC characteristics and that take into account the complex influencing educational context. Such instruments are currently lacking. This study aims at describing the development and validation of two qualitative classification instruments usable for systematically identifying changes in PLCs. The 11 characteristics and accompanying behavioural and attitudinal indicators of a recently designed PLC framework were converted into codes and measuring values. For this conversion the 'directed content analysis approach' was used. The result is two classification instruments that can be used to collect and classify behavioural and attitudinal data from interviews and observations. The instruments cover all the characteristics, respectively indicators of the PLC framework and are suitable for validation by triangulation. This study contributes to the methodology for investigating PLCs by presenting two theoretically grounded and validated sets of codes, and two sets of measuring values. The latter is important for systematically developing knowledge on PLCs in educational practice. In everyday educational practice the instruments can be used to determine the state of the PLC and steps can be taken to stimulate its development.

The past two decades research on teachers' professional development focusses increasingly on professional learning communities (PLCs). Based on a literature search a conceptual framework was developed including a comprehensive PLC-concept. Comprehensive because it takes into account the complexity of the educational context. The PLC-concept outlining 11 characteristics, is embedded in a framework, containing five external factors that influence. The framework was evaluated by interviewing educational experts. The literature search and the interviews showed a great diversity of distinct PLC characteristics and a wide range of external influencing factors. Based on expert interviews, no essential elements appeared to be lacking in the framework, moreover no other perspectives on the framework were mentioned. An additional search on recent literature confirmed the constructed framework. As yet we concluded that the developed concept is sufficiently comprehensive and feasible for conducting research on PLCs.

Fostering the development of Professional Learning communities (PLCs) should be a priority for education because of their capacity to enhance teachers' professional development. However, in everyday educational (school) contexts studying and supporting the growth of PLCs is a complex endeavour. As a consequence, there is a lack of instruments for extensively investigating PLCs on different levels of development as well as the factors that influence the development. This article describes the first steps of the construction of measuring instruments, suitable to investigate PLCs in the complex school context: a study of the operationalisation of PLC characteristics and influencing factors into attitudinal and behavioural indicators. The operationalisation is founded on relevant literature and on educational practice, the latter by focus groups consisting of educational practitioners. This study yielded indicators for the eleven characteristics of the PLC concept and two context factors, which were subsequently used for selecting and constructing instruments. This study contributes to the research methodology for investigating PLCs and to bridge the gap between educational research and educational practice. The latter by constructing instruments that are considered relevant, recognisable and practical by educational practitioners.

For learners, it can be difficult to imagine how to perform a complex skill based on textual information from solely a text-based analytic rubric. Rubrics lack (1) the contextual information needed to convey real-world attributes (2) the dynamic information (such as gesturing in the complex skill of presenting) (3) the procedural information required to support the automation of constituent skills. We propose to address the text-based rubric's deficiencies by adding video-modeling examples, self-explanation prompts, an intertwined educational- and instructional narrative, natural segmentation, and a non-verbal script. With the resulting Video Enhanced Rubric, we aim to improve the formative assessment of complex skills by fostering learner's mental model development, feedback quality, and complex skill mastery. Designing multimedia to support the formative assessment of complex skills using can cause dilemmas for instructional designers. For example, is learner control needed to foster intrinsic motivation or is it deemed to create extraneous cognitive load? Is it wise to use a video modeling example of peer-aged learners when the model does not display perfect performance? We found seven dilemmas around proven complex skill development, motivational design, and multimedia design guidelines. This paper presents a theoretical contribution to instructional design by introducing a framework to address dilemmas regarding such design dilemmas. As a practical contribution, we support educational researchers and practitioners by presenting six practical guidelines for designing a Video Enhanced Rubric. A use case of the Viewbrics-project provides insight into the practical application of the framework within the context of Dutch pre-university education.

This paper proposes a design framework to support science education through blended learning, based on a participatory and interactive approach supported by ICT-based tools, called "Science Learning Activities Model" (SLAM). The development of this design framework started as a response to complex changes in society and education (e.g. high turnover rate of knowledge, changing labour market), which require a more creative response of learners to the world problems that surround them. Many of these challenges are related to science and it would be expected that students are attracted to science, however the contrary is the case. One of the origins of this disinterest can be found in the way science is taught. Therefore, after reviewing the relevant literature we propose the SLAM framework as a tool to aid the design of science courses with high motivational impact on students. The framework is concerned with the assumption that science learning activities should be applicable and relevant to contemporary life and transferable to "real-world" situations. The design framework proposes three design dimensions: context, technology and pedagogy, and aims at integrating learning in formal and informal contexts through blended learning scenarios by using today's flexible, interactive and immersive technologies (e.g. mobile, augmented reality, virtual reality).

Positioning in learning networks is a process that assists learners in finding a starting point and an efficient route through the network that will foster competence building. In the past we have explored computational approaches to positioning based on the contents of the learning network and the behaviour of those participating in it, more or less ignoring different efforts to stimulate positioning and competence development from a top-down perspective. In this paper we introduce a research agenda for positioning in learning networks, discuss several cases and give an outlook on the development of a positioning service for learning networks. (Contains 1 figure.)

Trend watching reports are an indispensable resource in the e-learning domain. Many HRD departments consider these reports as essential cornerstones for the development of their e-learning strategy. But what is the quality of the forecasts made in these reports? In this article, several methods of forecasting trends are discussed, resulting in a checklist to evaluate the quality of trend studies. Next, this checklist is applied to evaluate some significant trend studies. The evaluation results show that the reports do not meet basic quality criteria, such as "sound methodology" and "objectivity". The article concludes with some critical remarks on the role trend watching reports play in the domain of e-learning.

To link to full-text access for this article, visit this link: http://dx.doi.org/10.1016/j.chb.2009.01.002 Byline: Ellen Rusman (a), Jan van Bruggen (a), Ron Corvers (b), Peter Sloep (a), Rob Koper (a) Keywords: Design pattern; Trust; Computer supported collaborative learning; Virtual team; Distance education Abstract: This article describes the implementation and evaluation of a design pattern that fosters trust in mediated collaborative settings. The pattern proposed here should provide a profile with static and/or dynamic information about the participants of a collaborative environment. It aims to foster initial (in the first two to three weeks) trust in situations in which people do not know each other and do not have a chance to meet, but need to collaborate. A simple and low cost implementation of this pattern was realized by conducting a case study with participants of the European Virtual Seminar on Sustainable Development (EVS). They were asked to fill in a template with personal information about themselves. In the EVS, students collaboratively have to address sustainable development problems. Afterwards, students were questioned on the use of this template, called PEXPI, and their impressions of each other during and after the EVS; questionnaires and a semi-structured interview were used. The results show that the implementation of the static profile, derived from the pattern, initially helped students to form an impression of each other. After this initial period, however, students base their impression on factors such as the quality of work-related contributions, behaviour during collaboration (e.g. responsiveness), and communication style. This case study also shows that the pattern could easily be applied and transferred to a new context, as long as the conditions described in the 'context'-section of the pattern, were met. The case study provided a means for evaluation of the pattern and a source for its refinement. Author Affiliation: (a) Educational Technology Expertise Centre, Open University of The Netherlands, Valkenburgerweg 167, 6401 DL, Heerlen, Netherlands (b) School of Science, Open University of The Netherlands, Valkenburgerweg 167, 6401 DL, Heerlen, Netherlands