Search

High-quality elaborative peer feedback is a blessing for both learners and teachers. However, learners can experience difficulties in giving high-quality feedback on complex skills using textual analytic rubrics. High-quality elaborative feedback can be strengthened by adding video-modeling examples with embedded self-explanation prompts, turning textual analytic rubrics (TR) into so-called 'video-enhanced analytic rubrics' (VER). This study contrasts two experimental conditions (TR, n = 54; VERs, n = 49) with their version of the anonymized online tool (used to collect the given feedback in 'Tips for improvement and Tops identifying strengths'). Peer feedback quality (concreteness and consistency) was evaluated using Natural Language Processing. As expected, the video-enhanced rubrics condition resulted in a higher quantity of words used and a lower amount of naive wording compared to the textual rubric condition. Contrary to our assumptions, it did not lower the amount of non-constructive wording nor improved the amount of behavioral and process-related feedback. Possibly, the transition from providing more feedback to delivering more accurate behavioral and process-related feedback has not yet been made in the time set for the study.

Higher education is faced with the question of how large numbers of students can be supported to learn complex skills without increasing teachers' supervision time proportionally and while preserving, or preferably improving, quality. Practicing skills only once does not work. Students need repetitive practice, feedback, and structured support to master a skill. They need to gain insight into what went well and what could be improved, so that they can further direct their attention while practicing. However, teachers cannot provide feedback after every practice session of students because their time is scarce. To solve this problem, an online formative assessment method for interactive and practice-oriented skills' training, Pe(e)rfectly Skilled, was developed. The Pe(e)rfectly Skilled method provides structured support for self-regulation, goal setting, feedback, and reflection. This method affords practicing skills repetitively, both individually and collaboratively, at students' own time, pace, and location. In this article, theoretical and practical underpinnings underlying the Pe(e) rfectly Skilled method are described.

Acquiring complex oral presentation skills is cognitively demanding for students and demands intensive teacher guidance. The aim of this study was twofold: (a) to identify and apply design guidelines in developing an effective formative assessment method for oral presentation skills during classroom practice, and (b) to develop and compare two analytic rubric formats as part of that assessment method. Participants were first-year secondary school students in the Netherlands (n = 158) that acquired oral presentation skills with the support of either a formative assessment method with analytic rubrics offered through a dedicated online tool (experimental groups), or a method using more conventional (rating scales) rubrics (control group). One experimental group was provided text-based and the other was provided video-enhanced rubrics. No prior research is known about analytic video-enhanced rubrics, but, based on research on complex skill development and multimedia learning, we expected this format to best capture the (non-verbal aspects of) oral presentation performance. Significant positive differences on oral presentation performance were found between the experimental groups and the control group. However, no significant differences were found between both experimental groups. This study shows that a well-designed formative assessment method, using analytic rubric formats, outperforms formative assessment using more conventional rubric formats. It also shows that higher costs of developing video-enhanced analytic rubrics cannot be justified by significant more performance gains. Future studies should address the generalizability of such formative assessment methods for other contexts, and for complex skills other than oral presentation, and should lead to more profound understanding of video-enhanced rubrics.

Learners in the process of developing complex skills need a rich mental model of what such skills entail. Textual analytics rubrics (TR) are a widely used instrument to support formative assessment of complex skills, supporting feedback, reflection, and thus mental model development of complex skills. However, the textual nature of a rubric limits the possibility to deliver contextual and dynamic (time-and behaviour-oriented) information. In the Viewbrics online tool, we developed a version supporting the delivery of contextual and dynamic information by adding video-modelling examples with embedded self-explanation prompts to textual analytics rubrics. We called this combination as video-enhanced rubrics (VERs). Our current study investigates whether the Viewbrics online tool supports complex skills development and whether either textual- or video-enhanced rubrics best support complex skills' mastery. The study was a three-group (VERS n = 49, TR n = 54, control n = 50) within-subjects design. Learners' performance of complex skills was measured through expert, peer, and self-assessment using the Viewbrics online tool. A multilevel regression analysis shows learners in the TR and VERS conditions consistently outperforming the control condition to varying degrees across skills. However, no differences have been found between the two experimental conditions. Positive results across different complex skills indicate the Viewbrics online tool can be used to support the development of a wide range of complex skills in secondary education.

Introductory Psychology courses in Higher Education lack application of knowledge in solving work-based problems. We develop and study mini-games that support a more active and experiential way of learning to increase professional awareness. This paper describes the instructional design and the factors under study: learner control and authenticity. We compared professional awareness and perceived authenticity for the old (without game, n = 130) and new course (with game, n = 197) by administering pre- and post-questionnaires. Participants were allocated to game variants, differing in freedom of assignment order (less or more learner control) and number of sources available (less or more authentic). We computer logged their game actions, and asked them additional questions about gameplay that concerned aspects like motivation, flow, performance, learnability and usability. Both courses improved professional awareness (knowledge gains of 4% for the old and 12% for the new course), but only in the new course with mini-games this was significant. Students evaluated gameplay and their content as "more than sufficient" to "good." A free order of assignments was found to produce more effective (in-game) performance, and the provision of more sources was found to improve learnability, authentic learning and appreciations of gameplay.

Background: Complex skills, like analytical thinking, are essential in preparing students for future professions. Serious games hold potential to stimulate the online acquisition of such professional skills in an active and experiential way. Objective: Rubrics are proven assessment and evaluation instruments, but were never directly integrated into actual gameplay. We present our approach towards the integration of analytical rubrics into gameplay, the implementation into existing (scenario‐based) serious games, and main findings from a study on learning effects and appreciation of the tooling (dashboard with progress information and reflective feedback). Method: Our experimental research setup randomly allocated 84 participating students into either a control group or an experimental group. The intervention for the control group consisted of an existing serious game with natural feedback as part of the scenario, supporting the gameplay itself. The intervention for the experimental group also provided an additional dashboard with reflective feedback, monitoring the acquisition of analytical skills through gameplay. Learning growth through gameplay was measured by pre‐/post‐test scores on a knowledge test, in‐game performance by means of computer logging, and appreciation of the dashboard and feedback by means of a questionnaire. Results and Conclusions: Participants receiving reflective feedback show higher learning increases (on a pre‐/post‐knowledge test), and more efficient gameplay behaviour (on performance scores). Participants receiving reflective feedback feel more confident about competence development. These findings appear promising for this approach on integrating formative assessment in serious gameplay to support the acquisition of complex skills. Lay Description: What is already known about this topic: Serious games hold potential to stimulate the online acquisition of complex professional skills in an active and experiential way. Analytical thinking is an important professional competence.(Analytical) Rubrics are proven formative assessment and evaluation instruments, also to support the acquisition of professional complex skills, like analytical thinking. What this paper adds: (Analytical) Rubrics were never directly integrated into actual gameplay. This article presents a content validation approach towards the integration of analytical rubrics for analytical thinking into gameplay.The article furthermore presents the actual implementation of tooling into existing (scenario‐based) serious games, and main findings from a study on learning effects and appreciation of the tooling (dashboard with progress information and reflective feedback). Implications for practice and/or policy: Participants receiving the dashboard information indeed show higher learning growth, and more efficient gameplay behaviour.Students appreciate the dashboard, which makes them more confident about their competence development.Both findings appear promising for this approach on integrating formative assessment in serious gameplay to support the acquisition of complex skills. [ABSTRACT FROM AUTHOR]

Although the importance of cognitive feedback in digital serious games (DSG) is undisputed, we are facing some major design challenges. First of all, we do not know to which extend existing research guidelines apply when we stand the risk of cognitive feedback distorting the delicate balance between learning and playing. Unobtrusive cognitive feedback has to be interspersed with gameplay. Second, many effective solutions for providing cognitive feedback we do know might simply be too costly. To face both challenges, this study offers an efficient approach for providing unobtrusive and retrospective cognitive feedback (RCF) in DSG. This approach was applied onto a game where feedback messages were triggered via simple rules about learners' questioning behavior on four dimensions. We found the experimental condition including such RCF to yield better learning outcomes while maintaining similar motivation.

Serious games foster the acquisition of complex problem-solving skills. Assessment of such skills should be in line with instruction, and within a serious game environment its content validity should equal face-to-face assessment. Research on assessment in serious gaming has remained rather scarce. This article shows how assessment can be implemented in serious gaming in a way that assures content validity. The core of the authors' validation method entails mapping learning activities (as contained in the game scenario) on performance indicators and outputs (as derived from formal attainment levels). They present how they have elaborated and applied the method for an assessment game for ICT managers in secondary vocational education. They describe the procedure and extent to which this assessment is content-valid compared to face-to-face assessment.

The teaching of communication skills is a labour-intensive task because of the detailed feedback that should be given to learners during their prolonged practice. This study investigates to what extent our FILTWAM facial and vocal emotion recognition software can be used for improving a serious game (the Communication Advisor) that delivers a web-based training of communication skills. A test group of 25 participants played the game wherein they were requested to mimic specific facial and vocal emotions. Half of the assignments included direct feedback and the other half included no feedback. It was investigated whether feedback on the mimicked emotions would lead to better learning. The results suggest the facial performance growth was found to be positive, particularly significant in the feedback condition. The vocal performance growth was significant in both conditions. The results are a significant indication that the automated feedback from the software improves learners' communication performances.

This paper presents the voice emotion recognition part of the FILTWAM framework for real-time emotion recognition in affective e-learning settings. FILTWAM (Framework for Improving Learning Through Webcams And Microphones) intends to offer timely and appropriate online feedback based upon learner's vocal intonations and facial expressions in order to foster their learning. Whereas the facial emotion recognition part has been successfully tested in a previous study, the here presented study describes the development and testing of FILTWAM's vocal emotion recognition software artefact. The main goal of this study was to show the valid use of computer microphone data for real-time and adequate interpretation of vocal intonations into extracted emotional states. The software that was developed was tested in a study with 12 participants. All participants individually received the same computer-based tasks in which they were requested 80 times to mimic specific vocal expressions (960 occurrences in total). Each individual session was recorded on video. For the validation of the voice emotion recognition software artefact, two experts annotated and rated participants' recorded behaviours. Expert findings were then compared with the software recognition results and showed an overall accuracy of Kappa of 0.743. The overall accuracy of the voice emotion recognition software artefact is 67% based on the requested emotions and the recognized emotions. Our FILTWAM-software allows to continually and unobtrusively observing learners' behaviours and transforms these behaviours into emotional states. This paves the way for unobtrusive and real-time capturing of learners' emotional states for enhancing adaptive e-learning approaches.

This paper presents a framework (FILTWAM (Framework for Improving Learning Through Webcams And Microphones)) for real-time emotion recognition in e-learning by using webcams. FILTWAM offers timely and relevant feedback based upon learner's facial expressions and verbalizations. FILTWAM's facial expression software module has been developed and tested in a proof-of-concept study. The main goal of this study was to validate the use of webcam data for a real-time and adequate interpretation of facial expressions into extracted emotional states. The software was calibrated with 10 test persons. They received the same computer-based tasks in which each of them were requested 100 times to mimic specific facial expressions. All sessions were recorded on video. For the validation of the face emotion recognition software, two experts annotated and rated participants' recorded behaviours. Expert findings were contrasted with the software results and showed an overall value of kappa of 0.77. An overall accuracy of our software based on the requested emotions and the recognized emotions is 72%. Whereas existing software only allows not-real time, discontinuous and obtrusive facial detection, our software allows to continuously and unobtrusively monitor learners' behaviours and converts these behaviours directly into emotional states. This paves the way for enhancing the quality and efficacy of e-learning by including the learner's emotional states.

Multiuser immersive learning scenarios hold strong potential for lifelong learning as they can support the acquisition of higher order skills in an effective, efficient, and attractive way. Existing virtual worlds, game development platforms, and game engines only partly cater for the proliferation of such learning scenarios as they are often inadequately tuned for learning. First, this study aims to identify architectures that more effectively support the development of multiuser immersive learning scenarios. Second, this study takes up the challenge to define and assemble more flexible architectures that cater for fast and easy development, which will become important in the current period of economic breakdown. Third, this study describes how such architectures should enable research into guidelines for multiuser immersive learning scenario design and development. This study outlines a method for defining and setting up such architectures by using experts and existing literature. (Contains 2 tables and 5 figures.)

This paper examines how learning outcomes from playing serious games can be enhanced by including scripted collaboration in the game play. We compared the quality of advisory reports, that students in the domain of water management had to draw up for an authentic case problem, both before and after collaborating on the problem with (virtual) peer students. Peers studied the case from either an ecological or governance perspective, and during collaboration both perspectives had to be confronted and reflected upon. This paper argues why such type of workplace-based learning scenarios are important for professional development, describes how serious gaming scenarios can be designed to support such complex learning, and reports data on student satisfaction and learning effects of including scripted collaboration. Preliminary results from a pilot study with 12 students show that including scripted collaboration significantly enhances the quality of learning outcomes.

The need to support users of the Internet with the selection of information is becoming more important. Learners in complex, self-organising Learning Networks have similar problems and need guidance to find and select most suitable learning activities, in order to attain their lifelong learning goals in the most efficient way. Several research questions regarding efficiency and effectiveness deal with adequate navigation support through recommender systems. To answer some of these questions an experiment was set up within an Introduction Psychology course of the Open University of the Netherlands. Around 250 students participated in this study and were monitored over an experimental period of four months. All were provided the same course materials, but only half of them were supported with a personalised recommender system. This study examined the effects of the navigation support on the completion of learning activities (effectiveness), needed time to comply them (efficiency), actual use of and satisfaction with the system, and the variety of learning paths. The recommender system positively influenced all measures, by having significant effects on efficiency, satisfaction and variety. (Contains 7 figures and 4 tables.)

For competences development of learners and professionals, target competences and corresponding competence development opportunities have to be identified. Personal Recommender Systems (PRS) provide personal recommendations for learners aimed at finding and selecting learning activities that best match their needs. This article argues that a clear-cut description of the concept of "competence" is needed for appropriate system-based personal recommendations. It is proposed to extend current initiatives on standardization of modelling competences with the concept of "Learning and performance situation" (LP-situation) and it is claimed that this extension has added value for personal recommendations for adequate selection of learning activities, for determining proficiency levels of competences, for the design of learning tasks, and for accreditation of prior knowledge. Advantages and disadvantages are discussed. (Contains 3 tables.)

Societal changes demand educators to apply new pedagogical approaches. Many educational stakeholders feel that serious games could play a key role in fulfilling this demand, and they lick their chops when looking at the booming industry of leisure games. However, current toolkits for developing leisure games show severe shortcomings when applied to serious games. Developing effective serious games in an efficient way requires a specific approach and tool set. This article describes the EMERGO methodology and generic toolkit for developing and delivering scenario-based serious games that are aimed at the acquisition of complex cognitive skills in higher education. Preliminary evaluation results with case developers using the EMERGO methodology and toolkit and with learners using EMERGO cases are presented. (Contains 4 figures and 5 tables.)

In an e-learning environment there is a need to integrate various e-learning services like assessment services, collaboration services, learning design services and communication services. In this article we present the design and implementation of a generic integrative service framework, called CopperCore Service Integration (CCSI). We will concentrate on the integration of two services: CopperCore, an IMS Learning Design service and an IMS Question and Test Interoperability service called Assessment Provision through Interoperable Segments (APIS). One of the design goals of the architecture was to minimize the intrusion for both the services as well as any legacy client that already uses these services. The result of this work is that the flow of learning activities can be made dependent on test results. (Contains 3 figures.)

Whole tasks for acquiring complex skills are often too difficult for novices. To solve this problem, "process support" divides the problem solving into phases, offers driving questions, and provides feedback. A multimedia program was used to teach sophomore law students ("N"=82) to prepare and carry out a plea. In a randomised 2x2 design with the factors number of phases and availability of driving questions, students solving a task with fewer phases performed better and more efficiently than students exposed to more phases. Also, students receiving driving questions performed better, although not more efficiently than students not receiving such questions. The results indicate that whole tasks should be accompanied by process support, although task characteristics might restrict the benefits of driving questions in this support.

Background: Carrying out whole tasks is often too difficult for novice learners attempting to acquire complex skills. The common solution is to split up the tasks into a number of smaller steps. The number of steps must be optimized for efficient and effective learning. Aim: The aim of the study is to investigate the relation between the number of steps provided to learners and the quality of their learning of complex skills. It is hypothesized that students receiving an optimized number of steps will learn better than those receiving either the whole task in only one step or those receiving a large number of steps. Sample: Participants were 35 sophomore law students studying at Dutch universities, mean age = 22.8 years (SD = 3.5), 63% were female. Method: Participants were randomly assigned to 1 of 3 computer-delivered versions of a multimedia programme on how to prepare and carry out a law plea. The versions differed only in the number of learning steps provided. Videotaped plea-performance results were determined, various related learning measures were acquired and all computer actions were logged and analyzed. Results: Participants exposed to an intermediate (i.e. optimized) number of steps outperformed all others on the compulsory learning task. No differences in performance on a transfer task were found. A high number of steps proved to be less efficient for carrying out the learning task. Conclusions: An intermediate number of steps is the most effective, proving that the number of steps can be optimized for improving learning.

An instrument for measuring the complexity of learning tasks in the field of Law was developed and tested in 3 experiments. In Experiments 1 and 2, teachers used the card-sort method to rate the complexity of learning tasks. Based on the outcomes, a benchmark scale with 4 criterion tasks was used in Experiment 3. The results showed the benchmark instrument to be valid and easy to use, allowing instructional designers to design competency-based learning environments that better take task complexity into account. The Instructional Design model, in which teachers determine task complexity, is briefly described.

Presents a two-phase instructional design model that focuses on optimizing step size in whole-task approaches to learning complex, mainly non-recurrent, cognitive skills. Step size in a multiple-step whole-task approach-needed for the process worksheets-is determined on the basis of estimated part-task complexity. A developmental study of the model is illustrated with examples from the domain of law. (AEF)

A plethora of studies stress students' self-regulated learning (SRL) skills to be conditional for successful learning in school and beyond. In general, self-regulated learners are actively engaged in constructing their own understanding also including the regulation of contextual features in the environment. Within the contextual features, the regulation of peer interaction is necessary, because college courses increasingly require peer learning. This goes along with the increasing interest for online learning settings, due in no small part to the recent COVID-19 pandemic. In the present study we explore how social presence (i.e., the degree to which the other person is perceived as physical "real"), social space (i.e., trust building between peers) and sociability (i.e., the degree to which the virtual learning environment supports social presence and social space) are essential elements in the regulation of online peer interaction. To shed light in this matter, higher education students were qualitatively followed for 1 year in an online academic writing course by using retrospective interviews (n = 7) and reflective questions (n = 62). Additionally, for social presence, students' perceptions were quantitatively measured with a validated questionnaire (n = 41). The results show that the planning phase is the most important phase for supporting students' social presence because that is where the regulation of peer interaction becomes important. The sociability has an important role here as well becoming less prominent further on in the self-regulation process. In the SRL follow-up phases, students look for other ways to increase their social presence and social space in order to shape the regulation of peer interaction from a position of trust. In the evaluation phase, students are aware of the importance of social presence but less of social space for the regulation of peer interaction. We conclude with some design principles to facilitate students' regulation of peer interaction in online settings. [ABSTRACT FROM AUTHOR]