Search

Acquiring complex oral presentation skills is cognitively demanding for students and demands intensive teacher guidance. The aim of this study was twofold: (a) to identify and apply design guidelines in developing an effective formative assessment method for oral presentation skills during classroom practice, and (b) to develop and compare two analytic rubric formats as part of that assessment method. Participants were first-year secondary school students in the Netherlands (n = 158) that acquired oral presentation skills with the support of either a formative assessment method with analytic rubrics offered through a dedicated online tool (experimental groups), or a method using more conventional (rating scales) rubrics (control group). One experimental group was provided text-based and the other was provided video-enhanced rubrics. No prior research is known about analytic video-enhanced rubrics, but, based on research on complex skill development and multimedia learning, we expected this format to best capture the (non-verbal aspects of) oral presentation performance. Significant positive differences on oral presentation performance were found between the experimental groups and the control group. However, no significant differences were found between both experimental groups. This study shows that a well-designed formative assessment method, using analytic rubric formats, outperforms formative assessment using more conventional rubric formats. It also shows that higher costs of developing video-enhanced analytic rubrics cannot be justified by significant more performance gains. Future studies should address the generalizability of such formative assessment methods for other contexts, and for complex skills other than oral presentation, and should lead to more profound understanding of video-enhanced rubrics.

Introductory Psychology courses in Higher Education lack application of knowledge in solving work-based problems. We develop and study mini-games that support a more active and experiential way of learning to increase professional awareness. This paper describes the instructional design and the factors under study: learner control and authenticity. We compared professional awareness and perceived authenticity for the old (without game, n = 130) and new course (with game, n = 197) by administering pre- and post-questionnaires. Participants were allocated to game variants, differing in freedom of assignment order (less or more learner control) and number of sources available (less or more authentic). We computer logged their game actions, and asked them additional questions about gameplay that concerned aspects like motivation, flow, performance, learnability and usability. Both courses improved professional awareness (knowledge gains of 4% for the old and 12% for the new course), but only in the new course with mini-games this was significant. Students evaluated gameplay and their content as "more than sufficient" to "good." A free order of assignments was found to produce more effective (in-game) performance, and the provision of more sources was found to improve learnability, authentic learning and appreciations of gameplay.

Although the importance of cognitive feedback in digital serious games (DSG) is undisputed, we are facing some major design challenges. First of all, we do not know to which extend existing research guidelines apply when we stand the risk of cognitive feedback distorting the delicate balance between learning and playing. Unobtrusive cognitive feedback has to be interspersed with gameplay. Second, many effective solutions for providing cognitive feedback we do know might simply be too costly. To face both challenges, this study offers an efficient approach for providing unobtrusive and retrospective cognitive feedback (RCF) in DSG. This approach was applied onto a game where feedback messages were triggered via simple rules about learners' questioning behavior on four dimensions. We found the experimental condition including such RCF to yield better learning outcomes while maintaining similar motivation.

Serious games foster the acquisition of complex problem-solving skills. Assessment of such skills should be in line with instruction, and within a serious game environment its content validity should equal face-to-face assessment. Research on assessment in serious gaming has remained rather scarce. This article shows how assessment can be implemented in serious gaming in a way that assures content validity. The core of the authors' validation method entails mapping learning activities (as contained in the game scenario) on performance indicators and outputs (as derived from formal attainment levels). They present how they have elaborated and applied the method for an assessment game for ICT managers in secondary vocational education. They describe the procedure and extent to which this assessment is content-valid compared to face-to-face assessment.

Multiuser immersive learning scenarios hold strong potential for lifelong learning as they can support the acquisition of higher order skills in an effective, efficient, and attractive way. Existing virtual worlds, game development platforms, and game engines only partly cater for the proliferation of such learning scenarios as they are often inadequately tuned for learning. First, this study aims to identify architectures that more effectively support the development of multiuser immersive learning scenarios. Second, this study takes up the challenge to define and assemble more flexible architectures that cater for fast and easy development, which will become important in the current period of economic breakdown. Third, this study describes how such architectures should enable research into guidelines for multiuser immersive learning scenario design and development. This study outlines a method for defining and setting up such architectures by using experts and existing literature. (Contains 2 tables and 5 figures.)

This paper examines how learning outcomes from playing serious games can be enhanced by including scripted collaboration in the game play. We compared the quality of advisory reports, that students in the domain of water management had to draw up for an authentic case problem, both before and after collaborating on the problem with (virtual) peer students. Peers studied the case from either an ecological or governance perspective, and during collaboration both perspectives had to be confronted and reflected upon. This paper argues why such type of workplace-based learning scenarios are important for professional development, describes how serious gaming scenarios can be designed to support such complex learning, and reports data on student satisfaction and learning effects of including scripted collaboration. Preliminary results from a pilot study with 12 students show that including scripted collaboration significantly enhances the quality of learning outcomes.

For competences development of learners and professionals, target competences and corresponding competence development opportunities have to be identified. Personal Recommender Systems (PRS) provide personal recommendations for learners aimed at finding and selecting learning activities that best match their needs. This article argues that a clear-cut description of the concept of "competence" is needed for appropriate system-based personal recommendations. It is proposed to extend current initiatives on standardization of modelling competences with the concept of "Learning and performance situation" (LP-situation) and it is claimed that this extension has added value for personal recommendations for adequate selection of learning activities, for determining proficiency levels of competences, for the design of learning tasks, and for accreditation of prior knowledge. Advantages and disadvantages are discussed. (Contains 3 tables.)

Societal changes demand educators to apply new pedagogical approaches. Many educational stakeholders feel that serious games could play a key role in fulfilling this demand, and they lick their chops when looking at the booming industry of leisure games. However, current toolkits for developing leisure games show severe shortcomings when applied to serious games. Developing effective serious games in an efficient way requires a specific approach and tool set. This article describes the EMERGO methodology and generic toolkit for developing and delivering scenario-based serious games that are aimed at the acquisition of complex cognitive skills in higher education. Preliminary evaluation results with case developers using the EMERGO methodology and toolkit and with learners using EMERGO cases are presented. (Contains 4 figures and 5 tables.)

Whole tasks for acquiring complex skills are often too difficult for novices. To solve this problem, "process support" divides the problem solving into phases, offers driving questions, and provides feedback. A multimedia program was used to teach sophomore law students ("N"=82) to prepare and carry out a plea. In a randomised 2x2 design with the factors number of phases and availability of driving questions, students solving a task with fewer phases performed better and more efficiently than students exposed to more phases. Also, students receiving driving questions performed better, although not more efficiently than students not receiving such questions. The results indicate that whole tasks should be accompanied by process support, although task characteristics might restrict the benefits of driving questions in this support.

Background: Carrying out whole tasks is often too difficult for novice learners attempting to acquire complex skills. The common solution is to split up the tasks into a number of smaller steps. The number of steps must be optimized for efficient and effective learning. Aim: The aim of the study is to investigate the relation between the number of steps provided to learners and the quality of their learning of complex skills. It is hypothesized that students receiving an optimized number of steps will learn better than those receiving either the whole task in only one step or those receiving a large number of steps. Sample: Participants were 35 sophomore law students studying at Dutch universities, mean age = 22.8 years (SD = 3.5), 63% were female. Method: Participants were randomly assigned to 1 of 3 computer-delivered versions of a multimedia programme on how to prepare and carry out a law plea. The versions differed only in the number of learning steps provided. Videotaped plea-performance results were determined, various related learning measures were acquired and all computer actions were logged and analyzed. Results: Participants exposed to an intermediate (i.e. optimized) number of steps outperformed all others on the compulsory learning task. No differences in performance on a transfer task were found. A high number of steps proved to be less efficient for carrying out the learning task. Conclusions: An intermediate number of steps is the most effective, proving that the number of steps can be optimized for improving learning.

An instrument for measuring the complexity of learning tasks in the field of Law was developed and tested in 3 experiments. In Experiments 1 and 2, teachers used the card-sort method to rate the complexity of learning tasks. Based on the outcomes, a benchmark scale with 4 criterion tasks was used in Experiment 3. The results showed the benchmark instrument to be valid and easy to use, allowing instructional designers to design competency-based learning environments that better take task complexity into account. The Instructional Design model, in which teachers determine task complexity, is briefly described.

Presents a two-phase instructional design model that focuses on optimizing step size in whole-task approaches to learning complex, mainly non-recurrent, cognitive skills. Step size in a multiple-step whole-task approach-needed for the process worksheets-is determined on the basis of estimated part-task complexity. A developmental study of the model is illustrated with examples from the domain of law. (AEF)