Search

Acquiring complex oral presentation skills is cognitively demanding for students and demands intensive teacher guidance. The aim of this study was twofold: (a) to identify and apply design guidelines in developing an effective formative assessment method for oral presentation skills during classroom practice, and (b) to develop and compare two analytic rubric formats as part of that assessment method. Participants were first-year secondary school students in the Netherlands (n = 158) that acquired oral presentation skills with the support of either a formative assessment method with analytic rubrics offered through a dedicated online tool (experimental groups), or a method using more conventional (rating scales) rubrics (control group). One experimental group was provided text-based and the other was provided video-enhanced rubrics. No prior research is known about analytic video-enhanced rubrics, but, based on research on complex skill development and multimedia learning, we expected this format to best capture the (non-verbal aspects of) oral presentation performance. Significant positive differences on oral presentation performance were found between the experimental groups and the control group. However, no significant differences were found between both experimental groups. This study shows that a well-designed formative assessment method, using analytic rubric formats, outperforms formative assessment using more conventional rubric formats. It also shows that higher costs of developing video-enhanced analytic rubrics cannot be justified by significant more performance gains. Future studies should address the generalizability of such formative assessment methods for other contexts, and for complex skills other than oral presentation, and should lead to more profound understanding of video-enhanced rubrics.

For learners, it can be difficult to imagine how to perform a complex skill based on textual information from solely a text-based analytic rubric. Rubrics lack (1) the contextual information needed to convey real-world attributes (2) the dynamic information (such as gesturing in the complex skill of presenting) (3) the procedural information required to support the automation of constituent skills. We propose to address the text-based rubric's deficiencies by adding video-modeling examples, self-explanation prompts, an intertwined educational- and instructional narrative, natural segmentation, and a non-verbal script. With the resulting Video Enhanced Rubric, we aim to improve the formative assessment of complex skills by fostering learner's mental model development, feedback quality, and complex skill mastery. Designing multimedia to support the formative assessment of complex skills using can cause dilemmas for instructional designers. For example, is learner control needed to foster intrinsic motivation or is it deemed to create extraneous cognitive load? Is it wise to use a video modeling example of peer-aged learners when the model does not display perfect performance? We found seven dilemmas around proven complex skill development, motivational design, and multimedia design guidelines. This paper presents a theoretical contribution to instructional design by introducing a framework to address dilemmas regarding such design dilemmas. As a practical contribution, we support educational researchers and practitioners by presenting six practical guidelines for designing a Video Enhanced Rubric. A use case of the Viewbrics-project provides insight into the practical application of the framework within the context of Dutch pre-university education.