Search

High-quality elaborative peer feedback is a blessing for both learners and teachers. However, learners can experience difficulties in giving high-quality feedback on complex skills using textual analytic rubrics. High-quality elaborative feedback can be strengthened by adding video-modeling examples with embedded self-explanation prompts, turning textual analytic rubrics (TR) into so-called 'video-enhanced analytic rubrics' (VER). This study contrasts two experimental conditions (TR, n = 54; VERs, n = 49) with their version of the anonymized online tool (used to collect the given feedback in 'Tips for improvement and Tops identifying strengths'). Peer feedback quality (concreteness and consistency) was evaluated using Natural Language Processing. As expected, the video-enhanced rubrics condition resulted in a higher quantity of words used and a lower amount of naive wording compared to the textual rubric condition. Contrary to our assumptions, it did not lower the amount of non-constructive wording nor improved the amount of behavioral and process-related feedback. Possibly, the transition from providing more feedback to delivering more accurate behavioral and process-related feedback has not yet been made in the time set for the study.

Contribution: This article presents a comprehensive overview of characteristics of educational designs of collaborative engineering design activities found in literature and how these characteristics mediate students' collaboration. Background: Engineers have to solve complex problems that require collaboration. In education, various collaborative engineering design activities have been implemented to prepare students for these professional practices. According to cultural historical activity theory (CHAT), educational activities can be described in terms of interrelated elements, i.e., subject, object, tools, rules, division of labor, and community, that influence learning outcomes. A key issue is how these elements mediate students' collaborative efforts and how they contribute to learning. Research Questions: 1) How is collaborative learning implemented in engineering design education? 2) How do the elements of CHAT and their interrelations mediate collaborative learning? and 3) What is the evidence that the implementation of collaborative learning contributed to the achievement of desired learning outcomes? Methodology: A systematic literature review following preferred reporting items for systematic review and meta-analyses protocols guidelines was conducted, including 111 articles published between 2011 and 2021. CHAT was used as analytical framework. Findings: Collaborative learning was implemented in engineering design activities to develop technical as well as nontechnical skills. For the CHAT elements, it was found that establishing a common object, rules for collaboration, and division of labor are essential for effective collaboration and can be enhanced through digital technologies (tools) and support from a community, for example, educators. Finally, results showed that there is evidence that described implementations contribute to learning. However, this evidence needs to be interpreted with care, due to methodological issues in some included articles.

This article illustrates that the task design and problem selection are of characteristic influence to evoke sense-reason-act programming (SRA) among primary school pupils when programming robots. Research shows that the task design influences the development of computational thinking (CT). The literature provides evidence that the context, the problem space, and the representation of the problem to apply SRA-programming require the programming task to be embedded in a dynamic context in which a programmable robot must use sensory information to anticipate changes in the environment. In order to ascertain whether the problem space and the task design influence the evocation of SRA-thinking, this article uses a research design comparing the differences between two programming conditions (static/dynamic). In these conditions, pupils use Lego EV-3 robots and Mindstorms software to solve programming problems. As a post-measurement, a Lego challenge is applied. In this article, it is shown that the integration of a dynamic task design to solve a programming problem is essential for a deeper understanding of CT skills. Furthermore, when pupils can immediately test the consequences of their program in a dynamic environment and, thus, the learning environment provides an appropriate problem, they gain a deeper understanding of the added value of sensors and will be better able to reason about complex problems. It is found that programming in a dynamic problem environment almost naturally evokes SRA-thinking, as opposed to programming in a static environment. The influence of SRA-programming as demonstrated identifies characteristics of CT.

Visual programming environments are popular instruments in teaching Computational Thinking (CT) in schools today. Applying Sense-Reason-Act (SRA) programming can influence the development of computational thinking when forcing pupils to anticipate the unforeseen in their computer programs. SRA-programming originates from the programming of tangible robots, but can also be of equal value in visual programming with on-screen output. The underlying rationale is that programming in a visual programming environment using SRA leads to more understanding of the computational concepts addressed, resulting in a higher level of computational skill compared to visual programming without the application of SRA. Furthermore, it has been hypothesised that if pupils in a visual programming environment can anticipate unforeseen events and solve programming tasks by applying SRA, they will be better able to solve complex computational thinking tasks. To establish if characteristic differences in the development of computational thinking can be measured when SRA-programming is applied in a visual programming environment with an on-screen output, we assessed the applicability of SRA-programming with visual output as the main component of the execution of developed code. This research uses a pre-test post-test design that reveals significant differences in the development of computational thinking in two treatment conditions. To assess CT, the Computational Thinking Test (CTt) was used. Results show that when using SRA-programming in a visual programming environment it leads to an increased understanding of complex computational concepts, which results in a significant increase in the development of computational thinking.

Learners in the process of developing complex skills need a rich mental model of what such skills entail. Textual analytics rubrics (TR) are a widely used instrument to support formative assessment of complex skills, supporting feedback, reflection, and thus mental model development of complex skills. However, the textual nature of a rubric limits the possibility to deliver contextual and dynamic (time-and behaviour-oriented) information. In the Viewbrics online tool, we developed a version supporting the delivery of contextual and dynamic information by adding video-modelling examples with embedded self-explanation prompts to textual analytics rubrics. We called this combination as video-enhanced rubrics (VERs). Our current study investigates whether the Viewbrics online tool supports complex skills development and whether either textual- or video-enhanced rubrics best support complex skills' mastery. The study was a three-group (VERS n = 49, TR n = 54, control n = 50) within-subjects design. Learners' performance of complex skills was measured through expert, peer, and self-assessment using the Viewbrics online tool. A multilevel regression analysis shows learners in the TR and VERS conditions consistently outperforming the control condition to varying degrees across skills. However, no differences have been found between the two experimental conditions. Positive results across different complex skills indicate the Viewbrics online tool can be used to support the development of a wide range of complex skills in secondary education.

Collaboration is one of the important 21st-century skills. It can take place in remote or co-located settings. Co-located collaboration (CC) is a very complex process that involves subtle human interactions that can be described with indicators like eye gaze, speaking time, pitch, and social skills from different modalities. With the advent of sensors, multimodal learning analytics has gained momentum to detect CC quality. Indicators (or low-level events) can be used to detect CC quality with the help of measurable markers (i.e., indexes composed of one or more indicators) which give the high-level collaboration process definition. However, this understanding is incomplete without considering the scenarios (such as problem solving or meetings) of CC. The scenario of CC affects the set of indicators considered: For instance, in collaborative programming, grabbing the mouse from the partner is an indicator of collaboration; whereas in collaborative meetings, eye gaze, and audio level are indicators of collaboration. This can be a result of the differing goals and fundamental parameters (such as group behavior, interaction, or composition) in each scenario. In this article, we present our work on profiles of indicators on the basis of a scenario-driven prioritization, the parameters in different CC scenarios are mapped onto the indicators and the available indexes. This defines the conceptual model to support the design of a CC quality detection and prediction system.

This study investigates the development of algorithmic thinking as a part of computational thinking skills and self-efficacy of primary school pupils using programmable robots in different instruction variants. Computational thinking is defined in the context of twenty-first century skills and describes processes involved in (re)formulating a problem in a way that a computer can process it. Programming robots offers specific affordances as it can be used to develop programs following a Sense-Reason-Act (SRA) cycle. The literature provides evidence that programming robots has the potential to enhance algorithmic thinking as a component of computational thinking. Specifically there are indications that pupils who use SRA-programming learn algorithmic skills better and achieve a higher level of self-efficacy in an open, scaffold learning environment than through direct instruction. In order to determine the influence of the instruction variant used, an experimental research design was made in which pupils solved algorithm-based mathematical problems (grid diagrams) in a preliminary measurement and their self-efficacy determined via a questionnaire. As an intervention, pupils learn to solve programming issues in pairs using "Lego NXT" robots and "Mindstorms" software in two instruction variants. The post-measurement consists of a Lego challenge, solving mathematical problems (grid diagrams), and a repeated self-efficacy questionnaire. This research shows an increase of our measures on algorithmic thinking dependent on the amount of SRA usage (though not significant). Programming using the SRA-cycle can be considered as the cause of the measured effect. The instruction variant used during the robotic intervention seems to play only a marginal role.

The increasingly digital landscape of higher education has highlighted the importance of self-regulated learning in digital learning environments. To support this, academic goal setting is frequently used to enhance self-regulated learning in order to improve academic performance. Although many studies have explored the implementation of goal setting activities as behavioral modifiers, the implementation of goal setting across these studies is varied, and there is little consensus on the components which should be included and reported when studying goal setting activities. To provide an overview of the current state of the field, a systematic review was carried out examining studies which implemented academic goal setting activities within higher education over the last 14 years (2010–2024) to determine for whom, in what contexts, and how goal setting has been implemented. The results from the 60 included studies reveal a wide array of goal setting implementations covering many countries and academic disciplines. Overall, these implementations are highly heterogeneous, with large differences between studies in how goal setting is carried out. However, results also show a strong trend toward partial digitalization, with most studies using technology to deliver their goal setting activities, but very few adopting technologies for any further enhancements or support. Overall, the review reveals a focus on non-experimental studies exploring the content of student goals, with only a small selection testing the effect of goal setting in experimental studies. Based on these results we suggest future work focuses on testing the effect of goal setting, especially focusing on the interplay between the design of the activities and individual student needs, as well as further investigation of how emerging educational technologies can be used to scale and enhance goal setting activities. [ABSTRACT FROM AUTHOR]

For learners, it can be difficult to imagine how to perform a complex skill based on textual information from solely a text-based analytic rubric. Rubrics lack (1) the contextual information needed to convey real-world attributes (2) the dynamic information (such as gesturing in the complex skill of presenting) (3) the procedural information required to support the automation of constituent skills. We propose to address the text-based rubric's deficiencies by adding video-modeling examples, self-explanation prompts, an intertwined educational- and instructional narrative, natural segmentation, and a non-verbal script. With the resulting Video Enhanced Rubric, we aim to improve the formative assessment of complex skills by fostering learner's mental model development, feedback quality, and complex skill mastery. Designing multimedia to support the formative assessment of complex skills using can cause dilemmas for instructional designers. For example, is learner control needed to foster intrinsic motivation or is it deemed to create extraneous cognitive load? Is it wise to use a video modeling example of peer-aged learners when the model does not display perfect performance? We found seven dilemmas around proven complex skill development, motivational design, and multimedia design guidelines. This paper presents a theoretical contribution to instructional design by introducing a framework to address dilemmas regarding such design dilemmas. As a practical contribution, we support educational researchers and practitioners by presenting six practical guidelines for designing a Video Enhanced Rubric. A use case of the Viewbrics-project provides insight into the practical application of the framework within the context of Dutch pre-university education.

Multimodality in learning analytics and learning science is under the spotlight. The landscape of sensors and wearable trackers that can be used for learning support is evolving rapidly, as well as data collection and analysis methods. Multimodal data can now be collected and processed in real time at an unprecedented scale. With sensors, it is possible to capture observable events of the learning process such as learner's behaviour and the learning context. The learning process, however, consists also of latent attributes, such as the learner's cognitions or emotions. These attributes are unobservable to sensors and need to be elicited by human-driven interpretations. We conducted a literature survey of experiments using multimodal data to frame the young research field of multimodal learning analytics. The survey explored the multimodal data used in related studies (the input space) and the learning theories selected (the hypothesis space). The survey led to the formulation of the Multimodal Learning Analytics Model whose main objectives are of (O1) mapping the use of multimodal data to enhance the feedback in a learning context; (O2) showing how to combine machine learning with multimodal data; and (O3) aligning the terminology used in the field of machine learning and learning science.

In today's technological era, emerging educational technologies, such as digital badges, have shown the potential for fostering student learning. To examine the major considerations undergirding the design of digital badges used in an educational context, a conceptual literature review of multidisciplinary electronic databases oriented towards computing and education was undertaken. The three core design dimensions found in our analysis were the specific function of badges, the structure of badge systems, and the different types of design and interaction features used with badges. We highlight the main effects of studies addressing these dimensions and provide recommendations and suggestions for further research to educational stakeholders who seek to better understand how to implement badges within their own institutions.