Search

Background: Learning Analytics (LA) is an emerging field concerned with measuring, collecting, and analysing data about learners and their contexts to gain insights into learning processes. As the technology of Learning Analytics is evolving, many systems are being implemented. In this context, it is essential to understand stakeholders' expectations of LA across Higher Education Institutions (HEIs) for large-scale implementations that take their needs into account. Objectives: This study aims to contribute to knowledge about individual LA expectations of European higher education students. It may facilitate the strategy of stakeholder buy-in, the transfer of LA insights across HEIs, and the development of international best practices and guidelines. Methods: To this end, the study employs a 'Student Expectations of Learning Analytics Questionnaire' (SELAQ) survey of 417 students at the Goethe University Frankfurt (Germany) Based on this data, Multiple Linear Regressions are applied to determine how these students position themselves compared to students from Madrid (Spain), Edinburgh (United Kingdom) and the Netherlands, where SELAQ had already been implemented at HEIs. Results and Conclusions: The results show that students' expectations at Goethe University Frankfurt are rather homogeneous regarding 'LA Ethics and Privacy' and 'LA Service Features'. Furthermore, we found that European students generally show a consistent pattern of expectations of LA with a high degree of similarity across the HEIs examined. European HEIs face challenges more similar than anticipated. The HEI experience with implementing LA can be more easily transferred to other HEIs, suggesting standardized LA rather than tailor-made solutions designed from scratch.

Background: Formative assessments are needed to enable monitoring how student knowledge develops throughout a unit. Constructed response items which require learners to formulate their own free-text responses are well suited for testing their active knowledge. However, assessing such constructed responses in an automated fashion is a complex task and requires the application of natural language processing methodology. In this article, we implement and evaluate multiple machine learning models for coding energy knowledge in free-text responses of German K-12 students to items in formative science assessments which were conducted during synchronous online learning sessions. Dataset: The dataset we collected for this purpose consists of German constructed responses from 38 different items dealing with aspects of energy such as manifestation and transformation. The units and items were implemented with the help of project-based pedagogy and evidence-centered design, and the responses were coded for seven core ideas concerning the manifestation and transformation of energy. The data was collected from students in seventh, eighth and ninth grade. Methodology: We train various transformer- and feature-based models and compare their ability to recognize the respective ideas in students' writing. Moreover, as domain knowledge and its development can be formally modeled through knowledge networks, we evaluate how well the detection of the ideas within responses translated into accurate co-occurrence-based knowledge networks. Finally, in terms of the descriptive accuracy of our models, we inspect what features played a role for which prediction outcome and if the models pick up on undesired shortcuts. In addition to this, we analyze how much the models match human coders in what evidence within responses they consider important for their coding decisions. Results: A model based on a modified GBERT-large can achieve the overall most promising results, although descriptive accuracy varies much more than predictive accuracy for the different ideas assessed. For reasons of comparability, we also evaluate the same machine learning architecture using the SciEntsBank 3-Way benchmark with an English RoBERTa-large model, where it achieves state-of-the-art results in two out of three evaluation categories.

This paper describes the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects training mistakes using recurrent neural networks. The CPR Tutor automatically recognises and assesses the quality of the chest compressions according to five CPR performance indicators. It detects training mistakes in real-time by analysing a multimodal data stream consisting of kinematic and electromyographic data. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. The mistake detection models of the CPR Tutor were trained using a dataset from 10 experts. Hence, we tested the validity of the CPR Tutor and the impact of its feedback functionality in a user study involving additional 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: (1) an architecture design, (2) a methodological approach for delivering real-time feedback using multimodal data and (3) a field study on real-time feedback for CPR training. This paper details the results of a field study by quantitatively measuring the impact of the CPR Tutor feedback on the performance indicators and qualitatively analysing the participants' questionnaire answers.

Training to run is not straightforward since without proper personalised supervision and planning, people will not improve their performance and will increase the risk of injuries. This study aims to identify the different factors that influence running training programmes, examining the benefits, challenges or limitations of personalised plans. Moreover, this study explores how multimodal, immersive and artificial intelligence technologies can support personalised training. We conducted an exploratory sequential mixed research consisting of interviews with 11 running coaches from different countries and a survey of 12 running coaches. Based on the interviews and survey analysis, we identified and extracted relevant factors of the training process. We identified four relevant aspects for running training: physical, technical, mental and body awareness. Using these aspects as a reference, we derived a framework using a bottom‐up approach. This framework proposes multimodal, immersive and artificial intelligence technologies to facilitate personalised running training. It also allows coaches to personally guide their athletes on each aspect. Practitioner notesWhat is already known about this topic Running is a popular sport that provides health benefits and is practised by many people around the world.Training is a process that enables athletes to improve their development in various aspects of their sport; in the case of running, it helps them to increase their speed and endurance.Personalised training supports the needs and abilities of athletes, by helping them to achieve their potential through individualised activities or programmes.Sports science research indicates that personalised training can be improved by applying technology to tackle its challenges and limitations.What this paper adds We show that personalising the training requires not only focusing on the runners' physical condition but also on their mental, technical and body awareness aspects, where each of them has a different adaptation to training.We show that multimodal and immersive technologies offer suitable and portable ways to measure and target the mental and body awareness aspects during running training.Implications for practice and/or policy This paper presents a list of factors, measures and devices that coaches can use to plan and design their training sessions in a more personalised manner.This study can serve as a foundation for future research that aims to identify and target the various factors that influence the learning and training of sports. [ABSTRACT FROM AUTHOR]

Multimodality in learning analytics and learning science is under the spotlight. The landscape of sensors and wearable trackers that can be used for learning support is evolving rapidly, as well as data collection and analysis methods. Multimodal data can now be collected and processed in real time at an unprecedented scale. With sensors, it is possible to capture observable events of the learning process such as learner's behaviour and the learning context. The learning process, however, consists also of latent attributes, such as the learner's cognitions or emotions. These attributes are unobservable to sensors and need to be elicited by human-driven interpretations. We conducted a literature survey of experiments using multimodal data to frame the young research field of multimodal learning analytics. The survey explored the multimodal data used in related studies (the input space) and the learning theories selected (the hypothesis space). The survey led to the formulation of the Multimodal Learning Analytics Model whose main objectives are of (O1) mapping the use of multimodal data to enhance the feedback in a learning context; (O2) showing how to combine machine learning with multimodal data; and (O3) aligning the terminology used in the field of machine learning and learning science.

Background: Formative assessments are needed to enable monitoring how student knowledge develops throughout a unit. Constructed response items which require learners to formulate their own free‐text responses are well suited for testing their active knowledge. However, assessing such constructed responses in an automated fashion is a complex task and requires the application of natural language processing methodology. In this article, we implement and evaluate multiple machine learning models for coding energy knowledge in free‐text responses of German K‐12 students to items in formative science assessments which were conducted during synchronous online learning sessions. Dataset: The dataset we collected for this purpose consists of German constructed responses from 38 different items dealing with aspects of energy such as manifestation and transformation. The units and items were implemented with the help of project‐based pedagogy and evidence‐centered design, and the responses were coded for seven core ideas concerning the manifestation and transformation of energy. The data was collected from students in seventh, eighth and ninth grade. Methodology: We train various transformer‐ and feature‐based models and compare their ability to recognize the respective ideas in students' writing. Moreover, as domain knowledge and its development can be formally modeled through knowledge networks, we evaluate how well the detection of the ideas within responses translated into accurate co‐occurrence‐based knowledge networks. Finally, in terms of the descriptive accuracy of our models, we inspect what features played a role for which prediction outcome and if the models pick up on undesired shortcuts. In addition to this, we analyze how much the models match human coders in what evidence within responses they consider important for their coding decisions. Results: A model based on a modified GBERT‐large can achieve the overall most promising results, although descriptive accuracy varies much more than predictive accuracy for the different ideas assessed. For reasons of comparability, we also evaluate the same machine learning architecture using the SciEntsBank 3‐Way benchmark with an English RoBERTa‐large model, where it achieves state‐of‐the‐art results in two out of three evaluation categories. Lay Description: What is already known about this topic?: Formative assessments are needed to test and monitor the development of learners' knowledge throughout a unit to provide them with appropriate automated feedback.Constructed response items which require learners to formulate their own free‐text responses are well suited for testing their active knowledge.Assessing constructed responses in an automated fashion is a widely researched topic, but the problem is far from solved and most of the work focuses on predicting holistic scores or grades.To allow for a more fine‐grained and analytic assessment of learners' knowledge, systems which go beyond predicting simple grades are required.To guarantee that models are stable and make their predictions for the correct reasons, methods for explaining the models are required. What this papers adds?: A core topic in physics education is the concept of energy.We implement and evaluate multiple systems based on natural language processing technology for assessing learners' conceptual knowledge about energy physics using transformer language models as well as feature‐based approaches.The systems assess students' knowledge about various forms of energy, indicators for the same and the transformation of energy from form into another.As our systems are based on machine learning methodology, we introduce a novel German short answer dataset for training them to detect the respective knowledge elements within students' free‐text responses.We evaluate the performance of these systems using this dataset as well as the well‐established SciEntsBand‐3‐Way dataset and manage to achieve, to our best knowledge, new state‐of‐the‐art results for the latter.Moreover, we apply methodology for explaining model predictions to assess whether predictions are carried out for the correct reasons. Implications for practice and/or policy: It is indeed possible to assess constructed responses for the demonstrated knowledge about energy physics in an analytic fashion using natural language processing.Transformer language models can outperform more specialized feature‐based approaches for this task in terms of predictive and descriptive accuracy.Co‐occurrences of different concepts within the same responses can lead models to learn undesired shortcuts which make them unstable. [ABSTRACT FROM AUTHOR]