Search

Simulation-based learning provides students with unique opportunities to develop key procedural and teamwork skills in close-to-authentic physical learning and training environments. Yet, assessing students' performance in such situations can be challenging and mentally exhausting for teachers. Multimodal learning analytics can support the assessment of simulation-based learning by making salient aspects of students' activities visible for evaluation. Although descriptive analytics have been used to study students' motor behaviours in simulation-based learning, their validity and utility for assessing performance remain unclear. This study aims at addressing this knowledge gap by investigating how indoor positioning analytics can be used to generate meaningful insights about students' tasks and collaboration performance in simulation-based learning. We collected and analysed the positioning data of 304 healthcare students, organised in 76 teams, through correlation, predictive and epistemic network analyses. The primary findings were (1) large correlations between students' spatial-procedural behaviours and their group performances; (2) predictive learning analytics that achieved an acceptable level (0.74 AUC) in distinguishing between low-performing and high-performing teams regarding collaboration performance; and (3) epistemic networks that can be used for assessing the behavioural differences across multiple teams. We also present the teachers' qualitative evaluation of the utility of these analytics and implications for supporting formative assessment in simulation-based learning.

Identifying students facing difficulties and providing them with timely support is one of the educator's key responsibilities. Yet, this task is becoming increasingly challenging as the complexity of physical learning spaces grows, along with the emergence of novel educational technologies and classroom designs. There has been substantial research and development work focused on identifying student social behaviours in digital platforms (eg, the learning management system) as predictors of academic progression. However, little work has investigated such relationships in physical learning spaces. This study explores the potential of using wearable trackers for the early detection of low-progress students based on their social and spatial (socio-spatial) behaviours at the school. Positioning data from 98 primary school students and six teachers were automatically captured over a period of eight weeks. Fourteen socio-spatial behavioural features were extracted and processed using a set of machine learning classifiers to model students' learning progression. Results illustrate the potential of prospectively identifying low-progress students from these features and the importance of adapting classroom learning analytics to differences in pedagogical designs.

Online learning is currently adopted by educational institutions worldwide to provide students with ongoing education during the COVID-19 pandemic. Even though online learning research has been advancing in uncovering student experiences in various settings (i.e., tertiary, adult, and professional education), very little progress has been achieved in understanding the experience of the K-12 student population, especially when narrowed down to different school-year segments (i.e., primary and secondary school students). This study explores how students at different stages of their K-12 education reacted to the mandatory full-time online learning during the COVID-19 pandemic. For this purpose, we conducted a province-wide survey study in which the online learning experience of 1,170,769 Chinese students was collected from the Guangdong Province of China. We performed cross-tabulation and Chi-square analysis to compare students' online learning conditions, experiences, and expectations. Results from this survey study provide evidence that students' online learning experiences are significantly different across school years. Foremost, policy implications were made to advise government authorises and schools on improving the delivery of online learning, and potential directions were identified for future research into K-12 online learning.

Multimodal learning analytics (MMLA) offers the potential to provide evidence‐based insights into complex learning phenomena such as collaborative learning. Yet, few MMLA applications have closed the learning analytics loop by being evaluated in real‐world educational settings. This study evaluates the effectiveness of an MMLA solution in enhancing feedback and reflection within a complex and highly dynamic collaborative learning environment. A two‐year longitudinal study was conducted with 399 students and 17 teachers, utilising an MMLA system in reflective debriefings in the context of healthcare education. We analysed the survey data of 74 students and 11 teachers regarding their perceptions of the MMLA system. We applied the Evaluation Framework for Learning Analytics, augmented by complexity, accuracy and trust measures, to assess both teachers' and students' perspectives. The findings illustrated that teachers and students both had generally positive perceptions of the MMLA solution. Teachers found the MMLA solution helpful in facilitating feedback provision and reflection during debriefing sessions. Similarly, students found the MMLA solution effective in providing clarity on the data collected, stimulating reflection on their learning behaviours, and prompting considerations for adaptation in their learning behaviours. However, the complexity of the MMLA solution and the need for qualitative measures of communication emerged as areas for improvement. Additionally, the study highlighted the importance of data accuracy, transparency, and privacy protection to maintain user trust. The findings provide valuable contributions to advancing our understanding of the use of MMLA in supporting feedback and reflection practices in intricate collaborative learning while identifying avenues for further research and improvement. We also provided several insights and practical recommendations for successful MMLA implementation in authentic learning contexts. Practitioner notesWhat is currently known about this topic Multimodal learning analytics (MMLA) seeks to generate data‐informed insights about learners' metacognitive and emotional states as well as their learning behaviours, by utilising intricate physical and physiological signals.MMLA has not only pioneered novel data analytic methods but also aspired to complete the learning analytics loop by crafting innovative, tangible solutions that relay these insights to the concerned stakeholders.A prominent direction within MMLA research has been the formulation of tools to support feedback and reflection in collaborative learning scenarios, given MMLA's capacity to discern intricate and dynamic learning behaviours.What this paper adds Teachers' and students' positive perceptions of an MMLA implementation in stimulating considerations of adaptations in their pedagogical practices and learning behaviours, respectively.Empirical evidence supporting the potential of MMLA in assisting teachers to facilitate students' reflective practices during intricate collaborative learning scenarios.The importance of addressing issues related to design complexity, interpretability for users with disabilities, aggregated data representation, and concerns related to trust for building a practical MMLA solution in real learning settings.Implications for practice and/or policy The MMLA solution can provide teachers with a comprehensive view of student performance, illuminate areas for improvement, and confirm learning scenario outcomes.The MMLA solution can stimulate students' reflections on their learning behaviours and promote considerations of adaptation in their learning behaviours.Providing clear explanations and guidance on how to interpret analytics, as well as addressing concerns related to data completeness and representation, are essential to maximising utility. [ABSTRACT FROM AUTHOR]

Background: Learner profiles detected from digital trace data are typically triangulated with survey data to explain those profiles based on learners' internal conditions (e.g., motivation). However, survey data are often analysed with limited consideration of the interconnected nature of learners' internal conditions. Objectives: Aiming to enable a thorough understanding of trace‐based learner profiles, this paper presents and evaluates a comprehensive approach to analysis of learners' self‐reports, which extends conventional statistical methods with psychological networks analysis. Methods: The study context is a massive open online course (MOOC) aimed at promoting physical activity (PA) for health. Learners' (N = 497) perceptions related to PA, as well as their self‐efficacy and intentions to increase the level of PA were collected before and after the MOOC, while their interactions with the course were logged as digital traces. Learner profiles derived from trace data were further examined and interpreted through a combined use of conventional statistical methods and psychological networks analysis. Results and Conclusions: The inclusion of psychological networks in the analysis of learners' self‐reports collected before the start of the MOOC offers better understanding of trace‐based learner profiles, compared to the conventional statistical analysis only. Likewise, the combined use of conventional statistical methods and psychological networks in the analysis of learners' self‐reports before and after the MOOC provided more comprehensive insights about changes in the constructs measured in each learner profile. Major Takeaways: The combined use of conventional statistical methods and psychological networks presented in this paper sets a path for a comprehensive analysis of survey data. The insights it offers complement the information about learner profiles derived from trace data, thus allowing for a more thorough understanding of learners' course engagement than any individual method or data source would allow. Lay Description: What is already known about this topic: Researchers have made extensive use of data from online learning platforms—often referred to as learning trace data—to understand how learners engage in online learning activities.Surveys are often used to gather information about learners' motivation, perceptions, and other internal factors, to complement the insight gleaned from learning traces and thus create a more complete picture of the learning behaviour.When analysing survey data, the focus is usually on individual factors without considering how different factors are connected and affect one another.Psychological networks analysis is a novel analytic approach to studying complex phenomena both in psychology and education. What this paper adds: The paper uses psychological networks as a method for examining mutual relations among learners' internal factors (e.g., motivation, perceptions) measured through self‐reports before and after a course.By combining conventional statistical analysis of self‐reports with psychological networks analysis, the paper develops a comprehensive picture of learners' internal factors measured through self‐reports, one that accounts for both individual and interconnected nature of those factors.This comprehensive approach to information extraction from surveys allows for better understanding of trace‐based learner profiles, that is, profiles mined from data about learners' interaction with online learning activities. Implications for practice and/or policy: The presented method paves a way for a comprehensive analysis of the data collected from learners through surveys.It helps us gain a better understanding of learners' engagement with online learning activities.It can advance the evaluation of digital educational programs that aim to encourage changes in health‐related behaviour. [ABSTRACT FROM AUTHOR]

Effective collaboration and teamwork skills are critical in high‐risk sectors, as deficiencies in these areas can result in injuries and risk of death. To foster the growth of these vital skills, immersive learning spaces have been created to simulate real‐world scenarios, enabling students to safely improve their teamwork abilities. In such learning environments, multiple dialogue segments can occur concurrently as students independently organise themselves to tackle tasks in parallel across diverse spatial locations. This complex situation creates challenges for educators in assessing teamwork and for students in reflecting on their performance, especially considering the importance of effective communication in embodied teamwork. To address this, we propose an automated approach for generating teamwork analytics based on spatial and speech data. We illustrate this approach within a dynamic, immersive healthcare learning environment centred on embodied teamwork. Moreover, we evaluated whether the automated approach can produce transcriptions and epistemic networks of spatially distributed dialogue segments with a quality comparable to those generated manually for research objectives. This paper makes two key contributions: (1) it proposes an approach that integrates automated speech recognition and natural language processing techniques to automate the transcription and coding of team communication and generate analytics; and (2) it provides analyses of the errors in outputs generated by those techniques, offering insights for researchers and practitioners involved in the design of similar systems.Practitioner notesWhat is currently known about this topic Immersive learning environments simulate real‐world situations, helping students improve their teamwork skills.In these settings, students can have multiple simultaneous conversations while working together on tasks at different physical locations.The dynamic nature of these interactions makes it hard for teachers to assess teamwork and communication and for students to reflect on their performance.What this paper adds We propose a method that employs multimodal learning analytics for automatically generating teamwork‐related insights into the content of student conversations.This data processing method allows for automatically transcribing and coding spatially distributed dialogue segments generated from students working in teams in an immersive learning environment and enables downstream analysis.This approach uses spatial analytics, natural language processing and automated speech recognition techniques.Implications for practitioners Automated coding of dialogue segments among team members can help create analytical tools to assist in evaluating and reflecting on teamwork.By analysing spatial and speech data, it is possible to apply learning analytics advancements to support teaching and learning in fast‐paced physical learning spaces where students can freely engage with one another. [ABSTRACT FROM AUTHOR]

Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (eg, question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs‐based innovations in authentic educational contexts. To address this, we conducted a systematic scoping review of 118 peer‐reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The findings revealed 53 use cases for LLMs in automating education tasks, categorised into nine main categories: profiling/labelling, detection, grading, teaching support, prediction, knowledge representation, feedback, content generation, and recommendation. Additionally, we also identified several practical and ethical challenges, including low technological readiness, lack of replicability and transparency and insufficient privacy and beneficence considerations. The findings were summarised into three recommendations for future studies, including updating existing innovations with state‐of‐the‐art models (eg, GPT‐3/4), embracing the initiative of open‐sourcing models/systems, and adopting a human‐centred approach throughout the developmental process. As the intersection of AI and education is continuously evolving, the findings of this study can serve as an essential reference point for researchers, allowing them to leverage the strengths, learn from the limitations, and uncover potential research opportunities enabled by ChatGPT and other generative AI models. Practitioner notesWhat is currently known about this topic Generating and analysing text‐based content are time‐consuming and laborious tasks.Large language models are capable of efficiently analysing an unprecedented amount of textual content and completing complex natural language processing and generation tasks.Large language models have been increasingly used to develop educational technologies that aim to automate the generation and analysis of textual content, such as automated question generation and essay scoring.What this paper adds A comprehensive list of different educational tasks that could potentially benefit from LLMs‐based innovations through automation.A structured assessment of the practicality and ethicality of existing LLMs‐based innovations from seven important aspects using established frameworks.Three recommendations that could potentially support future studies to develop LLMs‐based innovations that are practical and ethical to implement in authentic educational contexts.Implications for practice and/or policy Updating existing innovations with state‐of‐the‐art models may further reduce the amount of manual effort required for adapting existing models to different educational tasks.The reporting standards of empirical research that aims to develop educational technologies using large language models need to be improved.Adopting a human‐centred approach throughout the developmental process could contribute to resolving the practical and ethical challenges of large language models in education. [ABSTRACT FROM AUTHOR]