Search

Technology‐based, open‐ended learning environments (OELEs) can capture detailed information of students' interactions as they work through a task or solve a problem embedded in the environment. This information, in the form of log data, has the potential to provide important insights about the practices adopted by students for scientific inquiry and problem solving. How to parse and analyse the log data to reveal evidence of multifaceted constructs like inquiry and problem solving holds the key to making interactive learning environments useful for assessing students' higher‐order competencies. In this paper, we present a systematic review of studies that used log data generated in OELEs to describe, model and assess scientific inquiry and problem solving. We identify and analyse 70 conference proceedings and journal papers published between 2012 and 2021. Our results reveal large variations in OELE and task characteristics, approaches used to extract features from log data and interpretation models used to link features to target constructs. While the educational data mining and learning analytics communities have made progress in leveraging log data to model inquiry and problem solving, multiple barriers still exist to hamper the production of representative, reproducible and generalizable results. Based on the trends identified, we lay out a set of recommendations pertaining to key aspects of the workflow that we believe will help the field develop more systematic approaches to designing and using OELEs for studying how students engage in inquiry and problem‐solving practices. Practitioner notesWhat is already known about this topic Research has shown that technology‐based, open‐ended learning environments (OELEs) that collect users' interaction data are potentially useful tools for engaging students in practice‐based STEM learning.More work is needed to identify generalizable principles of how to design OELE tasks to support student learning and how to analyse the log data to assess student performance.What this paper adds We identified multiple barriers to the production of sufficiently generalizable and robust results to inform practice, with respect to: (1) the design characteristics of the OELE‐based tasks, (2) the target competencies measured, (3) the approaches and techniques used to extract features from log files and (4) the models used to link features to the competencies.Based on this analysis, we can provide a series of specific recommendations to inform future research and facilitate the generalizability and interpretability of results: Making the data available in open‐access repositories, similar to the PISA tasks, for easy access and sharing.Defining target practices more precisely to better align task design with target practices and to facilitate between‐study comparisons.More systematic evaluation of OELE and task designs to improve the psychometric properties of OELE‐based measurement tasks and analysis processes.Focusing more on internal and external validation of both feature generation processes and statistical models, for example with data from different samples or by systematically varying the analysis methods.Implications for practice and/or policy Using the framework of evidence‐centered assessment design, we have identified relevant criteria for organizing and evaluating the diverse body of empirical studies on the topic and that policy makers and practitioners can use for their own further examinations.This paper identifies promising research and development areas on the measurement and assessment of higher‐order constructs with process data from OELE‐based tasks that government agencies and foundations can support.Researchers, technologists and assessment designers might find useful the insights and recommendations for how OELEs can enhance science assessment through thoughtful integration of learning theories, task design and data mining techniques. [ABSTRACT FROM AUTHOR]

This paper presents an implementation of Connected Spaces (CxS)—an ambient help seeking interface designed and developed for a project‐based computing classroom. We use actor network theory (ANT) to provide an underutilized posthumanist lens to understand the creation of collaborative connections in this Computational Action‐based implementation. Posthumanism offers an emerging and critical extension to sociocultural perspectives on understanding learning, by pushing us to decenter the human, and consider the active roles that human and non‐human entities play in learning environments by actively shaping each other. We analyse how students in this class adjusted their help‐seeking and collaborative habits following the introduction of CxS, a tool designed to foster (more inter‐group) collaboration. ANT proposes generalized symmetry—a principle of considering human, non‐human and more than human entities with equivalent and comparable agency, leading to describing phenomena as networks of actors in different evolving relationships with each other. Analysing collaborative interactions as fostered by CxS using an ANT approach supports design‐based research—an iterative design revision process highlighting understandings about design as well as learning—by providing a temporal and informative lens into the relationship between actors and tools within the environment. Our key findings include a framing of technologies in classrooms as bridging agentic gaps between students and becoming actors engaging in different behaviours; learners enacting new agencies through technologies (for instance a more comfortable non‐intrusive help seeker), and the need for voicing and teachers to connect help networks in CxS equipped classrooms. Practitioner notesWhat is already known about this topic Collaborative learning is a valuable skill and practice; opportunities to mentor others are critical in empowering minoritized learners, especially in STEM and computing disciplines.School norms solidify a power and expertise hierarchy between teachers and learners and fail to productively support learners in learning from each other.Additionally, lack of awareness about peers' knowledge is a common hindrance in students knowing who to ask for help and how.What this paper adds An example of a designed interface called Connected Spaces with potential to foster more inter‐student collaboration, especially outside of mandated within‐group collaboration—in the form of cross‐group help seeking and help giving.A design based research study using actor network theory highlighting the limitations of Connected Spaces in sparking notable behaviour change among students by itself but being retooled as a teacher support tool in enabling cross‐group collaborations.Presenting conceptions of collaboration through technologies as bridging agentic gaps and acting with new agencies in performing help‐seeking related actions.Provoking the idea of testing emerging technologies in classrooms along with sharing our analyses and reflections with the classroom as a key idea in computing education—surfacing the gap between designed intentions and the different kinds of extra social work needed in the on‐ground success of different technologies.Implications for practice and/or policy Designers and researchers should create and test more interfaces alongside teachers across different classrooms and contexts aimed at supporting different kinds of voluntary collaborative interactions.Curricula, standards and school practices should further center providing students with opportunities to engage as mentors and build communities of learning across disciplines to empower minoritized students.Researchers engaging in design based research should consider using more posthumanist lenses to examine educational technologies and how they affect change in learning environments. [ABSTRACT FROM AUTHOR]

Socially shared regulation contributes to the success of collaborative learning. However, the assessment of socially shared regulation of learning (SSRL) faces several challenges in the effort to increase the understanding of collaborative learning and support outcomes due to the unobservability of the related cognitive and emotional processes. The recent development of trace‐based assessment has enabled innovative opportunities to overcome the problem. Despite the potential of a trace‐based approach to study SSRL, there remains a paucity of evidence on how trace‐based evidence could be captured and utilised to assess and promote SSRL. This study aims to investigate the assessment of electrodermal activities (EDA) data to understand and support SSRL in collaborative learning, hence enhancing learning outcomes. The data collection involves secondary school students (N = 94) working collaboratively in groups through five science lessons. A multimodal data set of EDA and video data were examined to assess the relationship among shared arousals and interactions for SSRL. The results of this study inform the patterns among students' physiological activities and their SSRL interactions to provide trace‐based evidence for an adaptive and maladaptive pattern of collaborative learning. Furthermore, our findings provide evidence about how trace‐based data could be utilised to predict learning outcomes in collaborative learning. Practitioner notesWhat is already known about this topic Socially shared regulation has been recognised as an essential aspect of collaborative learning success.It is challenging to make the processes of learning regulation 'visible' to better understand and support student learning, especially in dynamic collaborative settings.Multimodal learning analytics are showing promise for being a powerful tool to reveal new insights into the temporal and sequential aspects of regulation in collaborative learning.What this paper adds Utilising multimodal big data analytics to reveal the regulatory patterns of shared physiological arousal events (SPAEs) and regulatory activities in collaborative learning.Providing evidence of using multimodal data including physiological signals to indicate trigger events in socially shared regulation.Examining the differences of regulatory patterns between successful and less successful collaborative learning sessions.Demonstrating the potential use of artificial intelligence (AI) techniques to predict collaborative learning success by examining regulatory patterns.Implications for practice and/or policy Our findings offer insights into how students regulate their learning during collaborative learning, which can be used to design adaptive supports that can foster students' learning regulation.This study could encourage researchers and practitioners to consider the methodological development incorporating advanced techniques such as AI machine learning for capturing, processing and analysing multimodal data to examine and support learning regulation. [ABSTRACT FROM AUTHOR]

Gamification is typically implemented digitally. However, digital gamification is not always possible because of limited resources and logistical problems. It is thus necessary to explore low‐threshold gamification methods that can be easily adopted in classrooms. One such method is non‐digital gamification using physical game design elements. Currently, the literature provides a limited understanding of the design and effects of non‐digital gamification. It is also unclear whether mixed gamification (ie, a combination of digital and non‐digital gamification) is better than purely non‐digital gamification. We explored these topics using an explanatory sequential mixed‐methods design to compare the effectiveness of non‐digital gamification in a face‐to‐face course and mixed gamification in a blended course in terms of student learning achievement, cognitive engagement and course satisfaction. Two secondary grade classes were randomly assigned to the mixed (N = 52) or non‐digital (N = 52) gamification conditions. Contrary to our expectations, the quantitative results showed that the two approaches improved the students' learning achievement to the same extent. Although the students in the non‐digital gamification condition reported significantly higher‐course satisfaction than those in the mixed condition, the improvement in student cognitive engagement was significantly greater in the mixed gamification condition than in the non‐digital condition. Through focus group interviews, we analysed the students' learning experiences and synthesised the factors that influenced student cognitive engagement and course satisfaction. Practitioner notesWhat is already known about this topic Gamification is increasingly being used to increase student engagement and enhance learning.Gamification studies have mostly focused on digital modes, which often require gamified digital platforms or applications.Non‐digital gamification can be an effective method to foster learning, but the design and evaluation of non‐digital gamification is at an early stage.What this paper adds We designed non‐digital and mixed (ie, a combination of digital and non‐digital) gamification programmes and compared their effects using an explanatory sequential mixed‐methods design.Both gamification approaches led to a similar improvement in learning achievement.While non‐digital gamification fostered students' course satisfaction, mixed gamification was more effective in improving students' cognitive engagement.Implications for practice and/or policy Non‐digital gamification can be an effective low‐threshold solution for instructors. It is worth further improving the non‐digital gamification approach in school settings as it is cost‐effective and can be transferred between classrooms.The benefits of mixed gamification should not be ignored because of its affordances that improve students' cognitive engagement. [ABSTRACT FROM AUTHOR]

The flipped classroom (FC) is becoming an increasingly popular teaching method in mathematics education. However, few studies have rigorously evaluated its effectiveness, and less so in countries where students' level of mathematics achievement is low. In this study, we evaluated the impact of an FC intervention in Saudi Arabia, the country with the lowest level of maths achievement in the last iteration of the Trends International Mathematics and Science Study TIMSS. A total of 281 secondary school students received eight weeks of algebra training using either FC or traditional instruction, and were tested on algebra problems taken from past national standardised tests, as well as on a measure of self‐efficacy. Students who received the intervention showed higher self‐efficacy but no significant difference in maths achievement was observed. Students' and teachers' perceptions of the intervention were positive. Practitioner notes: What is already known about this topic The flipped classroom is an increasingly popular teaching method in mathematics education.There is evidence that the method is beneficial for students.However, there have been very few rigorous evaluation studies of flipped classrooms and most were conducted in the US, Taiwan and China.Whether the beneficial impact of flipped classrooms can be generalised to other countries, particularly countries with low proficiency in mathematics, remains unclear. What this paper adds We evaluated the impact of the flipped classroom in Saudi Arabia, a country with low proficiency in mathematics and limited technological resources.We used a rigorous design (with a control group) and educationally relevant outcome measures.We found that a flipped classroom can have a positive impact on students' self‐efficacy, and that the method was perceived positively by both teachers and students.Based on interview data, we also documented the perceptions and concerns of the participating teachers. Implications for practice and/or policy Flipped classrooms showed promising results in a context which differs markedly from previous evaluations, suggesting that the method can be beneficial in a range of contexts.This positive impact was observed despite the intervention being relatively short (six weeks) and implemented by teachers previously unfamiliar with the flipped classroom method. [ABSTRACT FROM AUTHOR]

Supporting productive disciplinary engagement (PDE) in low‐achieving students is an important but challenging goal in education. This study used a knowledge‐building inquiry approach augmented by reflective assessment to facilitate low‐achieving students' PDE. A quasi‐experimental design method was employed to examine the effects of reflective assessment in supporting low‐achieving students' PDE. The experimental class of tenth graders (n = 20) conducted inquiries in the visual arts in a knowledge‐building design augmented by reflective assessment, while the comparison class of tenth graders (n = 14) conducted inquiries in a regular knowledge‐building design without reflective assessment. This study lasted approximately 4 months and the primary data source was the online discourse. A comparative analysis of the knowledge‐building discourse characteristics and the sequential patterns of the discourse moves revealed higher cognitive, emotional and epistemic engagement in the experimental class compared with the comparison class. Epistemic network analysis showed that reflective assessment facilitated low‐achieving students' collective reflection, monitoring and regulation, as shown by their metacognitive discourse moves. Their metacognition further helped them to achieve higher levels of cognitive, emotional and epistemic engagement than the comparison class. This study provides insights into the connections between inquiry, engagement and assessment. Moreover, it has implications for designing technology‐supported collaborative inquiry environments to support low‐achieving students' engagement and higher‐level skills. Practitioner notesWhat is already known about this topic Supporting low‐achieving students' productive disciplinary engagement (PDE) is an important but challenging goal in education and limited research has focused on it.Knowledge‐building inquiry design augmented by reflective assessment is promising in supporting low‐achieving students' PDE.Little research has examined the effects of reflective assessment‐augmented knowledge building inquiry on low‐achieving students' PDE and the mechanism through which reflective assessment supports their PDE.What this paper adds Reflective assessment‐augmented knowledge‐building inquiry is effective in supporting low‐achieving students' PDE.Portfolio‐supported Reflective assessment in Knowledge Forum facilitated low‐achieving students' enactment of collective reflection, monitoring and regulation, and these metacognitive processes further helped low‐achieving students to achieve higher levels of cognitive, emotional and epistemic engagement.This study provides insights into the connections between inquiry, engagement, and assessment.Implications for practice and/or policy It is crucial to create a collaborative and reflective community culture to help low‐achieving students gradually gain agency.It is critical to help low‐achieving students to develop a productive belief that idea improvement and reflection is a continuous and iterative process.The effectiveness of reflective assessment requires face‐to‐face knowledge‐building discussion and collaborative reflection opportunities scaffolded by assessment tools and analytics that encourage low‐achieving students to engage in meta‐level discussion. [ABSTRACT FROM AUTHOR]

Conversational agents can deepen reasoning and encourage students to build on others' knowledge in collaborative learning. Embedding agents in group work, however, presents challenges where groups may ignore the agents, and this calls for designs where students perceive agents as learning partners. This study examines group interactions with two text‐based agents (ie, chatbots) that posed as an expert and a less knowledgeable peer in a high school marine biology lesson. Student messages (N = 1764) from 18 groups (52 students ages 14–15) received codes for reasoning, building on prior ideas, and responsiveness to the agents. Results indicate no differences between agents in how often each discussion move occurred. Interestingly, sequential pattern mining suggests that the less‐knowledgeable‐peer agent prompted groups to show questioning and building on others' ideas, similar to how students may act as peer tutors to the agent. Meanwhile, sequences with the expert agent resembled student‐teacher exchange, where groups responded to the agent's nudges and then provided reasoning. Findings illustrate the affordances of embedding humanized features in technology designs to promote discussion. Practitioner notesWhat is already known about this topic Conversational agents can facilitate group discussions, but can get abused or ignored by student groups.To engage students, agent designs can simulate the characteristics of familiar classroom figures, such as peers or teachers.There are limited explorations of how student groups adapt their interactions to different agent designs in collaborative settings.What the paper adds Illustrations of the utilities of diversifying agent designs in collaborative learning.Insights into the unique interaction patterns that student groups displayed to different designs, such as questioning and building on others' ideas with a less‐knowledgeable‐peer agent.Implications for practice and/or policy Embedding signs of humanness in classroom facilitation and technology design can foster responsiveness among users.Pedagogy can consider adaptive designs to promote interaction sequences that contribute to learning (eg, questioning, expanding on prior ideas) at opportune moments. [ABSTRACT FROM AUTHOR]

A discussion forum is a valuable tool to support student learning in online contexts. However, interactions in online discussion forums are sparse, leading to other issues such as low engagement and dropping out. Recent educational studies have examined the affordances of conversational agents (CA) powered by artificial intelligence (AI) to automatically support student participation in discussion forums. However, few studies have paid attention to the safety of CAs. This study aimed to address the safety challenges of CAs constructed with educational big data to support learning. Specifically, we proposed a safety‐aware CA model, benchmarked with two state‐of‐the‐art (SOTA) models, to support high school student learning in an online algebra learning platform. We applied automatic text analysis to evaluate the safety and socio‐emotional support levels of CA‐generated and human‐generated texts. A large dataset was used to train and evaluate the CA models, which consisted of all discussion post‐reply pairs (n = 2,097,139) by 71,918 online math learners from 2015 to 2021. Results show that while SOTA models can generate supportive texts, their safety is compromised. Meanwhile, our proposed model can effectively enhance the safety of generated texts while providing comparable support. Practitioner notesWhat is already known about this topic Online discussion forums have been plagued by a lack of interaction among students due to factors such as expectations to receive no response and perceptions of topic irrelevance which lead to low motivation to participate.AI‐based conversational agents can automatically support students' interactions in online discussion forums at a large scale, and their generated responses can be human‐like, contextually coherent and socio‐emotionally supportive.Unsafe discourse exchanges between students and conversational agents can be dangerous as identity attacks, aggravation and bullying behaviours embedded in discourses can disrupt students' knowledge inquiry and negatively influence student motivation and engagement. However, few educational studies have paid attention to the safety of conversational agents.What this paper adds This study proposes and synthesized strategies to build AI‐based conversational agents that automatically support online discussions with safe and supportive discourses.This study reveals the relationship between discourse safety and social support, suggesting supportive discourses can also be unsafe.This study enriches the literature on educational conversational agents by synthesizing a conceptual framework on discourse safety and social support, and by proposing concrete algorithmic strategies to improve the safety of conversational agents.Implications for practice and/or policy Researchers and practitioners can adopt strategies in this study such as generation control, open‐sourced models and public API services to evaluate students' discourse safety for early intervention or modify existing conversational agents to be safety‐aware.Practitioners can utilize the proposed conversational agent to automatically support students both safely and socio‐emotionally at a large scale.Practitioners should be cautious when examining social support with automatic analysis, as not all supportive texts are safe. While unsafe texts can provide emotional support, it does not justify their appropriateness in a learning environment. [ABSTRACT FROM AUTHOR]