Search

Stealth assessment in game-based learning environments has demonstrated significant promise for predicting student competencies and learning outcomes through unobtrusive data capture of student gameplay interactions. However, as machine learning techniques for student competency modeling have increased in complexity, the need for substantial data to induce such models has likewise increased. This raises scalability concerns, as capturing game interaction data is often logistically challenging yet necessary for supervised learning of student competency models. The generalizability of such models also poses significant challenges, and the performance of these models when applied to new domains or gameplay scenarios often suffers. To address these issues, we introduce a zero-shot learning approach that utilizes conditional generative modeling to generalize stealth assessment models for new domains in which prior data and competency labels may not exist. We evaluate our approach using observed student interactions with a game-based learning environment for introductory genetics. We use a conditional generative model to map latent embeddings of genetics concepts and student competencies to student gameplay patterns, enabling the generation of synthetic gameplay data associated with concepts and game levels that have not been previously introduced. Results indicate the zero-shot learning approach enhances the performance of the competency models on unseen game levels and concepts, pointing to more generalizable stealth assessment models and improved prediction of student competencies. [For the full proceedings, see ED623995.]

Recent years have seen significant interest in multimodal frameworks for modeling learner engagement in educational settings. Multimodal frameworks hold particular promise for predicting visitor engagement in interactive science museum exhibits. Multimodal models often utilize video data to capture learner behavior, but video cameras are not always feasible, or even desirable, to use in museums. To address this issue while still harnessing the predictive capacities of multimodal models, we investigate adversarial discriminative domain adaptation for generating modality-invariant representations of both unimodal and multimodal data captured from museum visitors as they engage with interactive science museum exhibits. This approach enables the use of pre-trained multimodal visitor engagement models in circumstances where multimodal instrumentation is not available. We evaluate the visitor engagement models in terms of early prediction performance using exhibit interaction and facial expression data captured during visitor interactions with a science museum exhibit for environmental sustainability. Through the use of modality-invariant data representations generated by the adversarial discriminative domain adaptation framework, we find that pre-trained multimodal models achieve competitive predictive performance on interaction-only data compared to models evaluated using complete multimodal data. The multimodal framework outperforms unimodal and non-adapted baseline approaches during early intervals of exhibit interactions as well as entire interaction sequences. [For the full proceedings, see ED615472.]

In recent years, game-based learning has shown significant promise for creating engaging and effective learning experiences. Developing models that can predict whether students will struggle with mastering certain concepts could guide adaptive support to assist students with mastering those concepts. Game-based learning environments offer significant potential for unobtrusively assessing student learning without interfering with gameplay through stealth assessment. Prior work on stealth assessment has focused on a single machine learning technique such as dynamic Bayesian networks or long short-term memory networks; however, a single modeling technique often does not guarantee the best predictive performance for all concepts of interest. In this paper, we present a hybrid data-driven approach to stealth assessment for predicting students' mastery of concepts through interactions with a game-based learning environment for introductory genetics. Stealth assessment models utilize students' observed gameplay behaviors using challenge- and session-based features to predict students' learning outcomes on identified concepts. We present single-task and multi-task models for predicting students' mastery of concepts and the results suggest that the hybrid stealth assessment framework outperforms individual models and holds significant potential for predicting student competencies. [For the full proceedings, see ED607784.]

Recent years have seen a growing interest in block-based programming environments for computer science education. While these environments hold significant potential for novice programmers, they lack the adaptive support necessary to accommodate students exhibiting a wide range of initial capabilities and dispositions toward computing. A promising approach to addressing this problem is introducing adaptive feedback. This work investigates a key capability for adaptive support: training student models that predict student success in block-based programming activities for novice programmers. The predictive student models utilize four categories of features: prior performance, hint usage, activity progress, and interface interaction. In addition to evaluating the accuracy of these models for multiple block-based programming activities, we also investigate how quickly the models converge to accurate prediction, and we evaluate the additive value of each of the four categories of features. Results show that the predictive models are able to predict whether a student will successfully complete an exercise with high accuracy, as well as converge on this prediction early in the sequence of student interactions. [For the full proceedings, see ED599096.]

Student interactions with game-based learning environments produce a wide range of in-game problem-solving sequences. These sequences can be viewed as trajectories through a game's problem-solving space. In this paper, we present a general framework for analyzing students' problem-solving behavior in game-based learning environments by filtering their gameplay action sequences into time series representing trajectories through the game's problem-solving space. This framework was investigated with data from a laboratory study conducted with 68 college students tasked with solving the problem scenario in a game-based learning environment for microbiology education, CRYSTAL ISLAND. Using this representation of student problem solving, we derive the slope of the problem-solving trajectories and lock-step Euclidean distance to an expert problem-solving trajectory. Analyses indicate that the trajectory slope and temporal distance to an expert path are both correlated with students' normalized learning gains, as well as a complementary measure of in-game problem-solving performance. The results suggest that the filtered time series framework for analyzing student problem-solving behavior shows significant promise for assessing the temporal nature of student problem solving during game-based learning. [For the full proceedings, see ED593090.]

A key affordance of game-based learning environments is their potential to unobtrusively assess student learning without interfering with gameplay. In this paper, we introduce a temporal analytics framework for stealth assessment that analyzes students' problem-solving strategies. The strategy-based temporal analytic framework uses long short-term memory network-based evidence models and clusters sequences of students' problem-solving behaviors across consecutive tasks. We investigate this strategy based temporal analytics framework on a dataset of problem solving behaviors collected from student interactions with a game-based learning environment for middle school computational thinking. The results of an evaluation indicate that the strategy-based temporal analytics framework significantly outperforms competitive baseline models with respect to stealth assessment predictive accuracy. [For the full proceedings, see ED593090.]

Tutorial dialogue is a highly effective way to support student learning. It is widely recognized that tutor dialogue moves can significantly influence learning outcomes, but the ways in which tutor moves, student affective response, and outcomes are related remains an open question. This paper presents an analysis of student affective response, as evidenced by multimodal data streams, immediately following tutor questions. The findings suggest that students' affect immediately following tutor questions is highly predictive of end-of-session self-reported engagement and frustration. Notably, facial action units which have been associated with emotional states such as embarrassment, disgust, and happiness appear to play important roles in students' expressions of frustration and engagement during learning. This line of investigation will aid in the development of a deeper understanding of the relationships between tutorial dialogue and student affect during learning. [For the full proceedings, see ED592609.]

Recent years have seen a growing interest in intelligent game-based learning environments featuring virtual agents. A key challenge posed by incorporating virtual agents in game-based learning environments is dynamically determining the dialogue moves they should make in order to best support students' problem solving. This paper presents a data-driven modeling approach that uses a Wizard-of-Oz framework to predict human wizards' dialogue acts based on a sequence of multimodal data streams of student interactions with a game-based learning environment. To effectively deal with multiple, parallel sequential data streams, this paper investigates two sequence-labeling techniques: long short-term memory networks (LSTMs) and conditional random fields. We train predictive models utilizing data corpora collected from two Wizard-of-Oz experiments in which a human wizard played the role of the virtual agent unbeknownst to the student. Empirical results suggest that LSTMs that utilize game trace logs and facial action units achieve the highest predictive accuracy. This work can inform the design of intelligent virtual agents that leverage rich multimodal student interaction data in game-based learning environments.

To support our understanding of how we might support the use of modeling in PBL activities for classroom use, this study explores how to scaffold collaborative scientific modeling activities. It draws on prior work with scientific modeling centered on the use of the Component-Mechanism-Phenomenon (CMP) conceptual framework (Hmelo-Silver et al., 2017; Jordan et al., 2014). Findings suggest that students' perception about the spatial configuration appear to support or hinder their generation of models. Additionally, it appears that the norms that underpin certain roles may be far more influential in supporting group collaboration than the introduction of the roles themselves.