Search

Building on the promise shown in game-based learning research, this paper explores methods for Game-Based Learning Assessments (GBLA) using a variety of educational data mining techniques (EDM). GBLA research examines patterns of behaviors evident in game data logs for the measurement of implicit learning—the development of unarticulated knowledge that is not yet expressible on a test or formal assessment. This paper reports on the study of two digital games showing how the combination of human coding with EDM has enabled researchers to measure implicit learning of Physics. In the game Impulse, researchers combined human coding of video with educational data mining to create a set of automated detectors of students' implicit understanding of Newtonian mechanics. For Quantum Spectre, an optics puzzle game, human coding of Interaction Networks was used to identify common student errors. Findings show that several of our measures of student implicit learning within these games were significantly correlated with improvements in external postassessments. Methods and detailed findings were different for each type of game. These results suggest GBLA shows promise for future work such as adaptive games and in-class, data-driven formative assessments, but design of the assessment mechanics must be carefully crafted for each game.

This chapter describes the analysis of 729 daily teacher logs from a 2013-14 national classroom implementation study with hundreds of high school physics students using the game, Impulse, finding classrooms using materials to bridge implicit and explicit science learning performed significantly better than control classrooms (Rowe et al., 2014). This effect was moderated by whether or not the class was a Honors/AP class. The authors examine the student and teacher demographics, science content, instructional materials and methods, and game-based pedagogies as potential explanations for those findings. The largest difference among Honors/AP vs. non-Honors/AP classrooms using any Bridge activities was their use of formal, teacher-led discussion.

Players can build implicit understanding of challenging scientific concepts when playing digital science learning games [1]. In this study, we examine implicit computational thinking (CT) skills among upper elementary and middle school students during Zoombinis gameplay. We report on the development of a human labeling system for gameplay evidence of four CT skills: problem decomposition, pattern recognition, algorithmic thinking, and abstraction. We define labels that identify use of these skills in three Zoombinis puzzles, based on analysis of video data from both CT novices (upper elementary and middle school students) and CT experts (computer scientists and expert Zoombinis players). Future work will involve the construction of detectors for implicit CT skills based on these human labels, in order to analyze gamelog data at scale and give feedback to teachers.

In this study we examine how playing Zoombinis can help upper elementary and middle school learners build implicit computational thinking (CT) skills. Building on prior methods used with the digital science learning games, Impulse and Quantum Spectre, we are combining video analysis and educational data mining to identify implicit computational thinking that emerges through gameplay [1]. This paper reports on the first phase of this process: developing a human labeling system for evidence of specific CT skills (e.g., problem decomposition, pattern recognition, algorithmic thinking, abstraction) in three Zoombinis puzzle by analyzing video data from a sample of elementary learners, middle school learners, game experts, and computer scientists. Future work will combine these human-labeled video data with game log data from these 70+ learners and computer scientists to create automated assessments of implicit computational thinking skills from gameplay behaviors in large player audiences. This poster with video examples will share results of this work-in-progress.

This paper reports on research of a game designed for scientific inquiry in a new and publicly available massively-multiplayer online environment (MMO). Educators and game designers worked together to create a highly immersive environment, a compelling storyline, and research-grounded tools for scientific inquiry within the game. The designers also played characters within the game that allowed them to deliver an evolving and responsive game narrative while also serving as participant observers for the research. Researchers integrated these observations with survey data, log data, artifact review, and interviews, to provide a broad picture of the player experience and the gaming environment. This study provides evidence that sustained scientific inquiry can be nurtured in an MMO game and that gamers’ relationships with characters in the game and other players may help facilitate that inquiry.

Our previous work has demonstrated that players who perceive a game as more challenging are likely to perceive greater learning from that game [8]. However, this may not be the case for all sources of challenge. In this study of a Science learning game called Quantum Spectre, we found that students' progress through the first zone of the game seemed to encounter a "roadblock" during gameplay, dropping out when they cannot (or do not want to) progress further. Previously we had identified two primary types of errors in the learning game, Quantum Spectre: Science Errors related to the game's core educational content; and Puzzle Errors related to rules of the game but not to science knowledge. Using this prior analysis, alongside Survival Analysis techniques for analyzing time-series data and drop-out rates, we explored players' gameplay patterns to help us understand player dropout in Quantum Spectre. These results demonstrate that modeling player behavior can be useful for both assessing learning and for designing complex problem solving content for learning environments.