Search

Advances in game-based learning and educational data mining enable novel methods of formative assessment that can reveal implicit understandings that students may demonstrate in games but may not express formally on a test. This chapter explores a framework of bridging in game-based learning classes, where teachers leverage and build upon students' game-based implicit learning experiences to support science classroom learning. Bridging was studied with two physics learning games in about 30 high-school classes per game. Results from both studies show that students in bridging classes performed better on external post-tests, when accounting for pre-test scores, than in classes that only played the game or did not play the game at all. These findings suggest the teachers' role is critical in game-based learning classes. Effective bridging includes providing teachers with common game examples along with actionable discussion points or activities to connect game-based learning with classroom content.

Computational Thinking (CT) and learning games both may have unique potential to engage a broad range of learners, including those with cognitively diverse needs. Many STEM interventions typically do not leverage the cognitive assets of learners [1] nor do these initiatives prioritize effective differentiated instruction and learning [2, 3]. The INFACT, Include Neurodiversity in Foundational and Applied Computational Thinking, project addresses this problem by leveraging two games, NumberFactory and Zoombinis, to build foundational CT practices in elementary and middle school learners. In this paper, we describe work in progress to design adaptive digital scaffolds that support a wide variety of learners’ Executive Function (EF) often associated with neurodiverse needs. Providing adaptive game environments through these embedded EF scaffolds may play a critical role in the ability of all learners to productively persist in the context of mathematical problem-solving and computational thinking.

Building on the promise shown in game-based learning research, this paper explores methods for Game-Based Learning Assessments (GBLA) using a variety of educational data mining techniques (EDM). GBLA research examines patterns of behaviors evident in game data logs for the measurement of implicit learning—the development of unarticulated knowledge that is not yet expressible on a test or formal assessment. This paper reports on the study of two digital games showing how the combination of human coding with EDM has enabled researchers to measure implicit learning of Physics. In the game Impulse, researchers combined human coding of video with educational data mining to create a set of automated detectors of students' implicit understanding of Newtonian mechanics. For Quantum Spectre, an optics puzzle game, human coding of Interaction Networks was used to identify common student errors. Findings show that several of our measures of student implicit learning within these games were significantly correlated with improvements in external postassessments. Methods and detailed findings were different for each type of game. These results suggest GBLA shows promise for future work such as adaptive games and in-class, data-driven formative assessments, but design of the assessment mechanics must be carefully crafted for each game.

This paper examines the growing field of computational thinking (CT) in education. A review of the relevant literature shows a diversity in definitions, interventions, assessments, and models. After synthesizing various approaches used to develop the construct in K-16 settings, we have created the following working definition of CT: The conceptual foundation required to solve problems effectively and efficiently (i.e., algorithmically, with or without the assistance of computers) with solutions that are reusable in different contexts. This definition highlights that CT is primarily a way of thinking and acting, which can be exhibited through the use particular skills, which then can become the basis for performance-based assessments of CT skills. Based on the literature, we categorized CT into six main facets: decomposition, abstraction, algorithm design, debugging, iteration, and generalization. This paper shows examples of CT definitions, interventions, assessments, and models across a variety of disciplines, with a call for more extensive research in this area.

Digital games provide engaging opportunities to support and assess implicit learning—the development of tacit knowledge and practices that may not be explicitly articulated by the learner. The assessment of implicit learning reveals learning not captured by traditional tests and may be critical to meet the needs of a broad range of neurodiverse learners. This chapter describes tools and methods designed to build implicit game-based learning assessment (GBLA), where research-grounded automated detectors identify implicit learning in gameplay. The detectors are based upon theoretical and empirical underpinnings, including extensive hand-labeling. The authors present a detailed overview of a six-step process for emergent GBLA, which has been applied and refined across multiple game-based learning studies. This chapter also includes a description of the data architecture and tools the authors designed and developed specifically for this approach.

Online education is a rapidly growing phenomenon for science teachers. Using a sample of 40 online science courses for teachers offered during the 2004–2005 academic year, the Learning Science Online (LSO) study examines the nature and variety of instructional methods and activities as well as communication, and students’ perceptions of supports within the course. This research is unique in that it is the first aggregate study of online science courses offered by a wide variety of educational programs. Descriptive analyses suggest the instructional methods employed in online science courses for teachersinclude frequent use of online discussions and students participated in minds-on activities, including articulation and reflection on their scientific ideas, posing questions, analyzing data, and drawing conclusions from evidence. Hands-on instructional activities were rarely used, and pen-and-paper and collaborative instructional activities were occasionally used. Technology was used primarily for communications such as discussion boards, email, and chat, but there were very few other computerbasedtools used within the courses. Students felt supported by instructors, other students, and the course design.

This paper reports on a research study of 45 classes in US schools (grades 3–8) using Zoombinis, a popular Computational Thinking (CT) learning game for ages 8 to adult. The study examined the relationship among student gameplay, related classroom activity, and the development of students’ CT practices in Zoombinis classes. A combination of research methods, including educational data-mining on game data logs, cluster analysis on teacher logs of classroom activity, and multilevel modeling, was used to determine the impact of the duration and nature of student gameplay, as well as the extent and nature of classroom activity, on student CT practices. Automated detectors of gameplay CT practices built for this research were significant predictors of external post-assessment scores, and thus show promise as implicit assessments of CT practices within gameplay. Students with high duration of gameplay and high gameplay CT practices scored highest on external post-assessment of CT practices, when accounting for pre-assessment scores. This research suggests that Zoombinis is an effective CT learning tool and CT assessment tool for elementary- and middle-school students.

In this paper, we investigate the impact of flow (operationalized as heightened challenge and skill), engagement, and immersion on learning in game-based learning environments. The data was gathered?through a survey from players (N?=?173) of two learning games (Quantum Spectre: N?=?134 and?Spumone: N?=?40). The results show that engagement in the game has a clear positive effect on learning, however, we did not find a significant effect between immersion in the game and learning. Challenge of the game had a positive effect on learning both directly and via the increased engagement. Being skilled in the game did not affect learning directly but by increasing engagement in the game. Both?the challenge of the game and being skilled in the game had a positive effect on both being engaged and immersed in the game. The challenge in the game was an especially strong predictor of learning outcomes. For the design of educational games, the results suggest that the challenge of the game should be able to keep up with the learners growing abilities and learning in order to endorse continued learning in game-based learning environments. Engagement in the game has a positive effect on learning.Immersion in the game does not have a significant effect on learning.Challenge affected learning both directly and via the increased engagement.Skill did not affect learning directly but only via the increased engagement.Perceived challenge was an especially strong predictor of learning outcomes.

This chapter describes the analysis of 729 daily teacher logs from a 2013-14 national classroom implementation study with hundreds of high school physics students using the game, Impulse, finding classrooms using materials to bridge implicit and explicit science learning performed significantly better than control classrooms (Rowe et al., 2014). This effect was moderated by whether or not the class was a Honors/AP class. The authors examine the student and teacher demographics, science content, instructional materials and methods, and game-based pedagogies as potential explanations for those findings. The largest difference among Honors/AP vs. non-Honors/AP classrooms using any Bridge activities was their use of formal, teacher-led discussion.

Players can build implicit understanding of challenging scientific concepts when playing digital science learning games [1]. In this study, we examine implicit computational thinking (CT) skills among upper elementary and middle school students during Zoombinis gameplay. We report on the development of a human labeling system for gameplay evidence of four CT skills: problem decomposition, pattern recognition, algorithmic thinking, and abstraction. We define labels that identify use of these skills in three Zoombinis puzzles, based on analysis of video data from both CT novices (upper elementary and middle school students) and CT experts (computer scientists and expert Zoombinis players). Future work will involve the construction of detectors for implicit CT skills based on these human labels, in order to analyze gamelog data at scale and give feedback to teachers.

Preference for and competence in problem-solving predicted science understanding.Preference for science-related features predicted teens' science understanding.Teens with a stronger science identity negatively evaluated their gaming groups.Competence in games with science-related features predicted science identity.Science games may reinforce self-efficacy and competence in scientific reasoning. This study explored the relationships between identity, science learning, and gaming. A survey of 1502 teenagers assessed gaming preferences, habits, science learning, science and gamer identities. Hierarchical regression analyses revealed that enjoyment of problem-solving games and identifying as a gamer were the strongest predictors for teens' science understanding. Teen preferences for games with science-related features, and competence in problem-solving games were significant predictors of teens' understanding of science. Teens who preferred collaborative social games over science-oriented games were less likely to understand the nature of science. Teens with a stronger science identity were more likely to negatively evaluate their gaming groups, preferred problem-solving games, and claimed greater competence in games with science-related features when compared to those who do not self-identify as science thinkers. Results suggest that games that seek to support those who do not feel successful in science learning should focus on social interaction and involve activities and experiences that could be utilized in the real world rather than problem solving games. Results suggest that science-focused games may reinforce perceived self-efficacy and sense of competence in real world scientific reasoning situations for those already predisposed to feel confident as science thinkers.

This paper reports on research of a game designed for scientific inquiry in a new and publicly available massively-multiplayer online environment (MMO). Educators and game designers worked together to create a highly immersive environment, a compelling storyline, and research-grounded tools for scientific inquiry within the game. The designers also played characters within the game that allowed them to deliver an evolving and responsive game narrative while also serving as participant observers for the research. Researchers integrated these observations with survey data, log data, artifact review, and interviews, to provide a broad picture of the player experience and the gaming environment. This study provides evidence that sustained scientific inquiry can be nurtured in an MMO game and that gamers’ relationships with characters in the game and other players may help facilitate that inquiry.

Our previous work has demonstrated that players who perceive a game as more challenging are likely to perceive greater learning from that game [8]. However, this may not be the case for all sources of challenge. In this study of a Science learning game called Quantum Spectre, we found that students' progress through the first zone of the game seemed to encounter a "roadblock" during gameplay, dropping out when they cannot (or do not want to) progress further. Previously we had identified two primary types of errors in the learning game, Quantum Spectre: Science Errors related to the game's core educational content; and Puzzle Errors related to rules of the game but not to science knowledge. Using this prior analysis, alongside Survival Analysis techniques for analyzing time-series data and drop-out rates, we explored players' gameplay patterns to help us understand player dropout in Quantum Spectre. These results demonstrate that modeling player behavior can be useful for both assessing learning and for designing complex problem solving content for learning environments.

Visualizing player behavior in complex problem solving tasks such as games is important for both assessing learning and for the design of content. We collected data from 195 high school students playing an optics puzzle game, Quantum Spectre, and modeled their game play as an interaction network, examining errors hypothesized to be related to a lack of implicit understanding of the science concepts embedded in the game. We found that the networks were useful for visualization of student behavior, identifying areas of student misconceptions and locating regions of the network where students become stuck. Preliminary regression analyses show a negative relationship between the science misconceptions identified during gameplay and implicit science learning.

Evidence Centered Game Design (ECgD) is an increasingly popular model used for stealth game assessments employing education data mining techniques for the measurement of learning within serious (and other) games (GlassLab, Psychometric considerations in game-based assessment. Institute of Play. Retrieved July 1, 2014, from http://www.instituteofplay.org/work/projects/glasslab-research/). There is a constant tension in ECgD between how pre-defined the learning outcomes and measures need to be, and how much important, but unanticipated, learning can be detected in gameplay. The EdGE research team is employing an emergent approach to developing a game-based assessment mechanic that starts empirically from what the players do in a well-crafted game and detects patterns that may indicate implicit understanding of salient phenomena. Implicit knowledge is foundational to explicit knowledge (Polanyi, The tacit dimension. University of Chicago Press, Chicago, IL,1966), yet is largely ignored in education because of the difficulty measuring knowledge that a learner has not yet formalized. This chapter describes our approach to measuring implicit science learning in the game, Impulse, designed to foster an implicit understanding of Newtonian mechanics using a combination of video analysis, game log analyses, and comparisons with pre-post assessment results. This research demonstrates that it is possible to reliably detect strategies that demonstrate an implicit understanding of fundamental physics using data mining techniques on user-generated data.

Educational games may lend themselves to innovative forms of learning assessment. This paper reports on game-based science learning assessments that explore how to measure the emergent learning that takes place in games by revealing tacit knowledge development. This research combines video analysis and educational data mining to identify cognitive strategies that emerge through gameplay. By studying the video and click data from high school learners playtesting the game, Impulse, we identify systematic ways of predicting the observed strategies and making possible connections to formal science learning.

Martian Boneyards is a prototype game run in a massively-multiplayer online environment designed to entice gamers to partake in collaborative scientific inquiry. This case study examines the steps designers took to foster a community of inquiry within the game. Designers played characters in the game, allowing them to be responsive to players' activities and accumulating knowledge. Players were drawn to the narrative and close relationships they developed with the designers' characters and other players. An informal and communal reward system was used to further nurture collaboration among the community. Findings suggest games like this one show promise for fostering science identity and scientific inquiry.