Search

The important but little understood problem that motivated this study was the lack of research on valid assessment methods to determine progress in higher-order learning in situations involving complex and ill-structured problems. Without a valid assessment method, little progress can occur in instructional design research with regard to designing effective learning environments to facilitate acquisition of expertise in complex, ill-structured knowledge domains. In this paper, we first present a method based on causal representations for assessing progress of learning in complex, ill-structured problem solving and discuss its theoretical framework. Then, we present an experimental study investigating its validity against adapted protocol analysis. This study explored the impact of a massively multiplayer online educational game, which was designed to support an interdisciplinary STEM education on ninth-grade students' complex, ill-structured problem solving skill acquisition. We identify conceptual similarities and differences between the two methods, present our comparative study and its results, and then discuss implications for diagnostics and applications. We conclude by determining how the two approaches could be used in conjunction for further research on complex and ill-structured problem solving. [ABSTRACT FROM AUTHOR]

This paper examines the use of mixed methods for analyzing users’ avatar-related activities in a virtual world. Server logs recorded keystroke-level activity for 595 participants over a six-month period in Whyville.net, an informal science website. Participants also completed surveys and participated in interviews regarding their experiences. Additionally, the study included online ethnographic observations of Whyville and offline observations of a subset of 88 users in classroom and after-school settings during their participation. A mixed-methods analysis identified a major user emphasis on avatar appearance and customization that was invariant across user typologies. Implications for the use of mixed methods in online environments are discussed with regard to three challenges resolved during the study: (1) appropriate reduction of the vast quantity of data, (2) integrated analysis of online and offline events, and (3) interactions between qualitative and quantitative data. [ABSTRACT FROM AUTHOR]

Educational learning environments have changed dramatically in the last 20 years. Advances in technology have enabled the World Wide Web and a sundry of other tools. In response, many researchers have argued that one way to understand learning in a complex world is to examine user interactions within Massively Multiplayer Online Games (MMOGs) [Gee (2003). What video games have to teach us about learning and literacy. New York: Palgrave/St. Martin's; Squire (2003). Educational Researcher, 35(8), 19–29; Young, Schrader, & Zheng, ]. However, few empirical investigations have explored MMOGs as a context for learning. As a result, a 20-item, Likert-type instrument was administered to 2140 participants who actively play MMOGs. Items were designed to measure players’ gaming experiences as they developed requisite skill sets and learned game content. Specifically, this investigation examined how participants’ age ranges and levels of expertise relate to behaviors, strategies, and skills exhibited with an MMOG environment. Although results are not necessarily conclusive, implications for understanding gaming expertise in contemporary educational environments are discussed. [ABSTRACT FROM AUTHOR]