Search

Pedagogical agents have been designed to support the significant challenges that learners face when self-regulating in advanced learning environments. Evidence suggests differences in learners' prior skills and abilities, in conjunction with excessive didactic support, can cause overreliance on these external aids, which in turn prevents deeper learning, and pedagogical agents can provide tailored scaffolding to accommodate learners' individual needs. However, there is less evidence about the impact of abstract scaffolding, such as the sharing of non-verbal metacognitive information via a pedagogical agent's facial expressions, on self-regulated learning. To assess factors in the passing of non-verbal metacognitive information via pedagogical agents in a multimedia learning environment, we used growth modeling with self-reports, eye-tracking, and log-file data collected from fifty (n = 50) undergraduates at a large North American university as they learned about human body systems while using MetaTutor-IVH, a multimedia learning environment with a pedagogical agent. We controlled for participant characteristics (perceived utility of emotions for self- and other-centered positive and negative emotions) and characteristics of the metacognitive monitoring information provided by a pedagogical agent (expression type and expression congruency) to assess factors in non-verbally communicating metacognitive information. Results suggest that learners attend to pedagogical agents less over time, but this rate of change is weaker when an agent is providing an expression that is congruent with the ground truth of the environment. Further, only the perceived information utility of other-centered negative emotions has a significant effect on this duration, suggesting learners are driven to consult pedagogical agents to avoid embarrassment or shame. We discuss design implications of these findings for technology-based learning environments.

Reflection is critical for adolescents' problem solving and learning in game-based learning environments (GBLEs). Yet challenges exist in the literature because most studies lack a theoretical perspective and clear operational definition to inform how and when reflection should be scaffolded during game-based learning. In this paper, we address these issues by studying the quantity and quality of 120 adolescents' written reflections and their relation to their learning and problem solving with Crystal Island, a GBLE. Specifically, we: (1) define reflection and how it relates to skill and knowledge acquisition; (2) review studies examining reflection and its relation to problem solving and learning with emerging technologies; and (3) provide direction for building reflection prompts into GBLEs that are aligned with the learning goals built into the learning session (e.g., learn about microbiology versus successfully solve a problem) to maximize adolescents' reflection, learning, and performance. Overall, our findings emphasize how important it is to examine not only the quantity of reflection but also the depth of written reflection as it relates to specific learning goals. We discuss the implications of using game-learning analytics to guide instructional decision making in the classroom.

The goal of this chapter is to propose a cyclical process of how teachers can use multimodal multichannel data of cognitive, affective, metacognitive, motivational, and social processes to assist with the understanding of their own and their students' self-regulated learning (SRL), and their subsequent instructional decision making. What differentiates our proposed framework from theories of self-regulated learning is that we propose this cycle occurs throughout all phases of self-regulated learning and in real time. In this chapter, we outline each component of the framework, followed by a discussion of future steps that discuss our future view of the classroom with teachers and how they can use multimodal multichannel data to help them with their instructional decision making. We aim to make it clear that: (1) our proposed stages are non-linear and interdependent, which represents the foundation of SRL theory and the structure of classroom learning, and (2) our proposed framework highlights that using any technological tool to help teachers, not replace them.

Game-based learning environments hold significant promise for facilitating learning experiences that are both effective and engaging. To support individualised learning and support proactive scaffolding when students are struggling, game-based learning environments should be able to accurately predict student knowledge at early points in students' gameplay. Student knowledge is traditionally assessed prior to and after each student interacts with the learning environment with conventional methods, such as multiple choice content knowledge assessments. While previous student modelling approaches have leveraged machine learning to automatically infer students' knowledge, there is limited work that incorporates the fine-grained content from each question in these types of tests into student models that predict student performance at early junctures in gameplay episodes. This work investigates a predictive student modelling approach that leverages the natural language text of the post-gameplay content knowledge questions and the text of the possible answer choices for early prediction of fine-grained individual student performance in game-based learning environments. With data from a study involving 66 undergraduate students from a large public university interacting with a game-based learning environment for microbiology, Crystal Island, we investigate the accuracy and early prediction capacity of student models that use a combination of gameplay features extracted from student log files as well as distributed representations of post-test content assessment questions. The results demonstrate that by incorporating knowledge about assessment questions, early prediction models are able to outperform competing baselines that only use student game trace data with no question-related information. Furthermore, this approach achieves high generalisation, including predicting the performance of students on unseen questions.

Self-regulated learning (SRL), learners' monitoring and control of cognitive, affective, metacognitive, and motivational processes, is essential for learning. However, cognitive and metacognitive SRL strategies are not typically used accurately leading to poor learning outcomes. Intelligent tutoring systems (ITSs) attempt to address this issue by prompting and scaffolding learners to engage in SRL via using pedagogical agents. However, current literature does not examine the extent to which learners' deployed strategies are functional or dysfunctional in relation to pedagogical agent scaffolding. The current study collected 117 undergraduate students' data as they learned with MetaTutor, an ITS about the human circulatory system. Participants were randomly assigned to either the (1) Prompt and Feedback Condition where pedagogical agents scaffolded cognitive and metacognitive SRL strategies or (2) Control Condition where no prompts or feedback were provided. Results demonstrated that learners who received prompts by the pedagogical agents to engage in SRL had higher learning gains as well as greater frequencies across most strategies compared to those in the Control Condition who relied on self-initiated strategy use. While sequential transitions across all strategies were not significant between conditions, further analysis grounded in Complex Systems Theory found that learners who were prompted to engage in strategies demonstrated a significantly lower degree of repetition and balance between repetitive and novel patterns of strategy use. The findings suggest that pedagogical agents within MetaTutor successfully scaffolded the functional deployment of cognitive and metacognitive SRL strategies and are indicative of higher learning after interacting with ITSs.

Self-regulated learning conducted through metacognitive monitoring and scientific inquiry can be influenced by many factors, such as emotions and motivation, and are necessary skills needed to engage in efficient hypothesis testing during game-based learning. Although many studies have investigated metacognitive monitoring and scientific inquiry skills during game-based learning, few studies have investigated how the sequence of behaviors involved during hypothesis testing with game-based learning differ based on both efficiency level and emotions during gameplay. For this study, we analyzed 59 undergraduate students' (59% female) metacognitive monitoring and hypothesis testing behavior during learning and gameplay with CRYSTAL ISLAND, a game-based learning environment that teaches students about microbiology. Specifically, we used sequential pattern mining and differential sequence mining to determine if there were sequences of hypothesis testing behaviors and to determine if the frequencies of occurrence of these sequences differed between high or low levels of efficiency at finishing the game and high or low levels of facial expressions of emotions during gameplay. Results revealed that students with low levels of efficiency and high levels of facial expressions of emotions had the most sequences of testing behaviors overall, specifically engaging in more sequences that were indicative of less strategic hypothesis testing behavior than the other students, where students who were more efficient with both levels of emotions demonstrated strategic testing behavior. These results have implications for the strengths of using educational data mining techniques for determining the processes underlying patterns of engaging in self-regulated learning conducted through hypothesis testing as they unfold over time; for training students on how to engage in the self-regulation, scientific inquiry, and emotion regulation processes that can result in efficient gameplay; and for developing adaptive game-based learning environments that foster effective and efficient self-regulation and scientific inquiry during learning.

This study examined 57 learners' emotions (i.e., joy, anger, confusion, frustration) as they engaged with scientific content while learning about microbiology with Crystal Island, a game-based learning environment (GBLE). Measures of learners' prior knowledge, in-game text comprehension, facial expressions of emotion, and posttest reading comprehension were collected to examine the relationship between emotions and single- and multiple-text comprehension. Analyses found that both discrete and non-discrete emotions were expressed during reading and answering in-game assessments of single-text comprehension. Learners expressed greater joy during reading and greater expressions of anger, confusion, and frustration during in-game assessments. Further results found that learners who expressed a high number of different emotions throughout reading and completing in-game assessments tended to have lower in-game comprehension scores whereas a higher number of different expressed emotions while completing in-game assessments was associated with greater posttest comprehension. Finally, while increased prior knowledge was associated with higher single- and multiple-text comprehension, there was no interaction between prior knowledge and emotions on multiple-text comprehension. Overall, this study found that (1) learners often express more than one emotion during GBLE activities, (2) emotions expressed while learning with a GBLE shift across different activities, and (3) emotions are related to demonstrated comprehension, but the type of activity influences this relationship. Results from this study provide implications for how emotions can be examined as learners engage in GBLE activities as well as the design of GBLEs to support learners' emotions accounting for different activity demands to increase comprehension of single and multiple texts.

The aim of this study was to predict university students' learning performance using different sources of performance and multimodal data from an Intelligent Tutoring System. We collected and preprocessed data from 40 students from different multimodal sources: learning strategies from system logs, emotions from videos of facial expressions, allocation and fixations of attention from eye tracking, and performance on posttests of domain knowledge. Our objective was to test whether the prediction could be improved by using attribute selection and classification ensembles. We carried out three experiments by applying six classification algorithms to numerical and discretized preprocessed multimodal data. The results show that the best predictions were produced using ensembles and selecting the best attributes approach with numerical data.

Game-based learning environments are designed to provide effective and engaging learning experiences for students. Predictive student models use trace data extracted from students' in-game learning behaviors to unobtrusively generate early assessments of student knowledge and skills, equipping game-based learning environments with the capacity to anticipate student outcomes and proactively deliver adaptive scaffolding or notify instructors. Reflection is a key component of self-regulated learning, and it is critical in effective learning. However, there is currently limited work exploring the utility of reflection for inducing accurate predictive student models. This article presents a predictive student modeling framework that leverages natural language responses to in-game reflection prompts to predict student learning outcomes in a game-based learning environment for middle school microbiology, CRYSTAL ISLAND. With data from a pair of classroom studies involving 118 middle school students, we investigate the accuracy of early prediction models that utilize features extracted from student trace data combined with word embedding-based representations (i.e., GloVe, ELMo) of student reflection responses. We evaluate the accuracy of the predictive models over time using data from incremental segments of each student's interaction with the game-based learning environment, and we compare against models that omit student reflection features. Results reveal that models encoding students' natural language reflections with ELMo word embeddings yield significantly improved accuracy compared to other representations, with the greatest accuracy demonstrated by an ensemble of predictive models. We discuss the implications of these results for the design of game-based learning environments.

Game-based learning environments (GBLEs) focus on enhancing learning by providing learners with various representations of information (e.g., text, diagrams, etc.) while allowing full autonomy, or control over their actions. Challenges arise as research shows that learners inaccurately use cognitive and metacognitive processes when given full autonomy. This study examined 105 undergraduates who were randomly assigned to autonomy conditions (i.e., full, partial, and no autonomy) as they interacted with scientific informational text presentations (i.e., non-player characters [NPCs], books and research articles, posters) during learning with Crystal Island, a GBLE. We assessed how learners' eye-tracking (e.g., fixation durations on objects) and log-file (e.g., durations of activities) data reflected how learners interacted with text presentations and selected pretest-relevant items (i.e., text providing answers to questions on the pretest). Results showed that participants in the partial autonomy condition (n = 38) demonstrated higher learning gains than those in the full autonomy condition (n = 45). Time spent interacting with all books and research articles within Crystal Island were positively correlated with learning gains. There were significant differences in learners' duration and fixation duration on informational text presentation interactions between conditions and within types of presentations as well as significant interactions between pretest-relevant items and types of presentations. Overall, autonomy and pretest relevancy impact the time interacting with informational text presentations which influence learning. Implications are provided for applying autonomy during game-based learning, and how this may direct future implementations of AI within GBLEs to provide implicit scaffolding via adaptively limiting learners' autonomy as they interact with informational text.

A distinctive feature of game-based learning environments is their capacity to create learning experiences that are both effective and engaging. Recent advances in sensor-based technologies such as facial expression analysis and gaze tracking have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics informed by multimodal data captured during students' interactions with game-based learning environments hold significant promise for developing a deeper understanding of game-based learning, designing game-based learning environments to detect maladaptive behaviors and informing adaptive scaffolding to support individualized learning. This paper introduces a multimodal learning analytics approach that incorporates student gameplay, eye tracking and facial expression data to predict student posttest performance and interest after interacting with a game-based learning environment, CRYSTAL ISLAND. We investigated the degree to which separate and combined modalities (i.e., gameplay, facial expressions of emotions and eye gaze) captured from students (n = 65) were predictive of student posttest performance and interest after interacting with Crystal Island. Results indicate that when predicting student posttest performance and interest, models utilizing multimodal data either perform equally well or outperform models utilizing unimodal data. We discuss the synergistic effects of combining modalities for predicting both student interest and posttest performance. The findings suggest that multimodal learning analytics can accurately predict students' posttest performance and interest during game-based learning and hold significant potential for guiding real-time adaptive scaffolding.

The goal of this study was to examine college students' (n = 61) contextualized emotions during in-game actions while playing "Crystal Island," a game-based learning environment where students are tasked with solving the mystery of what illness impacted all island inhabitants. We examined emotions during in-game actions: during book reading, after scanning food items for the transmission source, and after submitting a final diagnosis. We dichotomized each activity's feedback into a positive or negative outcome: a relevant or irrelevant book for solving the mystery, testing food items that generate a positive or negative result, or submitting a correct or incorrect final diagnosis. Results revealed that expressing joy while reading a relevant book and expressing confusion after a positive scan significantly positively predicted overall game score, which we used as a proxy for problem-solving performance. Implications include understanding different levels of emotions students express during learning with all advanced learning technologies.

The goal of this study was to use eye-tracking and log-file data to investigate the impact of prior knowledge on college students' (N = 194, with a subset of n = 30 for eye tracking and sequence mining analyses) fixations on (i.e., looking at) self-regulated learning-related areas of interest (i.e., specific locations on the interface) and on the sequences of engaging in cognitive and metacognitive self-regulated learning processes during learning with MetaTutor, an Intelligent Tutoring System that teaches students about the human circulatory system. Results revealed that there were no significant differences in fixations on single areas of interest by the prior knowledge group students were assigned to; however there were significant differences in fixations on pairs of areas of interest, as evidenced by eye-tracking data. Furthermore, there were significant differences in sequential patterns of engaging in cognitive and metacognitive self-regulated learning processes by students' prior knowledge group, as evidenced from log-file data. Specifically, students with high prior knowledge engaged in processes containing cognitive strategies and metacognitive strategies whereas students with low prior knowledge did not. These results have implications for designing adaptive intelligent tutoring systems that provide individualized scaffolding and feedback based on individual differences, such as levels of prior knowledge.

In this paper, we present the results obtained using a clustering algorithm (Expectation-Maximization) on data collected from 106 college students learning about the circulatory system with MetaTutor, an agent-based Intelligent Tutoring System (ITS) designed to foster self-regulated learning (SRL). The three extracted clusters were validated and analyzed using multivariate statistics (MANOVAs) in order to characterize three distinct profiles of students, displaying statistically significant differences over all 12 variables used for the clusters formation (including performance, use of note-taking and number of sub-goals attempted). We show through additional analyses that variations also exist between the clusters regarding prompts they received by the system to perform SRL processes. We conclude with a discussion of implications for designing a more adaptive ITS based on an identification of learners' profiles.

Research on collaborative learning between humans and virtual pedagogical agents represents a necessary extension to recent research on the conceptual, theoretical, methodological, analytical, and educational issues behind co- and socially-shared regulated learning between humans. This study presents a novel coding framework that was developed and used to describe collaborations between learners and a pedagogical agent (PA) during a subgoal setting activity with MetaTutor, an intelligent tutoring system. Learner-PA interactions were examined across two scaffolding conditions: prompt and feedback (PF), and control. Learners' compliance to follow the PA's prompts and feedback in the PF condition were also examined. Results demonstrated that learners followed the PA's prompts and feedback to help them set more appropriate subgoals for their learning session the majority of the time. Descriptive statistics revealed that when subgoals were set collaboratively between learners and the PA, they generally lead to higher proportional learning gains when compared to less collaboratively set goals. Taken together, the results provide preliminary evidence that learners are both willing to engage in and benefit from collaborative interactions with PAs when immediate, directional feedback and the opportunity to try again are provided. Implications and future directions for extending co- and socially-shared regulated learning theories to include learner-PA interactions are proposed.

This study examined the nature of cognitive, metacognitive, and affective processes among a medical team experiencing difficulty managing a challenging simulated medical emergency case by conducting in-depth analysis of process data. Medical residents participated in a simulation exercise designed to help trainees to develop medical expertise, effective leadership, and team management skills. Purposive sampling was used to select one team for case study based on overall performance. Video and audio data were collected from the simulation and debriefing session and a follow-up interview was conducted with the team leader. Performance measures were also collected from expert raters (i.e., experienced staff physicians). Video data were reviewed and coded for cognitive, metacognitive, and emotional events exhibited by team members during the simulation. Interview and debriefing transcripts were coded for themes related to these regulatory processes. Results from quantitative and qualitative analyses revealed that the team exhibited lower-order cognitive and metacognitive process (e.g., summarizing, providing information) more often than higher-order processes (e.g., evaluation, reasoning). Furthermore, team members expressed negative emotions (e.g., anxiety) more often than positive emotions (e.g., enjoyment). Chi square analyses of the team leader revealed that negative emotions were significantly more frequently preceded by lower-order processes compared to higher-order processes. Qualitative thematic analyses provided further corroboration of these findings. The findings suggest that medical trainees (particularly teams experiencing difficulty managing a challenging case) may require further scaffolding in their use of regulatory processes within medical emergencies. The results from this study are discussed in terms of implications for theories of self-regulation, methodological advances, and instructional design for medical education.

Engagement is one of the most widely misused and overgeneralized constructs found in the educational, learning, instructional, and psychological sciences. The articles in this special issue represent a wide range of traditions and highlight several key conceptual, theoretical, methodological, and analytical issues related to defining and measuring engagement. All the approaches exemplified by the contributors show different ways of conceptualizing and measuring engagement and demonstrate the strengths and weaknesses of each method to significantly augment our current understanding of engagement. Despite the numerous issues raised by the authors of this special issue and in my commentary, I argue that focusing on process data will lead to advances in models, theory, methods, analytical techniques, and ultimately instructional recommendations for learning contexts that effectively engage students.

Hypermedia learning environments (HLE) unevenly present new challenges and opportunities to learning processes and outcomes depending on learner characteristics and instructional supports. In this experimental study, we examined how one such HLE--MetaTutor, an intelligent, multi-agent tutoring system designed to scaffold cognitive and metacognitive self-regulated learning (SRL) processes--interacts with learner's prior domain knowledge to affect their note-taking activities and subsequent learning outcomes. Sixty (N = 60) college students studied with MetaTutor for 120 min and took notes on hypermedia content of the human circulatory system. Log-files and screen recordings of learner-system interactions were used to analyze notes for several quantitative and qualitative variables. Results show that most note-taking was a verbatim copy of instructional content, which negatively related to the post-test measure of learning. There was an interaction between prior knowledge and pedagogical agent scaffolding, such that low prior knowledge students took a greater quantity of notes compared to their high prior knowledge counterparts, but only in the absence of MetaTutor SRL scaffolding; when agent SRL scaffolding was present, the note-taking activities of low prior knowledge students were statistically equivalent to the number of notes taken by their high prior knowledge counterparts. Theoretical and instructional design implications are discussed.

In this study, we examined the effect of two metacognitive scaffolds on the accuracy of confidence judgments made while diagnosing dermatopathology slides in SlideTutor. Thirty-one (N = 31) first- to fourth-year pathology and dermatology residents were randomly assigned to one of the two scaffolding conditions. The cases used in this study were selected from the domain of nodular and diffuse dermatitides. Both groups worked with a version of SlideTutor that provided immediate feedback on their actions for 2 h before proceeding to solve cases in either the "Considering Alternatives" or "Playback" condition. No immediate feedback was provided on actions performed by participants in the scaffolding mode. Measurements included learning gains (pre-test and post-test), as well as metacognitive performance, including Goodman-Kruskal Gamma correlation, bias, and discrimination. Results showed that participants in both conditions improved significantly in terms of their diagnostic scores from pre-test to post-test. More importantly, participants in the Considering Alternatives condition outperformed those in the "Playback" condition in the accuracy of their confidence judgments and the discrimination of the correctness of their assertions while solving cases. The results suggested that presenting participants with their diagnostic decision paths and highlighting correct and incorrect paths helps them to become more metacognitively accurate in their confidence judgments.

We used a false-biofeedback methodology to manipulate physiological arousal in order to induce affective states that would influence learners' metacognitive judgments and learning performance. False-biofeedback is a method used to induce physiological arousal (and resultant affective states) by presenting learners with audio stimuli of false heart beats. Learners were presented with accelerated, baseline, or no heart beat (control) while they completed a challenging learning task. We tested four hypotheses about the effect of false-biofeedback. The "alarm" vs. "alert hypothesis" predicted that false biofeedback would be appraised as either a signal of distress and would impair learning (alarm), or as a signal of engagement and would facilitate learning (alert). The "differential biofeedback hypothesis" predicted that the alarm and alert effects would be dependent on the type of biofeedback (accelerated vs. baseline). The "question depth hypothesis" predicted that these effects would be more pronounced for challenging inference questions. Lastly, the "self vs. recording hypothesis" predicted that effects would only occur if participants believed that false biofeedback was indicative of their own physiological arousal. In general, learners experienced more positive/activating affective states, made more confident metacognitive judgments, and achieved higher learning when they received accelerated or baseline biofeedback while answering a challenging inference question, irrespective of the perceived source of the biofeedback. Thus, our findings supported the alert and question depth hypotheses, but not the differential biofeedback or self vs. recording hypotheses. Implications of the findings for the integration of affective processes into models of cognitive and metacognitive processes during learning are discussed. (Contains 5 tables and 7 figures.)

This article describes the problem of detecting the student mental models, i.e. students' knowledge states, during the self-regulatory activity of prior knowledge activation in MetaTutor, an intelligent tutoring system that teaches students self-regulation skills while learning complex science topics. The article presents several approaches to automatically detecting students' mental models in MetaTutor based on paragraphs generated by students during prior knowledge activation. Three major categories of methods (content-based overlap methods; cohesion analysis of text; and tf-idf based weighted representations) were developed and combined with machine learning algorithms in order to automatically infer the underlying parameters. A detailed comparison among the methods and across all machine learning algorithms is provided. The evaluation of the proposed methods is performed by comparing the methods' predictions with human judgments on a set of 309 prior knowledge activation paragraphs collected from experiments with the MetaTutor system on college students. According to the experiments, a word-weighting method, which uses tf-idf values calculated from the corpus, combined with a Bayes Nets machine learning algorithm, offers the most accurate results. Second best performance is given by a Latent Semantic Analysis-based approach enhanced with lexical features and combined with the machine learning algorithm of Logistic Regression. (Contains 2 figures, 12 tables, and 2 footnotes.)

A number of authors have presented data that challenge the validity of self-report of strategy use or choice of strategy. We created a multiple-choice measure of students' strategy use based on the work of Kozminsky, E., and Kozminsky, L. (2001), and tested it with three samples as part of a series of studies testing the fit of the DIME model of reading comprehension. One study was conducted at the high school level (N = 175) and two at the undergraduate level (N = 185 and 737). Over the three studies with three different samples, we found good evidence for the internal consistency reliability and concurrent validity of this type of measure. Commonality analysis suggested that strategy use mainly makes a shared contribution to comprehension with other predictors, especially inference, and to some extent vocabulary, background knowledge, and word reading. The measure was relatively easy to construct and easy to administer to large numbers of students, and showed much higher evidence of concurrent validity than self-ratings of frequency of use of strategies. [This paper was presented at the 2009 annual conference of the European Association for Research in Learning and Instruction, Amsterdam, The Netherlands, August 24-27, 2009.]

This study examined the temporal and dynamic nature of students' self-regulatory processes while learning about the circulatory system with hypermedia. A total of 74 undergraduate students were randomly assigned to 1 of 2 conditions: independent learning or externally assisted learning. Participants in the independent learning condition used a hypermedia environment to learn about the circulatory system on their own, while participants in the externally assisted condition used the same hypermedia environment, but were given prompts and feedback from a human tutor during the session to facilitate their self-regulatory behavior. Previously published pretest-posttest shifts toward more mature understanding of the circulatory system indicate that the externally assisted condition leads to greater learning. The present article uses think-aloud data during learning to explore process issues in light of models of self-regulated learning and conditions of engagement that may affect those processes. Results indicate that access to a human tutor influences the deployment of regulatory processes, intervals of use, and temporal dependencies. For example, there is significantly more planning during the final time interval of the learning session in the externally assisted condition; students in both conditions deploy more learning strategies in the first and second time intervals, compared to the last two time intervals. Additionally, in the externally assisted condition participants were more likely to shift from planning to monitoring and less likely to shift from learning strategies to planning. We discuss theoretical, conceptual, and methodological issues pertaining to these results, as well as implications for future research and the design of adaptive hypermedia systems. (Contains 2 figures, 5 tables, and 3 footnotes.)

Self-regulated learning (SRL) skills are enacted dynamically over the course of learning tasks, and the frequency and quality of their use can fluctuate dramatically. Further, students' SRL skills can vary from one academic domain to another and even from one task to another within a single domain. Thus, while SRL skills are essential for effective learning, they can look quite different depending on the content to be learned, the context of learning, and even the characteristics of the students themselves. Increasingly, computer-based learning environments (CBLEs) are being introduced to the college classroom context, but their value-added depends critically on students' ability to take advantage of CBLEs' capabilities. CBLEs are most powerful when students can effectively self-regulate. In this chapter, the authors explore the various types of CBLEs, the challenges students face in using them to learn about complex topics, and their potential as cognitive and metacognitive tools to support learning. Then they discuss how SRL skills are a key requirement for the successful use of CBLEs and how students' SRL skills can be scaffolded. Finally, they discuss the educational implications of catalyzing the potential of CBLEs through the effective implementation of SRL scaffolds and training.

Previous studies in our laboratory have shown the benefits of immediate feedback on cognitive performance for pathology residents using an intelligent tutoring system (ITS) in pathology. In this study, we examined the effect of immediate feedback on metacognitive performance, and investigated whether other metacognitive scaffolds will support metacognitive gains when immediate feedback is faded. Twenty-three participants were randomized into intervention and control groups. For both groups, periods working with the ITS under varying conditions were alternated with independent computer-based assessments. On day 1, a within-subjects design was used to evaluate the effect of immediate feedback on cognitive and metacognitive performance. On day 2, a between-subjects design was used to compare the use of other metacognitive scaffolds (intervention group) against no metacognitive scaffolds (control group) on cognitive and metacognitive performance, as immediate feedback was faded. Measurements included learning gains (a measure of cognitive performance), as well as several measures of metacognitive performance, including Goodman-Kruskal gamma correlation ("G"), bias, and discrimination. For the intervention group, we also computed metacognitive measures during tutoring sessions. Results showed that immediate feedback in an intelligent tutoring system had a statistically significant positive effect on learning gains, "G" and discrimination. Removal of immediate feedback was associated with decreasing metacognitive performance, and this decline was not prevented when students used a version of the tutoring system that provided other metacognitive scaffolds. Results obtained directly from the ITS suggest that other metacognitive scaffolds do have a positive effect on "G" and discrimination, as immediate feedback is faded. We conclude that immediate feedback had a positive effect on both metacognitive and cognitive gains in a medical tutoring system. Other metacognitive scaffolds were not sufficient to replace immediate feedback in this study. However, results obtained directly from the tutoring system are not consistent with results obtained from assessments. In order to facilitate transfer to real-world tasks, further research will be needed to determine the optimum methods for supporting metacognition as immediate feedback is faded.

Models of personal epistemology have not been sufficiently integrated despite conceptual similarities. We attempted to model both dimensional and positional aspects of personal epistemology, as well as examine the domain specificity of these phenomena. The conceptual framework for this study was a new model of epistemic and ontological cognitive development that addresses numerous concerns in the literature. A model-based quantitative instrument was created and administered to 740 students ranging from middle-school through graduate school. Results of confirmatory factor and factor-mixture model analyses provided evidence for the construct validity and reliability of scores from the instrument as well as support for many aspects of the underlying conceptual model. Implications for further model development and research are discussed. (Contains 8 tables and 4 figures.)

Self-regulated learning (SRL) with hypermedia environments involves a complex cycle of temporally unfolding cognitive and metacognitive processes that impact students' learning. We present several methodological issues related to treating SRL as an event and strengths and challenges of using online trace methodologies to detect, trace, model, and foster students' SRL processes. We first describe a scenario illustrating the complex nature of SRL processes during learning with hypermedia. We provide our theoretically driven assumptions regarding the use of several cognitive methodologies, including concurrent think aloud protocols, and provide several examples of empirical evidence regarding the advantages of treating SRL as an event. Last, we discuss challenges for measuring cognitive and metacognitive processes in the context of MetaTutor, an intelligent adaptive hypermedia learning environment. This discussion includes the roles of pedagogical agents in goal-generation, multiple representations, agent-learner dialogue, and a system's ability to detect, track, and model SRL processes during learning. (Contains 3 footnotes, 3 tables, and 3 figures.)

While research has documented the key role of monitoring processes during hypermedia learning, limited empirical research has used process data to examine the possibility that these processes mediate the relationship between motivational constructs (such as self-efficacy) and cognitive factors (such as prior domain knowledge) with hypermedia learning outcomes. This multi-method study addressed this issue by examining: (1) The extent to which the relationship between self-efficacy and hypermedia learning outcomes is mediated by the use of specific monitoring processes and; (2) The extent to which the relationship between prior domain knowledge and hypermedia learning outcomes is mediated by the use of specific monitoring processes. Participants included 68 education majors. A self-report questionnaire was used to measure self-efficacy, a pretest was used to measure prior domain knowledge, a posttest was used to measure learning outcomes, and a think-aloud protocol were used to identify the deployment of monitoring processes during a 30-min hypermedia learning task. Results indicated that the relationship between self-efficacy and specific monitoring processes ("Monitoring Understanding", "Monitoring Environment", and "Monitoring Progress Towards Goals") was significantly detectable. Additionally, the relationship between prior domain knowledge and "Monitoring Understanding" was significantly detectable. Lastly, regression analyses revealed that the relationship between self-efficacy and hypermedia learning outcomes was mediated by the extent to which participants monitored their understanding and the environment.

New literacies researchers have identified a core set of strategies for locating information, one of which is "reading a Web page to locate information that might be present there" (Leu et al. in: Rush, Eakle, Berger (eds) "Secondary school reading and writing: What research reveals for classroom practices," 2007, p. 46). Do middle-school, high school, and undergraduate students (N = 51) differ in effectiveness at locating information within extended hypermedia? Students completed a pretest measure of knowledge about the circulatory system. They then gave verbal answers to 10 researcher-developed questions about the circulatory system, which they answered by searching the environment and thinking aloud about the task. Consistent with large-scale national and international studies, students were only moderately successful at locating information. Successfully locating information was significantly associated with having more prior knowledge, efficient searching, and giving better quality answers to the researcher-posed questions. It was also associated with specific strategies only at the level of individual questions. That is, the "ideal" strategy depended on the question and how the answer was phrased in the text. Implications of the results for teaching students how to search in hypermedia are offered.

In this study, we used think-aloud verbal protocols to examine how various macro-level processes of self-regulated learning (SRL; e.g., planning, monitoring, strategy use, handling of task difficulty and demands) were associated with the acquisition of a sophisticated mental model of a complex biological system. Numerous studies examine how specific micro-level SRL processes such as judgments of learning or prior knowledge activation are related to learning outcomes. However, it is also valuable to look at these processes in macro-level aggregates because efficacy and use of micro-level strategies can vary due to individual differences. Two hundred and nineteen high-school and middle-school students produced think-aloud protocols while learning with a hypermedia environment. We transcribed and coded participants' learning sessions for the use of micro- and macro-level SRL processes. Participants' developmental level, prior knowledge, and monitoring behaviors were associated with posttest mental model sophistication. These results illustrate that monitoring is a key SRL process when developing an understanding of a complex science topic using hypermedia. (Contains 2 tables.)

We propose an integration of aspects of several developmental and systems of beliefs models of personal epistemology. Qualitatively different positions, including realism, dogmatism, skepticism, and rationalism, are characterized according to individuals' beliefs across three dimensions in a model of epistemic and ontological cognition. This model incorporates ideas from philosophical epistemology, including a focus on students' multiple means of justification. Concerns regarding the psychometric qualities of quantitative measures of personal epistemology inform this model and its proposed operationalization. We suggest a means of statistical analysis that can be used to assess the construct and predictive validity of this conceptual model by testing the relations among positions, dimensions, and covariates of interest. Future research directions include investigating how individuals choose among various forms of justification. (Contains 1 table.)

We collected think-aloud, pre-test, post-test, and motivation data from 43 undergraduates to examine the impact of conceptual scaffolds on the fluctuation of certain motivation constructs and use of self-regulatory processes during learning with hypermedia. Participants were randomly assigned to either the No Scaffolding (NS) or Conceptual Scaffolding (CS) condition. During the experimental session, each participant individually completed a pre-test on the circulatory system, a pre-task motivation questionnaire, one 30-min hypermedia learning task during which they learned about the circulatory system, a motivation questionnaire at three regular intervals during this learning task, a post-test on the circulatory system, and a post-task motivation questionnaire. Results indicated that while participants in both conditions gained declarative knowledge, participants who received conceptual scaffolds during learning demonstrated deeper understanding of the circulatory system on the post-test. In terms of self-regulatory processes, the results indicated that participants in the CS condition used significantly more planning processes during learning than participants in the NS condition. Additionally, participants in both conditions significantly decreased their use of strategies as they progressed through the learning task. Regarding motivation while learning with hypermedia, results indicated that participants in both conditions reported significantly increased levels of interest as they progressed through the learning task. Furthermore, participants in the CS condition reported the task as being easier and putting forth less effort than participants in the NS condition.

Research involving gifted and grade-level students has shown that they display differences in their knowledge of self-regulatory strategies. However, little research exists regarding whether these students differ in their actual use of these strategies. This study aimed to address this question by examining think-aloud data collected from 98 gifted and grade-level students engaging in a complex learning task: utilizing a hypermedia environment to learn about the circulatory system. We also examined both declarative knowledge and mental model measures of learning to determine whether these groups differed in their actual performance. Our results show that gifted students did outperform grade-level students in all outcome measures. In addition, gifted students more often utilized more sophisticated self-regulatory strategies (e.g. coordinating informational sources) than grade-level students. Grade-level students were more likely to use less effective strategies that are less likely to promote the acquisition of knowledge (e.g. mnemonics). Recommendations for future intervention studies are based upon these findings.

Think-aloud and pre-test data were collected from 49 undergraduates with varying levels of prior domain knowledge to examine the relationship between prior domain knowledge and self-regulated learning with hypermedia. During the experimental session, each participant individually completed a pretest on the circulatory system, and then one 40-min hypermedia learning task during which he or she learned about the circulatory system. Think-aloud data were collected during the 40-min learning task to measure each participant's use of specific self-regulated learning processes related to planning, monitoring, and strategy use. Results indicate that prior domain knowledge is significantly related to how the participants self-regulated their learning during the 40-min learning task with hypermedia. Specifically, prior domain knowledge is positively related to participants' monitoring and planning and negatively related to their use of strategies during the hypermedia learning task.

We examined how self-regulated learning (SRL) and externally-facilitated self-regulated learning (ERL) differentially affected adolescents' learning about the circulatory system while using hypermedia. A total of 128 middle-school and high school students with little prior knowledge of the topic were randomly assigned to either the SRL or ERL condition. Learners in the SRL condition regulated their own learning, while learners in the ERL condition had access to a human tutor who facilitated their self-regulated learning. We converged "product" (pretest-posttest shifts in students' mental models and declarative knowledge measures) with "process" (think-aloud protocols) data to examine the effectiveness of self- versus externally-facilitated regulated learning. Findings revealed that learners in the ERL condition gained statistically significantly more declarative knowledge and that a greater number of participants in this condition displayed a more advanced mental model on the posttest. Verbal protocol data indicated that learners in the ERL condition regulated their learning by activating prior knowledge, engaging in several monitoring activities, deploying several effective strategies, and engaging in adaptive help-seeking. By contrast, learners in the SRL condition used ineffective strategies and engaged in fewer monitoring activities. Based on these findings, we present design principles for adaptive hypermedia learning environments, engineered to foster students' self-regulated learning about complex and challenging science topics.