Search

05 maart 2023, Item does not contain fulltext, Background: Many learners struggle to productively self-regulate their learning. To support the learners' self-regulated learning (SRL) and boost their achievement, it is essential to understand the cognitive and metacognitive processes that underlie SRL. To measure these processes, contemporary SRL researchers have largely utilized think aloud or trace data, however, not without challenges. Objectives: In this paper, we present the findings of a study that investigated how concurrent analysis and integration of think aloud and trace data could advance the measurement of SRL and assist in better understanding the mechanisms of SRL processes, especially those details that remain obscured by observing each data channel individually. Methods: We concurrently collected think aloud and trace data generated by 44 university students in a laboratory setting and analysed those data relative to the same timeline. Results: We found that the two data channels could be interchangeably used to measure SRL processes for only 17.18% of all the time segments identified in a learning task. Moreover, SRL processes for around 45% of all the time segments could be detected via either trace data or think aloud data. For another 27.17% of all the time segments, different SRL processes were detected in both data channels. Conclusions Our results largely suggest that the two data collection methods can be used to complement each other in measuring SRL. In particular, we found that think aloud and trace data could provide different perspectives on SRL. The integration of the two methods further allowed us to reveal a more complex and more comprehensive temporal associations among SRL processes compared to using a single data collection method. In future research, the integrated measurement of SRL can be used to improve the detection of SRL processes and provide a fuller picture of SRL.

Contains fulltext : 296280.pdf (Publisher’s version ) (Open Access), Self-Regulated Learning (SRL) is related to increased learning performance. Scaffolding learners in their SRL activities in a computer-based learning environment can help to improve learning outcomes, because students do not always regulate their learning spontaneously. Based on theoretical assumptions, scaffolds should be continuously adaptive and personalized to students' ongoing learning progress in order to promote SRL. The present study aimed to investigate the effects of analytics-based personalized scaffolds, facilitated by a rule-based artificial intelligence (AI) system, on students' learning process and outcomes by real-time measurement and support of SRL using trace data. Using a pre-post experimental design, students received personalized scaffolds (n = 36), generalized scaffolds (n = 32), or no scaffolds (n = 30) during learning. Findings indicated that personalized scaffolds induced more SRL activities, but no effects were found on learning outcomes. Process models indicated large similarities in the temporal structure of learning activities between groups which may explain why no group differences in learning performance were observed. In conclusion, analytics-based personalized scaffolds informed by students’ real-time SRL measured and supported with AI are a first step towards adaptive SRL supports incorporating artificial intelligence that has to be further developed in future research.

Contains fulltext : 295676.pdf (Publisher’s version ) (Open Access), Self-regulated learning (SRL) is the ability to regulate cognitive, metacognitive, motivational, and emotional states while learning and is posited to be a strong predictor of academic success. It is therefore important to provide learners with effective instructions to promote more meaningful and effective SRL processes. One way to implement SRL instructions is through providing real-time SRL scaffolding while learners engage with a task. However, previous studies have tended to focus on fixed scaffolding rather than adaptive scaffolding that is tailored to student actions. Studies that have investigated adaptive scaffolding have not adequately distinguished between the effects of adaptive and fixed scaffolding compared to a control condition. Moreover, previous studies have tended to investigate the effects of scaffolding at the task level rather than shorter time segments - obscuring the impact of individual scaffolds on SRL processes. To address these gaps, we (a) collected trace data about student activities while working on a multi-source writing task and (b) analyzed these data using a cutting-edge learning analytic technique - ordered network analysis (ONA) - to model, visualize, and explain how learners' SRL processes changed in relation to the scaffolds. At the task level, our results suggest that learners who received adaptive scaffolding have significantly different patterns of SRL processes compared to the fixed scaffolding and control conditions. While not significantly different, our results at the task segment level suggest that adaptive scaffolding is associated with earlier engagement in SRL processes. At both the task level and task segment level, those who received adaptive scaffolding, compared to the other conditions, exhibited more task-guided learning processes such as referring to task instructions and rubrics in relation to their reading and writing. This study not only deepens our understanding of the effects of scaffolding at different le

Item does not contain fulltext, Self-regulation is an essential skill for lifelong learning. Research has shown that self-regulated learning (SRL) leads to greater academic achievement and sustainable education, but students often struggle with SRL. Scaffolds are widely reported as an effective and efficient support method for SRL. To further improve digital scaffolds’ effectiveness, real-time detection of learning behavior can be used to personalize scaffolds. Therefore, the present study aimed to inform the field of scaffolding SRL by reporting on the design and evaluation of digital scaffolds. We present decisions made during the design process of personalized scaffolds to inform future scaffold designs. We evaluated how scaffolds were personalized based on real-time detection SRL, how university students respond to the scaffolds (i.e., compliance), and how this response is related to learning outcomes (i.e., quality of an essay). The research design was a pre-posttest with three conditions (no, generalized (same for all), or personalized scaffolds). A 45-minute reading and writing task was used, during which SRL processes were recorded in real-time. Findings revealed that different real-time SRL processes could be used to personalize scaffolds, meaning that we were able to personalize the content of scaffold based on students' actual learning behavior. In addition, students in the personalized condition complied more with the scaffolds than students in the other conditions. This compliance with the scaffolds was generally associated with better learning outcomes. To conclude, our approach showed how design decisions could be evaluated and provided insight into the personalization of scaffolds., 08 november 2023, 28 p.

Item does not contain fulltext, Self-regulated learning (SRL) is an essential skill for achieving one's learning goals, particularly in Digital Learning Environments (DLEs) where system support is often limited compared to traditional classroom settings. However, research has found that learners often struggle to adapt their behaviour to the self-regulatory demands of DLEs. Furthermore, existing SRL analysis tools have limited utility for real-time or individualized prescriptive support of a learner''s SRL strategy during a study session. In response to these challenges, we propose a novel approach using reinforcement learning as a framework to optimize the sequence of SRL processes for a learning task. This framework allows us to model and optimize the SRL strategy as a sequential decision-making problem, where each decision corresponds to an SRL process. The goal is to find an optimal sequence of decisions that maximizes performance in learning outcomes such as assessment score or learning gains. We compare the performance of our reinforcement learning framework with other sequential machine learning tools, such as Long Short-Term Memory (LSTM) neural networks and Genetic Algorithms (GA). The results of our study show that our reinforcement learning model outperforms GA and LSTM models in optimizing SRL strategy. The contributions of this work can facilitate the development of a tool which can detect sub-optimal SRL strategy in real-time and enable individualized SRL focused scaffolding., 14 p.

06 juni 2022, Item does not contain fulltext, In recent years, unobtrusive measures of self-regulated learning (SRL) processes based on log data recorded by digital learning environments have attracted increasing attention. However, researchers have also recognised that simple navigational log data or time spent on pages are often not fine-grained enough to study complex SRL processes. Recent advances in data-capturing technologies enabled researchers to go beyond simple navigational logs to measure SRL processes with multi-channel data. What multi-channel data can reveal about SRL processes, and to what extent can the addition of peripheral and eye-tracking data with navigational log data change and improve the measurement of SRL are key questions that require further investigation. Hence, we conducted a study and collected learning trace data generated by 25 university students in a laboratory setting, that aimed to address this problem by enhancing navigational log data with peripheral and eye-tracking data. We developed a trace-based measurement protocol of SRL, which interpreted raw trace data from multi-channel data into SRL processes. Specifically, the study compared the frequency and duration of SRL processes detected, how much duration and times of occurrences of the detected SRL processes were affected or refined. We also used a process mining technique to analyses how temporal sequencing of the detected SRL processes changed by enriching navigational log data with peripheral and eye-tracking data. The results revealed that by adding new data channels, we improved the capture of learning actions and detected SRL processes while enhancing the granularity of the measurement. In comparison to the use of navigational logs only, the completeness of temporal sequencing relationships between SRL processes with multi-channel data improved. In addition, we concluded that eye-tracking data is valuable for measuring and extracting SRL processes, and it should receive more attention in the future.

LAK22: 12th International Learning Analytics and Knowledge Conference (Online, USA, March 21-25, 2022), Item does not contain fulltext, Self-regulated learning (SRL) skills are essential for successful learning in a technology-enhanced learning environment. Learning Analytics techniques have shown a great potential in identifying and exploring SRL strategies from trace data in various learning environments. However, these strategies have been mainly identified through analysis of sequences of learning actions, and thus interpretation of the strategies is heavily task and context dependent. Further, little research has been done on the association of SRL strategies with different influencing factors or conditions. To address these gaps, we propose an analytic method for detecting SRL strategies from theoretically supported SRL processes and applied the method to a dataset collected from a multi-source writing task. The detected SRL strategies were explored in terms of their association with the learning outcome, internal conditions (prior-knowledge, metacognitive knowledge and motivation) and external conditions (scaffolding). The study results showed our analytic method successfully identified three theoretically meaningful SRL strategies. The study results revealed small effect size in the association between the internal conditions and the identified SRL strategies, but revealed a moderate effect size in the association between external conditions and the SRL strategy use.

04 mei 2022, Item does not contain fulltext, Contemporary research that looks at self-regulated learning (SRL) as processes of learning events derived from trace data has attracted increasing interest over the past decade. However, limited research has been conducted that looks into the validity of trace-based measurement protocols. In order to fill this gap in the literature, we propose a novel validation approach that combines theory-driven and data-driven perspectives to increase the validity of interpretations of SRL processes extracted from trace-data. The main contribution of this approach consists of three alignments between trace data and think aloud data to improve measurement validity. In addition, we define the match rate between SRL processes extracted from trace data and think aloud as a quantitative indicator together with other three indicators (sensitivity, specificity and trace coverage), to evaluate the "degree" of validity. We tested this validation approach in a laboratory study that involved 44 learners who learned individually about the topic of artificial intelligence in education with the use of a technology-enhanced learning environment for 45 minutes. Following this new validation approach, we achieved an improved match rate between SRL processes extracted from trace-data and think aloud data (training set: 54.24%; testing set: 55.09%) compared to the match rate before applying the validation approach (training set: 38.97%; test set: 34.54%). By considering think aloud data as "reference point", this improvement of the match rate quantified the extent to which validity can be improved by using our validation approach. In conclusion, the novel validation approach presented in this study used both empirical evidence from think aloud data and rationale from our theoretical framework of SRL, which now, allows testing and improvement of the validity of trace-based SRL measurements.

13 december 2022, Item does not contain fulltext, Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while developing the product. However, formative feedback in writing tasks based on multiple texts has almost exclusively focused on essay product and rarely included SRL processes. Objectives We explored the viability of using product and process features to develop machine learning classifiers that identify low- and high-performing essays in a multi-text writing task. Methods: We examined learning processes and essay submissions of 163 graduate students working on an authentic multi-text writing assignment. We utilised learners' trace data to obtain process features and state-of-the-art natural language processing methods to obtain product features for our classifiers. Results and Conclusions: Of four popular classifiers examined in this study, Random Forest achieved the best performance (accuracy = 0.80 and recall = 0.77). The analysis of important features identified in the Random Forest classification model revealed one product (coverage of reading topics) and three process (elaboration/organisation, re-reading and planning) features as important predictors of writing quality. Major Takeaways The classifier can be used as a part of a future automated writing evaluation system that will support at scale formative assessment in writing tasks based on multiple texts in different courses. Based on important predictors of essay performance, a guidance can be tailored to learners at the outset of a multi-text writing task to help them do well in the task.

19th Biennial EARLI Conference for Research on Learning and Instruction (23-27 Aug 2021), Item does not contain fulltext