Search

Background: Learner profiles detected from digital trace data are typically triangulated with survey data to explain those profiles based on learners' internal conditions (e.g., motivation). However, survey data are often analysed with limited consideration of the interconnected nature of learners' internal conditions. Objectives: Aiming to enable a thorough understanding of trace‐based learner profiles, this paper presents and evaluates a comprehensive approach to analysis of learners' self‐reports, which extends conventional statistical methods with psychological networks analysis. Methods: The study context is a massive open online course (MOOC) aimed at promoting physical activity (PA) for health. Learners' (N = 497) perceptions related to PA, as well as their self‐efficacy and intentions to increase the level of PA were collected before and after the MOOC, while their interactions with the course were logged as digital traces. Learner profiles derived from trace data were further examined and interpreted through a combined use of conventional statistical methods and psychological networks analysis. Results and Conclusions: The inclusion of psychological networks in the analysis of learners' self‐reports collected before the start of the MOOC offers better understanding of trace‐based learner profiles, compared to the conventional statistical analysis only. Likewise, the combined use of conventional statistical methods and psychological networks in the analysis of learners' self‐reports before and after the MOOC provided more comprehensive insights about changes in the constructs measured in each learner profile. Major Takeaways: The combined use of conventional statistical methods and psychological networks presented in this paper sets a path for a comprehensive analysis of survey data. The insights it offers complement the information about learner profiles derived from trace data, thus allowing for a more thorough understanding of learners' course engagement than any individual method or data source would allow. Lay Description: What is already known about this topic: Researchers have made extensive use of data from online learning platforms—often referred to as learning trace data—to understand how learners engage in online learning activities.Surveys are often used to gather information about learners' motivation, perceptions, and other internal factors, to complement the insight gleaned from learning traces and thus create a more complete picture of the learning behaviour.When analysing survey data, the focus is usually on individual factors without considering how different factors are connected and affect one another.Psychological networks analysis is a novel analytic approach to studying complex phenomena both in psychology and education. What this paper adds: The paper uses psychological networks as a method for examining mutual relations among learners' internal factors (e.g., motivation, perceptions) measured through self‐reports before and after a course.By combining conventional statistical analysis of self‐reports with psychological networks analysis, the paper develops a comprehensive picture of learners' internal factors measured through self‐reports, one that accounts for both individual and interconnected nature of those factors.This comprehensive approach to information extraction from surveys allows for better understanding of trace‐based learner profiles, that is, profiles mined from data about learners' interaction with online learning activities. Implications for practice and/or policy: The presented method paves a way for a comprehensive analysis of the data collected from learners through surveys.It helps us gain a better understanding of learners' engagement with online learning activities.It can advance the evaluation of digital educational programs that aim to encourage changes in health‐related behaviour. [ABSTRACT FROM AUTHOR]

Introduction: Media comparison studies examining the effectiveness of immersive virtual reality in education have yielded inconclusive findings, leaving the question of its impact on learning compared to conventional media unanswered. To address this issue, our study employs a novel approach that combines media comparison with an investigation on the influence of broader educational context. Methods: In the experiment, 262 students from two distinct types of lower secondary schools—comprehensive school and multi‐year gymnasium (a selective academic school)—participated. The students received a lesson on topography and contour line interpretation, using either an immersive virtual environment or a PowerPoint slideshow. A transfer test was carried out before, immediately after, and 1 month after the lesson to measure knowledge application. Results: The impact of the media (immersive virtual reality vs. PowerPoint slideshow) on learning outcomes was found to be minimal, with no clear advantage of one over the other in any experimental condition. In contrast, a significant influence on learning gains was observed due to school type. Multi‐year gymnasium students consistently outperformed comprehensive school students in the pre‐test and demonstrated greater learning gains, regardless of the learning media. Conclusion: The present study employs an innovative approach by integrating a comparative analysis of different media types with an investigation into the impact of learner characteristics. By including students from distinct types of schools, the study provides insights into the differential effects of immersive virtual reality in varied educational contexts. Lay Description: What is already known about this topic: Media comparison studies typically assess the effectiveness of immersive virtual reality as a learning medium in comparison to other instructional methods in education.The findings of these studies differ markedly.Numerous factors impact learning outcomes in an immersive virtual environment. What this paper adds: Recommendation that research in the field should explore not only if but also when and how IVR learning is effective.Evidence that learning achievements may not be affected by the medium used.Evidence that the type of school can significantly influence learning gains.Evidence that cognitive dispositions, such as spatial abilities, can be another factor that influence learning outcomes. Implications for practice: A technology‐centred approach alone does not allow for a proper evaluation of the learning gain potential of IVR, learner characteristics and broader educational context should also be taken into account.IVR as a learning medium may not be appropriate for all students and every educational setting.To ensure reliable evaluation of learning gains, it is necessary to align the content and format of the intervention in both learning mediums as closely as possible.Measuring learning gains not only immediately after the intervention but also with a significant time delay can contribute to a more comprehensive understanding. [ABSTRACT FROM AUTHOR]

Background: Skill integration is vital in students' mastery development and is especially prominent in developing code tracing skills which are foundational to programming, an increasingly important area in the current STEM education. However, instructional design to support skill integration in learning technologies has been limited. Objectives: The current work presents the development and empirical evaluation of instructional design targeting students' difficulties in code tracing particularly in integrating component skills in the Trace Table Tutor (T3), an intelligent tutoring system. Methods: Beyond the instructional features of active learning, step‐level support, and individualized problem selection of intelligent tutoring systems (ITS), the instructional design of T3 (e.g., hints, problem types, problem selection) was optimized to target skill integration based on a domain model where integrative skills were represented as combinations of component skills. We conducted an experimental study in a university‐level introductory Python programming course and obtained three findings. Results and Conclusions: First, the instructional features of the ITS technology support effective learning of code tracing, as evidenced by significant learning gains (medium‐to‐large effect sizes). Second, performance data supports the existence of integrative skills beyond component skills. Third, an instructional design focused on integrative skills yields learning benefits beyond a design without such focus, such as improving performance efficiency (medium‐to‐large effect sizes). Major Takeaways: Our work demonstrates the value of designing for skill integration in learning technologies and the effectiveness of the ITS technology for computing education, as well as provides general implications for designing learning technologies to foster robust learning. Lay Description: What is currently known about the subject matter?: Skill integration is vital in students' mastery development, and is especially prominent in developing coding tracing skills.Coding tracing skills are foundational to programming, an increasingly important area in the current STEM education, yet many novice students struggle with code tracing.Instructional design to support skill integration in learning technologies has been limited.One of the most effective technologies for skill mastery, intelligent tutoring systems (ITS), is underused or only used in a partial form in programming education. What our paper adds to this?: We present the development and empirical evaluation of instructional design targeting students' difficulties in code tracing particularly in integrating component skills in an ITS, the Trace Table Tutor.The instructional features of the ITS technology (e.g., active learning, step‐level support, individualized problem selection) support effective learning of code tracing.Students' performance data supports the existence of integrative skills beyond component skills.An instructional design that provides deliberate practice and focused practice on integrative skills yields learning benefits beyond a design without such features. The implications of study findings for practitioners: Designing for skill integration is valuable and should receive more attention in designing learning technologies to foster robust learning.The instructional features of full‐scale ITSs are effective in supporting learning and merit further application in learning technologies in computing education. [ABSTRACT FROM AUTHOR]

Background: The coronavirus pandemic has affected education worldwide. Estonian schools moved to distance learning on March 16. Objectives: This paper outlines the stages of unexpected and temporary changes affecting different stakeholders based on activity and content of the messages in the Facebook group 'Homeschooling with technology'. Also the applicability of the concerns‐based adoption model was explored to explain these changes. Methods: The investigated timespan includes the week before the emergency situation and the weeks of the school term and school holidays during the emergency situation, with 52 days in total, divided into seven periods. The 872 messages were coded by two researchers and changes were examined. Results and Conclusions: Four stages were identified, from low exposure and involvement with the change to focusing on collaboration and skipping the consequence stage. Furthermore, the new changes in the educational system might focus people again on personal concerns. Implications: The results help capture the changes in teaching and learning situation caused by the coronavirus pandemic in order to learn from these changes and be prepared for such situations in the future. Lay Description: What is already known about this topicIn the emergency situation, even well‐experienced teachers were not sure what and how to teach.There are several models and there are different models on handling the changes in education.Concerns‐based adoption model (CBAM) looks at how change affects people.Emotions have changed from negative to neutral in social media during school closures What this paper addsPrevious studies have been based mostly on surveys, but this paper adds an in‐depth analysis of messages from social media.Four stages describing how changes affect people have identified.New changes brought back personal concerns and no impact concerns raised.CBAM model is applicable in explaining unexpected temporary changes Implications for practice and/or policyIn teacher training, curricula courses about distance learning methodologies and tools should be added.National guidelines about the management of distance learning situations should be prepared. [ABSTRACT FROM AUTHOR]

Serious games have proven to be a powerful tool in education to engage, motivate, and help students learn. However, the change in student knowledge after playing games is usually measured with traditional (paper) prequestionnaires–postquestionnaires. We propose a combination of game learning analytics and data mining techniques to predict knowledge change based on in‐game student interactions. We have tested this approach in a case study for which we have conducted preexperiments–postexperiments with 227 students playing a previously validated serious game on first aid techniques. We collected student interaction data while students played, using a game learning analytics infrastructure and the standard data format Experience API for Serious Games. After data collection, we developed and tested prediction models to determine whether knowledge, given as posttest results, can be accurately predicted. Additionally, we compared models both with and without pretest information to determine the importance of previous knowledge when predicting postgame knowledge. The high accuracy of the obtained prediction models suggests that serious games can be used not only to teach but also to measure knowledge acquisition after playing. This will simplify serious games application for educational settings and especially in the classroom easing teachers' evaluation tasks. Lay Description: What is currently known about the subject matter Serious games are a powerful tool to engage, motivate, and help students learn.Pre‐post experiments are commonly used to measure knowledge acquisition.Game learning analytics can be applied to interaction data from games. What this paper adds We present a two‐step approach combining game learning analytics and data mining to predict players' performance in serious games based on their interactions.The approach is tested in a case study with pre‐post experiments collecting interaction data with 227 students playing a serious game to determine if performance can be accurately predicted.The comparison of prediction models has helped to determine if pretest information is essential.The highly accurate prediction models obtained suggest that games can be used to teach and measure knowledge acquisition after playing. Implications of study findings for practitioners The approach aims to simplify the measurement of players' learning with serious games.It may be generalized at least to similar scenarios (e.g., games for procedural learning or game‐likesimulations) where similar interaction data are feasible.Game mechanics and educational design should define the interaction data to capture.Using an accepted standard tracking profile is a clear recommendation. [ABSTRACT FROM AUTHOR]

Background: There is a lack of critical or empirical work interrogating the nature and purpose of evaluating technology use in education. Objectives: In this study, we examine the values underpinning the evaluation of technology use in education through field specialist perceptions. The study also poses critical reflections about the rigour of evaluation instruments development in the educational technology field. Method: A total of 48 domain specialists were surveyed to investigate the face and content validity of 39 items under eight constructs, with both qualitative and quantitative data from the survey analysed. Results: There was an alignment between the constructs that the specialists felt were important and the constructs that have typically been the focus of empirical studies, with field specialists indicating high relevance scores for technology (M = 3.24/4), learning outcomes (M = 3.20/4), affective elements (M = 3.19/4), behaviour (M = 3.15/4), presence/community (M = 3.07/4), teaching/pedagogy (M = 3.01/4), design (M = 2.96/4), and institutional environment (M = 2.86/4). Only a minority of other studies were found to perform face and content validity checks and even then only with small samples of respondents (usually n ≤ 5). Implications: Specialists in educational technology research confirm that all eight dimensions are important to consider when evaluating the use of technology in education. Thorough face and content validity processes should be adopted when developing educational technology evaluation instruments. Further work has validated an eight factor 28 item instrument for evaluating the use of technology in education using a large sample of students from a global open learning online course. Lay Description: What is already known about this topic: Comprehensive evaluation of technology use is essential to make decisions on learning technology strategies.There is a lack of critical work interrogating how technology use is evaluated in education.There is a need for developing evaluation instruments in the educational technology field. What this paper adds: An examination of 48 field specialist perceptions of important constructs when evaluating technology use in education.Field specialists agreed that eight constructs identified in empirical studies were relevant to educational technology evaluation.Evidence of limited face and content validation processes based on previous empirical work.A comprehensive 39‐item instrument for evaluating technology use in education. Implications of study findings for practitioners: Adopt a broad range of constructs when evaluating the use of technology in education.Apply face and content validation processes involving a large sample of experts.Consider using the 39‐item instrument to evaluate educational technology use. [ABSTRACT FROM AUTHOR]

Retrieval practice promotes retention more than restudying (i.e., the testing effect) and is applied to many educational settings. However, little research has investigated means to enhance this effect in educational settings. Theoretical accounts assume retrieval practice to be the most effective whenever retrieval is difficult but successful. Therefore, we developed a novel retrieval practice procedure, which adapts to learners' abilities and can be applied irrespective of learning content. This adaptive procedure aims to make retrieval gradually easier whenever students provide an incorrect answer. In a field experiment, students read book chapters as part of a weekly university course. In three consecutive weeks, they then practiced reading assignments by (a) adaptive testing, (b) non‐adaptive testing and (c) restudy. In Week 4, a surprise criterial test took place. Restudy outperformed both testing conditions, whereas adaptive testing performed equally well as non‐adaptive testing. However, exploratory analyses revealed that with increasing retention intervals, the superiority of restudy disappeared. Furthermore, whenever participants fully read the assignments and retention intervals increased, adaptive testing outperformed non‐adaptive testing. In sum, adaptive retrieval practice did not prove to be generally superior, but retention interval and students' preparation for class might be conditions rendering adaptive retrieval useful in educational settings. Lay Description: What is already known about this topic: Testing is a commonly used technique to assess progress in computer‐assisted learning environmentsStudents often benefit from the testing effect, that is practicing retrieval of information (testing) from memory benefits long‐term retention compared to restudyingCurrently, empirical evidence how to enhance the benefits of this technique in educational contexts is scarce What this paper adds: We propose a new approach to increase the benefit of the testing effect: For each question in retrieval practice the individual ability to retrieve this information from memory is consideredTo do so, cue informativeness of retrieval cues adapts to the individual ability level and thus makes the tests cognitively effortful but solvableThis adaptive approach is put to test in a higher education context and by using multiple‐choice questionsIn sum, adaptive retrieval practice did not prove to be generally superior to non‐adaptive retrieval practice or restudy However, retention interval and students' preparation for class might be conditions rendering adaptive retrieval useful in educational settings Implications for practice and/or policy: Findings can be implemented in educational contexts when multiple‐choice questions are used to enhance learningLearning can be optimized in computer‐assisted learning environments that allow adaptive testingWe advise adaptive retrieval only when students are well prepared for practice and when time between practice and final assessment exceeds two weeks [ABSTRACT FROM AUTHOR]

Online search involves multitasking and may demand better working‐memory capacities (WMC) and additional cognitive aids. Given the constraints of human cognition, we tested the effectiveness of note‐taking strategies on university students' online search performance. Also examined were the profile configurations of WMC tests in silence and in irrelevant speech as well as its interaction effects with note‐taking strategies on online search performance. Among the participants of 60 university students, we found a four‐group WMC profile composition that differed significantly across WMC scores in both silence and irrelevant speech. Results showed that WMC profiles, note‐taking groups and their interaction effect were significant factors on online search performance. Both Free Note and Matrix Note were beneficial for the students in low silent WMC profile groups. Study results have both theoretical and practical implications, advancing our understanding of the interrelationship of WMC under different conditions and informing instructional practice of online search performance. Lay Description: What is already known about this topicWorking memory capacity is fundamental to multitasking ability.Online search is a typical multitasking behavior that requires constant switching between different information sources.Students who take note perform better than those who do not while reading materials with seductive but irrelevant information.What this paper addsStudents working memory capacity is examined under silent and irrelevant speech conditions.Students exhibited different profile patterns in their working memory capacity under silent and irrelevant speech conditions.Taking note, either in matrix form or free form, can significantly improve students' online search performance for those with high WMC ONLY in irrelevant speech or for those with low WMC in both silent and irrelevant speech conditions.Implications for practiceNote‐taking as auxiliary support may facilitate students' integration and evaluation of the online information across multiple sources during their online search.Instructions on how to self‐construct matrix note are needed to help students develop advanced note‐taking skills for their learning involving online search. [ABSTRACT FROM AUTHOR]

Emotions are critical to learning. However, the function of emotions in the emerging context of massive open online courses (MOOCs) has been under‐researched. The present study complemented this line of research by modelling the relation between learner emotions, engagement with videos, engagement with assignments, and self‐perceived achievement in a small private online course—a localized instance of an extended MOOC. The results suggested that whereas enjoyment, excitement, boredom, and annoyance were all significant predictors of video engagement, only excitement and annoyance were significant predictors of assignment engagement. Excitement, in particular, was the strongest or stronger predictor of both types of engagement. In addition, both types of engagement predicted self‐perceived achievement, but video engagement predicted self‐perceived achievement via the mediation of assignment engagement. Implications for designing more effective MOOCs were proposed. Lay Description: What is currently known about the subject Emotions predict engagement in learning settings other than MOOCs.Engagement predicts achievement in learning settings including MOOCs.Assignment engagement predicts achievement in MOOCs better than video engagement. What this paper adds The two types of engagement are predicted by different sets of emotions.Engagement predicts self‐perceived achievement in the SPOC.Video engagement predicts self‐perceived achievement via the mediation of assignment engagement.The majority of systematic reviews did not evaluate the quality of the primary papers which they examined. Implications for practitioners MOOC researchers should pay more attention to the design of assignments.MOOC developers should try to build more emotionally‐sound courses and platforms. [ABSTRACT FROM AUTHOR]

Abstract: The aim of this paper is to provide insights into differences between pre‐service teachers based on the areas of technological pedagogical content knowledge (TPACK) and the areas of theory of planned behaviour (TPB) in the context of using information and communication technology in education. The target group consisted of 267 first‐year pre‐service teachers at 3 Finnish universities. Differences between pre‐service teachers were outlined using cluster analysis based on their knowledge of different TPACK areas (the respondents' weak and strong areas). To see how the TPB areas aligned with the 4 TPACK clusters, 1‐way analysis of variance was used. Statistically significant differences between the 4 clusters were found in all the TPB areas except subjective norms. These results provide insights into the differences among pre‐service teachers in the context of TPACK and the TPB. They also suggest that instead of considering pre‐service teachers as one homogenous group, we need to understand the variations among their abilities and knowledge in order to be able to provide them with support they need within teacher education. [ABSTRACT FROM AUTHOR]

This study compared higher education dental undergraduate student performance in online assessments with performance in traditional paper-based tests and investigated students' perceptions of the fairness and acceptability of online tests, and showed performance to be comparable. The project design involved two parallel cross-over trials, one in Year 3 ( n = 132) of the dental curriculum, one in Year 5 ( n = 134). A total of 266 students undertook related tests of the same knowledge base in both online and traditional paper formats. Order of presentation of the test form was randomized. One hundred and thirty-two students in Year 3 additionally completed questions concerning their perceptions of the acceptability and fairness of the online examination process. Focus group discussions were held with groups of students ( n = 30) from both Year 3 and 5 to explore their perceptions of the online format. The online tests were created using the commercial software package MaxExam (Bristol, UK). The results were analysed using repeated measures analysis of variance (ANOVA) with the order of presentation being entered as a variable in the analysis. The results of the study showed that for both Years 3 and 5 students there was a high degree of consistency in performance between the two versions of the tests, although a minority of students performed better in the online version of the test. Over 70% of students rated the online test as acceptable, and 90% felt that the online format did not disadvantage them even in a 'high-stakes' situation. From these results, we conclude that online assessment does not confer any disadvantage to students. Students perceive such assessment to be both fair and acceptable, and to have some advantages over the traditional paper-based methods of assessment. [ABSTRACT FROM AUTHOR]