Search

Background: Digital‐first assessments leverage the affordances of technology in all elements of the assessment process: from design and development to score reporting and evaluation to create test taker‐centric assessments. Objectives: The goal of this paper is to describe the engineering, machine learning, and psychometric processes and technologies of a test security framework (part of a larger ecosystem; Burstein et al., 2021) that can be used to create systems that protect the integrity of test scores. Methods: We use the Duolingo English Test to exemplify the processes and technologies that are presented. This includes methods for actively detecting and deterring malicious behaviour (e.g., a custom desktop app). It also includes methods for passively detecting and deterring malicious behaviour (e.g., a large item bank created through automatic generation methods). We describe the certification process that each test administration undergoes, which includes both automated and human review. Additionally, we describe our quality assurance dashboard which leverages psychometric data mining techniques to monitor test quality and inform decisions about item pool maintenance. Results and Conclusions: As assessment developers transition to online delivery and to a design approach that places the test taker at the centre, it becomes increasingly important to take advantage of the tools and methodological advances in different fields (e.g., engineering, machine learning, psychometrics). These tools and methods are essential to maintaining the security of assessments so that the score reliability is sustained and the interpretations and uses of test scores remain valid. Lay Description: What is known about this topic?: As more and more testing programmes transition to test taker‐centric administrations, effective measures to prevent cheating and protect content are critical to ensure the validity and integrity of scores.Two of the most common forms of cheating in online testing are (a) having someone other than the person who has registered take the test, and (b) stealing content and providing it to others to assist them in achieving a higher score. What does this paper add?: In designing a test taker‐centric digital‐first assessment, a test security framework must inform decisions from end‐to‐end (i.e., registration, onboarding, communications regarding test taker behaviours, test preparation and practice, test administration, and post‐administrative activities including scoring).Security is safeguarded through active and passive design methods; active methods include having test takers attest that they will follow the rules governing testing and informing them that they will be videoed during testing; passive methods include a computer adaptive design that limits item exposure and test overlap rates, development of a large item pool using automated item generation, and applying artificial intelligence to review test administration videos to flag unauthorized behaviours for human review. Implications: With more educational and assessment programmes transitioning to online digital models, the paper presents a comprehensive review of security issues and identifies an integrated approach for preventing cheating and other unauthorized behaviours. [ABSTRACT FROM AUTHOR]

Background Study: The research discusses the need for a paradigm shift in engineering education current practices to accommodate the digital native students. The paper emphasizes the importance of integrating disruptive technologies, namely Virtual Reality (VR) through Head Mounted Displays VR (HMD VR) and Desktop Based VR (DB VR) and comparing it with the traditional teaching methods. The research focuses on second‐year undergraduate students enrolled in a concrete structures course. Objectives: The objective of the study is to analyse the impact of DB VR and HMD VR on students' motivation, performance and workload in comparison with traditional 2D drawings. The study also aims to investigate how simulation sickness and system usability could impact students' motivation and performance when interpreting concrete structures. Methods: The study divided second‐year civil engineering students into three groups, 2D, DB VR and HMD VR. The performance and motivation levels of students were assessed based on the number of mistakes identified and an intrinsic motivation survey. Furthermore, data on system usability and simulation sickness were collected. Spearman rank correlation was conducted to analyse the relationships between the variables tested. Results: The results have shown that students' motivation and performance were much higher when using VR in general and HMD VR specifically, students' efficiency and accuracy were also observed. On the other hand, usability was found to be moderately correlated with motivation, with students rating HMD VR as the most usable approach. A moderate inverse correlation was observed between simulation sickness and motivation, indicating that higher simulation sickness led to lower motivation levels. Conclusion: The study concludes that integrating VR in general, DB VR, or HMD VR into a concrete structures course for undergraduate students can significantly improve students' performance and motivation. Furthermore, enhancing usability of any of the methods used can also improve the performance and motivation. On the other hand, simulation sickness inversely affects motivation and performance, therefore, VR developers and educators should not neglect it. Lay Description: What is already known about the topic: Traditional teaching methods in engineering education require improvement.Integrating digital technologies into engineering education enhances performance.Digital technologies can potentially enhance motivation. What this paper adds: Compares traditional teaching methods for concrete structures interpretation with virtual reality.Establishes correlations between motivation, performance and usabilityIntroduces how desktop‐based VR and head‐mounted display VR can produce varying results. Implications for practice and/or policy: Decision‐makers can use the information to integrate updated pedagogy into engineering curricula.Allows for further exploration of different digital technologies that can be used for concrete structure interpretation.Allows instructors to introduce virtual reality in their courses. [ABSTRACT FROM AUTHOR]

Background: The substantial growth in gamification research has connected gamified learning to enhanced engagement, improved performance, and greater motivation. Similar to gamification, personalized learning analytics dashboards can enhance student engagement. Objectives: This study explores the student experiences and academic achievements using a gamified dashboard in a large, introductory STEM course. Methods: We examined two groups of students enrolled in different sections of a one‐semester‐long physical geology course with a total enrollment of 223 students. The only difference between the groups was that one had access to the dashboard. The data collection included students' assignments, overall performances, and exam scores. Students in both sections completed a Science Literacy Concept Inventory survey at the beginning and end of the term. Additionally, students completed an end‐of‐term survey containing open‐ended questions on their experience and interactions with specific elements. Results: Students shared mostly positive comments about their experience with the dashboard, and the final grade of students with access to the dashboard was 13% higher, on average, compared to their peers in the non‐dashboard section. Conclusion: With low costs and little time invested, gamified dashboards could have a significant impact on student performance in large STEM lecture courses. Lay Description: What is already known about this topic: Leaner dashboards that provide personalized feedback are linked to improved student performance in undergraduate STEM courses. However, students may be reticent to adopt dashboards, especially students in the middle range of the performance distribution, for whom the impact of the dashboard may be most significant.Prior research on gamification in STEM courses suggests that adding gameful elements like progress trackers, badges, and rewards can increase students' engagement in undergraduate coursework.The combination of personalized learning in dashboards and the motivational experience of gamification should boost student engagement, dashboard adoption, and ultimately foster academic achievement. What this paper adds: In this paper, we examine the differences in academic performance between students with and without access to a gamified learning dashboard in an undergraduate Geology course.We observe that students with access to the dashboard (the treatment group), outperform students without access (the control group). The greatest benefit is observed for students who use the dashboard more frequently.Students reported that the progress tracker, the badges, and the rewards all incentivized their use and provided useful feedback about their performance. Students who did not adopt the dashboard expressed a need for more detail about the tool and its purpose. Implications for practice and/or policy: Instructors and instructional designers for large enrollment undergraduate STEM courses should consider the affordances of gamified learner dashboards for increasing engagement and for providing synchronous feedback about learners' strategies in the course.Aligning the dashboard to the content of the curriculum—by ensuring that badges reflect key concepts‐ encouraged participation and helped students identify the utility of the tool. Dashboards should be personalized to individuals and to the course.Adding gameful elements in a low‐cost intervention for increasing adoption and engagement of dashboards in large enrollment introductory courses. [ABSTRACT FROM AUTHOR]

Background: Effective learning in computer programming courses has been a constant challenge for university teachers and has become a relevant competence for current professionals. The literature on gamification in learning presents mixed results, mainly due to problems in instructional design and inconsistency in gamification. Studies with positive empirical results on computer programming formative activities and showing under what conditions gamification is effective still need to be included. Objectives: This study aimed to quantify the effects of gamification techniques in software tools that support computer programming learning applied to data‐structures university students. The study focused on determining whether there are differences in learning when using gamification techniques in online exercising formative activities. The effects of gamification on learning objectives with diverse complexity are studied. Methods: The study was conducted on computer engineering university students following a pretest‐intervention‐posttest design. They were divided into control and experimental groups, where both used a bank of exercises in Moodle to support learning. The experimental group considered gamification techniques, including points, leaderboards, and badges. Results and Conclusions: Gamification techniques significantly outperformed in learning gains in exercising activities without gamification. Regarding learning complexity, increasing taxonomic complexity did not negatively affect gamification learning effects. Lay Description: What is currently known about the subject matter?: Gamification has been explored as a strategy to enhance learning motivation in various educational contexts, including computer programmingLiterature shows mixed quantitative results on learning improvements under gamification.Few studies cover formative exercises, deal with computer programming tasks, or use a LMS like Moodle.Lack of literature exploring the conditions for effective gamification use, and involving learning objective complexity. What does this paper add to the existing body of research?: Quantifies the impact of gamification techniques on learning outcomes, specifically focusing on computer‐programming in university.Provides empirical evidence on gamification significantly improving learning gains compared to non‐gamified data‐structures exercisesGamification is effective in tasks of diverse levels of taxonomic complexity, with consistent effectiveness regardless of learning difficulty. Implications of study findings for practitioners: Gamification techniques, can be used in higher education to enhance student engagement and learning outcomes in computer programmingThe gamification information (e.g. points and ranking) do not require an automatic update, can be processed manually and published daily.It is effective to use gamification for learning tasks of medium‐high difficulty in Moodle on computer programming topics.It is key a thoughtful instructional design when applying gamification, ensuring alignment with learning objectives and task complexity. [ABSTRACT FROM AUTHOR]

Background: Online learning has recently surged due to the COVID‐19 global pandemic. Because of the pandemic, many universities were forced to move to online learning, and various online teaching and learning environments emerged, both asynchronous and synchronous. Objective: This study explores how a large university in the Southeastern United States of America converted an in‐person Lean Manufacturing professional course into synchronous online learning for industry participants. Method: The study analysed the performance of 212 in‐person and 43 online industry participants and examined the views of online participants about the training. Paired t‐tests, one‐way multivariate analysis of covariance (MANCOVA), and univariate analysis of covariance (ANCOVA) were utilized to evaluate the performance of the in‐person and online groups on the three competencies (Lean culture, problem‐solving and Lean systems). Results and Conclusion: Both online and in‐person programs performed equally well in two Lean Manufacturing competencies: Lean systems and problem solving. The online program outperformed the in‐person program in the third competency, Lean culture. Overall, the participants had a positive experience with the structure and organization of the online program, and their impression of the online learning environment for the program was positive. The study findings indicated that Lean Manufacturing could be trained effectively online and in person. Best practices are suggested for the conversion of in‐person to synchronous online engineering continuing education programs. Lay Description: What is already known about this topic: During the COVID‐19 pandemic, many in‐person lectures were converted to online lectures for undergraduate students, and some online lectures satisfied the students.Many researchers reported that the online participants yielded positive results, and the performance of online participants was comparable to that of in‐person participants.However, the participants mentioned that some of the challenges of online learning include poor concentration, internet losses and poor understanding of the use of technology. What this paper adds: A description of how in‐person training can be converted to online training for adult learners is a unique context since they bring experience to the learning process.A comparison of the performance of adult learners trained using in‐person performed compared to those trained in‐person. The implications of study findings for practitioners: The study's results indicated that both the online and the traditional in‐person programs yield almost similar results; thus, online education programs can be used when in‐person is impossible.Online education programs can be used when in‐person is impossible, and the best practice guidelines drawn from the study can benefit similar engineering continuing education programs. [ABSTRACT FROM AUTHOR]

Background: Despite its obvious relevance to computer science, computational thinking (CT) is transdisciplinary with the potential of impacting one's analytical ability. Although countless efforts have been invested across K‐12 education, there is a paucity of research at the postsecondary level about the extent to which CT can contribute to sustainable learning outcomes. Objectives: The current study examines how a series of Arduino‐based robotics learning activities capture the fuller essence of concepts related to CT. Methods: College students (n = 50) completed a series of six robotics learning activities. Think‐alouds, student reflections and performance scores were used to assess students' CT through a robotics challenge in virtual and physical learning environments. Results and Conclusions: Students verbalized CT concepts related to algorithmic thinking much more than abstraction, problem decomposition and testing and debugging. Students exposed to active learning performed better in a virtual robotics challenge compared to their peers in a traditional‐oriented classroom. Students' scores on the physical robotics challenge increased as a function of the number of references they made to CT concepts during the think‐alouds. It is possible to design pedagogical experiences that tap into various dimensions of CT at incremental levels of complexity through a series of Arduino‐based robotics activities. With the integration of an online simulation, students can visualize and transfer their CT skills between a virtual and physical learning environment, thus leading to more sustainable learning outcomes. Lay Description: What is already known about this topic: Computational thinking (CT) is the conceptual foundation for problem‐solving.CT shifted from unplugged to plugged activities.CT concepts can be fostered through educational robotics. What this paper adds: A multi‐dimensional approach to assessing CT.Applications of CT provide insights into skill transfer.An integrative theoretical approach to study the 'how' and the 'what' of CT. Implications for practice: Educational robotics activities can cater to a fuller range of CT skills.Virtual simulations with constructive feedback are beneficial for students in traditional settings.Mixed learning environments contribute to making links between lectures and labs. [ABSTRACT FROM AUTHOR]

Background: The surge of online platforms has generated interest in how specialized platforms support formal and informal learning in various disciplinary domains. Knowledge is still limited regarding how undergraduate students navigate and use platforms to learn. Objectives: This study explores computer and software engineering students' learning practices, wherein online platforms are used as resources for both curricular learning activities and students' interest‐driven learning. Methods: The learning practices of 73 students were examined using a mixed‐methods design and a conceptualization of practices accounting for the context and purpose of their enactment. The dataset includes students' self‐reports on domain‐specific learning activities, three‐month‐long web‐browsing history of multiple platforms and stimulated‐recall interviews. The data were analysed through latent class and thematic analyses. Results and Conclusion: Five distinct patterns were found in the use of online platforms. These patterns show that different types of platforms were used purposely and in combination during curricular and interest‐driven activities aimed at learning software development. Moreover, students' purposes were driven by both the need to progress in their learning activities and the development of their interest in software development. The findings highlight the complexity of students' learning with online platforms, which develop quite extensively beyond curricular boundaries. Implications: The findings stress the need to recognize that undergraduate students' learning practices involve multiple online platforms and activities beyond the formal curriculum and that these play a key role in developing students' interests in learning software development. Moreover, our findings indicate the importance of taking into account the way students' learning practices unfold within platforms and how these relate to domain‐specific practices. Lay Description: What is already known about this topic?: Online platforms are mostly used by teaching staff in undergraduate education to support student learning during formal curricular tasks.Research knowledge is limited mostly to specific features of social networking platforms, such as Facebook Groups or instant messaging technologies.Students informally use interest‐driven activities and both social and domain‐specific platforms to support their learning.Research on students' interest in using massive open online courses suggested that students' learning is motivated by academic success, prospective economic opportunities, and personal interests. What this paper adds?: Students combine various types of online platforms purposely (e.g., search engines or Q&A sites), many of which are domain‐specific (e.g., Stack Overflow).Students use online platforms informally during work on curricular tasks to, for example, collaborate, solve problems, and understand concepts or methods.Domain‐specific (e.g., software engineering) platforms are used during other study‐related leisure and entrepreneurial activities.Online platforms play a key role in how students develop their interests in learning. Through them, students connect not only with other students but also with professionals, companies, and information about them. Implications of study findings for practitioners: Curriculum developers and faculty should consider preparing students to use disparate types of online platforms beyond what is included in the curriculum.Curriculum developers should also consider other activities beyond the classroom that are study‐related and meaningful for students and their learning. [ABSTRACT FROM AUTHOR]

Industrial systems can be complex and not intuitive to perceive. Therefore, students in technology and engineering programs can benefit from developing mental models of industrial systems during their journey in college. However, more often than not, these students do not have access to industrial facilities; thus, developing mental models for systems is a challenge. This paper examines the merit of an Immersive Virtual Reality (IVR) framework application in creating proper mental models for industrial systems in technology and engineering students. Two IVR applications were developed. One IVR application afforded interaction with components of a prefabricated industrial cooling water system (CWVR). In the other application, students designed and built industrial systems with IVR (system designer VR [SDVR]). SDVR facilitated constructive‐generative engagement. A group of 33 students was divided into two; one group (the Design, experimental group) was tasked with building a system with SDVR and interacting with the cooling water system in CWVR. The other group was tasked with directly interacting with the CWVR without building a system with SDVR (the Interaction, comparison group). Students' mental models of the cooling water system in CWVR were evaluated following completing the interaction experience with CWVR. The results demonstrate that the causal model notion of the mental model of the cooling water system was significantly higher in the Design, experimental group. The results suggest that designing a rich IVR application that facilitates constructive‐generative engagements may carry merit in informing student mental models of complex technical concepts. Lay Description: Industrial systems can be complex and not intuitive to perceive.Students in technology‐related programs may struggle to develop mental models for systems.Immersive virtual reality applications for industrial systems were developed and tested.Results indicate that virtual reality might inform mental models of systems in students. [ABSTRACT FROM AUTHOR]

Background: The number of artificial intelligence (AI) literacy studies in K‐12 education has recently increased, with most research focusing on primary and secondary education contexts. Little research focuses on AI literacy programs in early childhood education. Objectives: The aim of this mixed‐methods study is to examine the feasibility of an AI literacy program called "AI4KG" and explore how it might affect kindergarteners' perceptions of robots and attitudes towards engineering and science. Methods: A total of 26 child–parent dyads recruited from a Hong Kong kindergarten were involved in this study, consisting of 26 children (Mage = 4 years, SD = 0.28) and their parents. Quantitative and qualitative data were collected through surveys and interviews designed to explore children's perceptions of robots and attitudes towards engineering and science, and parents' perceptions of the AI4KG intervention. Results: It is found that children have increased their perceptions of robots after the AI literacy program, but the AI4KG curriculum had no significant effects on kindergarten children's engineering and science attitudes. Most parents (22 out of 26) agreed that their children's AI knowledge, AI skills, and AI attitudes have been enhanced after learning through the AI4KG curriculum. Conclusions: This study suggests that the AI4KG curriculum is potentially effective in promoting early AI literacy and favourable attitudes towards the technology, but further research is needed to develop age‐appropriate measures and assess its long‐term impact on children's education and career paths. Lay Description: What is currently known about this topic?: AI literacy is becoming important as AI transforms society, but few studies have explored AI literacy in early childhood education.Prior research found AI activities can positively influence older children's views of robots and engineering. What does this paper add?: It evaluates an 8‐week AI literacy curriculum called AI4KG for kindergarteners in Hong Kong.The curriculum used tools like Teachable Machine and taught concepts through play and storytelling.It improved children's perceptions of robots but did not significantly impact engineering/science attitudes.Parents perceived improvements in children's AI knowledge, skills, and attitudes after the curriculum. Implications for practice/policy: The curriculum provides an example for developing age‐appropriate AI literacy activities.More rigorous research on impacts over time is needed, but results suggest potential benefits of AI literacy in early childhood.Policymakers could consider incorporating AI literacy into kindergarten curricula to prepare children for the future. [ABSTRACT FROM AUTHOR]

Background: This study aims to establish the mechanism by which virtual reality (VR) media characteristics affect learning transfer in a VR‐based job‐training environment. The two main determinants of learning transfer are the technology and the learner, but not many studies have considered both factors simultaneously. Objectives: This study aims to ascertain the mechanism by which VR media characteristics affect learning transfer with flow as a mediator, and to explore whether presence moderates the effect of VR media characteristics on flow. Methods: A survey of semiconductor facilities engineers who received VR‐based job training was conducted. A total of 106 responses were used in the statistical analysis. A path analysis was conducted to determine the promotional mechanism that connects VR media characteristics, flow, and learning transfer. Results and Conclusions: The results were as follows. First, VR media characteristics had a positive impact on learning transfer. Second, flow acted as a mediator in the relationship between VR media characteristics and learning transfer. Third, presence played a moderating role in the relationship between VR media characteristics and flow. Key takeaways: This study is significant as it identifies elements to consider in relation to both the technological and learner aspects of VR while designing a VR‐based job‐training programme. A corporation should consider VR media characteristics from a technological perspective, and flow and presence from a learner's perspective. Lay Description: What is already known about this topic: Corporations provide virtual reality (VR)‐based job training because of its positive impact.Interactions between the VR environment and learners impact flow.Presence is important as learner's subjective awareness affects efficacy. What this paper adds: VR media characteristics influence learning transfer in VR‐based job training.VR media characteristics affect flow, which, in turn, affects learning transfer.Presence moderates the relationship between VR media characteristics and flow. Implications for practice and/or policy: VR media characteristics influence learning transfer in VR‐based job training.VR media characteristics affect flow, which, in turn, affects learning transfer.Presence moderates the relationship between VR media characteristics and flow. [ABSTRACT FROM AUTHOR]

Background: Collaborative problem solving (CPS) is important for success in the 21st century, especially for teamwork and communication in technology‐enhanced environments. Measurement of CPS skills has emerged as an essential aspect in educational assessment. Modern research in CPS relies on theory‐driven measurements that are usually carried out as manual annotations over recorded logs of collaborative activities. However, manual annotation has limited scalability and is not conductive towards CPS assessments at scale. Objective: We explore possibilities for automated annotation of actions in collaborative‐teams, especially chat messages. We evaluate two approaches that employ machine learning for automated classification of CPS events. Method: Data were collected from engineering, physics and electronics students' participation in a simulation‐based task on electronics concepts, in which participants communicated via text‐chat messages. All task activities were logged and time stamped. Data have been manually classified for the CPS skills, using an ontology that includes both social and cognitive dimensions. In this article, we describe computational linguistic methods for automatically classifying the CPS skills from logged data, with a view towards automating CPS assessments. Results: We applied two machine learning methods to our data. A Naïve Bayes classifier has been previously used in CPS research, but it is only moderately successful on our data. We also present a k‐nearest‐neighbours (kNN) classifier that uses distributional semantic models for measuring text similarity. This classifier shows strong agreement between automated and human annotations. The study also demonstrates that automatic spelling correction and slang normalization of chat texts are useful for accurate automated annotation. Implications: Our results suggest that a kNN classifier can be very effective for accurate annotation of CPS events. It achieves reasonably strong results even when trained on only half of the available data. This shows a promise towards reduction of manual data annotation for CPS measurement. Lay Description: What is already known about this topic: Collaborative problem solving (CPS) is important for success in the 21st century, especially when teamwork and communication occur in technology‐enhanced environments.Measurement of CPS skills requires classification and annotation of team interactions, in particular their communications, such as chat messages. What this paper adds: We explore possibilities for automated classification of CPS activities using machine learning approaches. Implications for practice: One of the approaches we explored is particularly effective and can help reduce the amount of manual annotations needed for CPS measurements. [ABSTRACT FROM AUTHOR]

Background: Previous work has identified that the benefits of learning with videogames and learning from simulations. However, recent meta‐analytic work has also identified that little research directly compares learning with videogames and learning with simulations. Objectives: This study examines two learning technologies and their corresponding pedagogical approaches and compares them for learning the Science, Technology, Engineering and Mathematics topic of electric charges. Methods: Participants were randomly assigned to either an intervention using a computer simulation for inquiry‐based learning or a computer videogame for game‐based learning. Their learning gains, self‐reported emotional state and experienced cognitive load were recorded. Results: We found that both learning environments improved conceptual learning, and there were no statistically significant differences between the two conditions. Participants did perceive the game‐based environment to be more engaging as well as more frustrating. We also found that cognitive load did not predict learning—however, different types of cognitive load correlated with different emotions. Overall, participants in both conditions were engaged and perceived understanding of the topic, yet they also experienced both confusion and task‐unrelated thoughts. Takeaways: When learning with simulations and videogames, educators need to align intended learning outcomes with pedagogical approaches enabled by technology. In addition, a balance between principles of multimedia learning to reduce or prevent extraneous processing, and scaffolding to reduce negative effects of learning with technology, need to be considered. Lay Description: What is already known about this topic: Research on the use of computer simulations for learning has identified positive learning effects regarding achievement outcomes, inquiry skills and noncognitive outcomes.Research on the use of videogames for learning has suggested learning benefits, such as improved working memory, development of creativity skills and achievement outcomes.Recent meta‐analytic work has identified that there is little research that directly compares learning with videogames and learning with simulations. What this paper adds: Learning benefits of difficult Science, Technology, Engineering and Mathematics (STEM) concepts may be different for game‐based learning (GBL) than inquiry‐based learning (IBL).Positive emotions can be experienced when learning via GBL than when learning via IBL.Students, who experience more positive emotions may also achieve higher learning gains.Students' cognitive load may be different when learning with GBL than when learning via IBL.Students experiencing high cognitive load (i.e., intrinsic and extraneous) may show lower learning gains.Students experiencing high cognitive load (i.e., intrinsic and extraneous) may show higher level of mind‐wandering, frustration, or confusion. Implications for practice and/or policy: Learning with simulated inquiry base environment and game base environment improved conceptual learning.GBL was more engaging and more frustrating for students, when learning with computer videogames.Cognitive load did not predict STEM learning with technologies.Different cognitive load correlated with different learning emotions. [ABSTRACT FROM AUTHOR]

This study investigated changes in learners' motivation, engagement, performance, and spatial reasoning over time and across different levels of virtual reality (VR) immersion. Undergraduate participants explored a virtual solar system via a moderately immersive or highly immersive VR platform over three sessions. In a third condition, participants initially learned with moderate immersion and transitioned to higher immersion after the second session. Following research on novelty effects, we explored whether subjective experiences and performance would decline over time (e.g., decreasing motivation or performance) as participants became familiar with the virtual environment and tools. However, we hypothesized that transitional immersion (i.e., switching from moderate to higher immersion) might lead to a renewed sense of novelty. Results suggested that both moderate and higher levels of immersion were motivating, engaging, and supportive of learning. In contrast to predictions based on novelty effects, these outcomes did not decline overall as learners gained familiarity with the systems. However, transitional immersion emerged as a promising and testable pedagogical approach for future VR education. All participants also showed gains in spatial reasoning. Lay Description: What is already known about this topic?: Virtual reality can support motivation and learning in STEM education.Novelty may elicit motivation, engagement and learning.Virtual reality may benefit development of spatial reasoning skills. What this paper adds?: Motivation, engagement and performance persist across multiple sessions of VR learning.Switching from moderate to higher immersion increases motivation and engagement.Both moderate and higher immersion support improvements in spatial reasoning. Implications for practice and/or policy: Novelty effects are not a significant concern when implementing VR education over multiple sessions.More immersive VR tools may be progressively introduced during learning. [ABSTRACT FROM AUTHOR]

In engineering design, engineers must be able to think creatively, effectively toggling between divergent thinking (developing multiple novel ideas) and convergent thinking (pursuing an appropriate idea using engineering analyses). However, creative thinking is not emphasized in many undergraduate engineering programs. In this empirical study, we analyze the divergent thinking of teams working on a virtual laboratory project. Fifteen student teams' solution paths–as represented by Model Maps–were analyzed to characterize and compare the various elements of divergent thinking: fluency, flexibility, and originality. The solution paths of these teams were compared in two physical laboratory projects and to experts completing the same virtual laboratory project. We found that students demonstrated more divergent thinking in the virtual laboratory project than in the physical laboratory projects; yet, divergent thinking and quality of solution did not correlate. There was little difference between measured elements of divergent thinking between student teams and experts. Lay Description: What is currently known about the subject matter: Creative thinking is an essential skill for engineers to develop innovative products and processes.Creative thinking involves toggling between divergent thinking (coming up with many different original ideas) and convergent thinking (executing an appropriate idea based on engineering analyses).Despite its importance, creative thinking is not often encouraged nor taught in undergraduate engineering curricula, and, similarly, is not well‐researched in that context.The development of creative thinking is supported by students working in Problem‐Based Learning (PBL), in which students work in teams on an open‐ended, "real world" problem. What our paper adds to this: We developed an innovative methodology to analyze aspects of divergent thinking in an open‐ended, computer‐based virtual laboratory project.Students engaged in divergent thinking significantly more in the virtual laboratory project as compared to two physical laboratory projects in the same course.The degree of divergent thinking did not correlate to the quality of solution in the virtual laboratory project.The degree of divergent thinking of student teams was about the same as that of experts. The implications of study findings for practitioners: Findings support the use of open‐ended, computer‐based virtual laboratory projects to provide students opportunities to practice divergent thinking.The aspects of student teams' divergent thinking in this context is similar to experts.The relationship between divergent thinking and quality of solution is complex.More work needs to be done to understand how to encourage broader creative thinking where student teams toggle between divergent and convergent thinking. [ABSTRACT FROM AUTHOR]

The main objective of this study was to examine the effectiveness of immersive virtual reality (VR) as a medium for delivering laboratory safety training. We specifically compare an immersive VR simulation, a desktop VR simulation, and a conventional safety manual. The sample included 105 first year undergraduate engineering students (56 females).  We include five types of learning outcomes including post‐test enjoyment ratings; pre‐ to post‐test changes in intrinsic motivation and self‐efficacy; a post‐test multiple choice retention test; and two behavioral transfer tests. Results indicated that the groups did not differ on the immediate retention test, suggesting that all three media were equivalent in conveying the basic knowledge. However, significant differences were observed favoring the immersive VR group compared to the text group on the two transfer tests involving the solving problems in a physical lab setting (d = 0.54, d = 0.57), as well as enjoyment (d = 1.44) and increases in intrinsic motivation (d = 0.69) and self‐efficacy (d = 0.60). The desktop VR group scored significantly higher than the text group on one transfer test (d = 0.63) but not the other (d= 0.11), as well as enjoyment (d =1.11) and intrinsic motivation (d =0.83). Lay Description: What is currently known about the subject matter: Virtual reality (VR) is increasingly being used to deliver learning and training material.One field where the affordances of VR are particularly relevant is in safety training.VR provides the opportunity for trainees to safely act out realistic scenarios where making the right decisions is pivotal and training in real life would otherwise be impractical or impossible.Most studies that investigate the effectiveness of learning with VR do not include behavioural transfer tests. What this paper adds to this research: This study used a broad array of assessment methods to evaluate the effectiveness of safety training delivered with three different instructional media: an immersive VR simulation, a desktop VR simulation, and a conventional text‐based safety manual.There were no differences between conventional and VR training on a retention test.The immersive VR group significantly outperformed the conventional group on two behavioural transfer tests, perceived enjoyment, and increase in intrinsic motivation and self‐efficacy.The desktop VR group scored significantly higher than the conventional group on one behavioural transfer test, perceived enjoyment, and increase in intrinsic motivation.The results suggest that behavioural measures of transfer in realistic settings may be necessary to accurately assess the instructional value of VR learning environments. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Assisted Learning is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Studies have shown that educational robotics (ER) may impact student learning, especially in relation to STEM (science, technology, engineering, and mathematics) areas. In the STEM framework, particularly for younger children, the "E" and the "T" are considered to be missing letters, because few studies have concentrated on teaching and evaluating technology and engineering through ER activities. This study aimed to develop and test the efficacy of an ER protocol to teach robotics in a sample of 389 students, hypothesizing that girls would be as successful as boys. A Robotics Questionnaire assessing the basics of robotics was developed for this study. A Wilcoxon nonparametric test was performed in order to evaluate improvements (p < 0.05). A Mann–Whitney nonparametric test was performed in order to test the presence of gender differences (p < 0.05). Data indicated significant improvements for all the age ranges considered. No gender differences were found. In order to evaluate the efficacy of a didactic intervention utilizing ER, it is important to assess the impact on children's technological and engineering (robotics, in particular) knowledge. Lay Description: What is already known about this topic: Studies have shown that educational robotics (ER) has a potential impact on student learning.In the STEM framework, the "E" and the "T" are considered the STEM missing letters.What this paper adds: This study aims at developing and testing the efficacy of an ER protocol to teach robotics.The hypothesis that girls will be equally successful than boys is explored too, and no gender differences were found.Implications for practice and/or policy: Our results suggest that ER can be utilized to learn robotics, bringing technology in the schools.ER could be considered as a tool able to contribute to girl's involvement in STEM. [ABSTRACT FROM AUTHOR]

Time on task has been recognized as an important variable in academic learning, but self‐report measures of study time are problematic. Therefore, this study employs an automated system for recording time spent reading a course textbook. College students in an introductory engineering course accessed their textbook online. The book contained pages of instructional text, worked examples, homework problems, and answers to homework problems. An instrumented document reader program called "STL Reader" recorded the time each student spent on each page, thus providing detailed measures of reading habits. Across the 10‐week course, students spent an average of 1.9 hr reading instructional text, 1.4 hr on worked examples, 22.1 hr on homework problems, and 0.9 hr on homework answers, indicating a preference for practicing to solve test problems (i.e., self‐testing) rather than being told (i.e., receiving direct instruction). Furthermore, course grade (based largely on solving problems on exams and quizzes) correlated significantly and positively with time viewing homework problems, but not with time viewing either instructional text or worked examples, indicating that achievement was related to time spent practicing for solving test problems but not to time spent being instructed. Results suggest a revision of the time‐on‐task hypothesis to include the value of spending time on tasks aligned to test requirements. Lay Description: What is currently known?: Time‐on‐task theory states that students' time engaged in relevant material is an important factor in learning and achievement.How students choose to process presented information is important for academic learning.Undergraduate STEM students often read very little of the assigned course textbook. What this paper adds: Technology‐enhanced data collection provides more accurate measure of students' engagement with e‐textbook.Time spent viewing homework problems is significantly and positively related to achievement in an undergraduate engineering course.Student grades were not positively correlated with time spent viewing instructional text or worked examples from the textbook. Implications: Suggests revision of time‐on‐task hypothesis to include the value of spending time on tasks aligned to test requirements. [ABSTRACT FROM AUTHOR]

Abstract: Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates the use of diverse sensors, including computer vision, user‐generated content, and data from the learning objects (physical computing components), to record high‐fidelity synchronised multimodal recordings of small groups of learners interacting. We processed and extracted different aspects of the students' interactions to answer the following question: Which features of student group work are good predictors of team success in open‐ended tasks with physical computing? To answer this question, we have explored different supervised machine learning approaches (traditional and deep learning techniques) to analyse the data coming from multiple sources. The results illustrate that state‐of‐the‐art computational techniques can be used to generate insights into the "black box" of learning in students' project‐based activities. The features identified from the analysis show that distance between learners' hands and faces is a strong predictor of students' artefact quality, which can indicate the value of student collaboration. Our research shows that new and promising approaches such as neural networks, and more traditional regression approaches can both be used to classify multimodal learning analytics data, and both have advantages and disadvantages depending on the research questions and contexts being investigated. The work presented here is a significant contribution towards developing techniques to automatically identify the key aspects of students success in project‐based learning environments, and to ultimately help teachers provide appropriate and timely support to students in these fundamental aspects. [ABSTRACT FROM AUTHOR]

This paper describes a novel approach to the development and implementation of an educational tool based on knowledgebased technology employing an expert system shell. Software has been developed which, after providing basic background information, proceeds to analyse the student's learning pattern to establish which next question, explanation, or topic to propound in order to aid the learning process and ensure that fundamental knowledge is gained by the student at his/her own pace. While the system described has been designed and implemented specifically to supplement, rather than supplant, the taught part of an MSc. course in Manufacturing Systems Engineering and Management, the methodology formulated can be used to develop similar knowledge-based systems for other technical as well as non-technical subjects at both undergraduate and postgraduate levels in Higher Education. [ABSTRACT FROM AUTHOR]

Learning a foreign language requires interaction with language input while involved in a task. Given that problem‐based learning (PBL) offers hands‐on application in realistic contexts, and that virtual reality (VR) enables learners to interact with multiple modalities of information, this study examines how the integration of VR technology into PBL contexts affects students' motivation for, problem‐solving during, and vocabulary acquisition in learning English as a foreign language (EFL). A total of 84 engineering majors who enrolled in a course of English for specific purposes were randomly assigned to either an experimental group or a control group. Students in the experimental group participated in a VR‐assisted PBL context, in which they were to view a PBL scenario using VR technology and then to create VR videos about solving the given problems. Those in the control group participated in a PBL context without the use of VR technology for viewing and solving the identical scenario. After the intervention, all the students wrote a problem‐solving analysis, took a vocabulary knowledge test, completed a learning motivation questionnaire, and participated in individual interviews. The results showed that the students in the experimental group significantly outperformed those in the control group in terms of vocabulary acquisition, and were more motivated to learn English related to their future careers, whereas there was no significant difference in the problem‐solving performance of the two groups. Implications of these findings highlight the value of engaging EFL learners in immersive environments for contextualized learning through the integrated use of VR and PBL. Lay Description: What is already known about this topic?: Virtual reality (VR) technology has been shown to enhance students' knowledge and skills in STEM fields.Problem‐based learning (PBL) has been widely used in medical education but not so much in foreign language education.English as a foreign language (EFL) students usually have very limited access to the target language beyond the classroom, thus lack a motivating reason for them to learn English after graduation. What this paper adds?: The VR‐assisted PBL approach proposed in this study was found to significantly enhance the EFL students' specialized vocabulary knowledge.The VR‐assisted PBL approach was found to motivate and sustain the EFL students' interest in career English.Exposing the EFL students to PBL contexts was capable of developing their problem‐solving performance, regardless of VR integration. Implications for practice and/or policy: The proposed approach has the potential to facilitate student learning in EFL education, particularly English for occupational purposes.EFL educators may apply the VR‐assisted PBL approach to narrow the gap between language classrooms and workplace contexts by immersing students in authentic contexts that were relevant to their occupational knowledge and workplace English. [ABSTRACT FROM AUTHOR]

Background: The advent of new and emerging technologies and industries has highlighted the need to equip youth with a unique skillset necessary to cope with a rapidly changing and complex digital era and adapt to modern societies' demands. This need has led to the development of teaching approaches to equip students with creative and innovative skills to help prevent any future skills gap. This shift has fuelled the growth of STEAM (Science, Technology, Engineering, Arts and Mathematics) Education worldwide. Objectives: Our goal was to engage in a systematic review of the literature to identify the application and prevalence of emerging technologies within the landscape of STEAM Education. Methods: We engaged in a systematic review of the literature. Following the application of exclusion criteria to 461 studies, 43 studies were extracted and analysed. Findings and Conclusions: Analysis of these studies provides evidence of the fast‐growing use of innovative emerging technologies within the STEAM landscape across all levels of education, from early childhood to college‐level settings. Our analysis reveals an emphasis on developing STEAM‐related disciplinary knowledge and the desire to develop students' 21st‐century skills with a notable lack of targeted emphasis on developing understandings in the arts disciplines. We identify the need for carefully designed intervention studies involving collaboration between multidisciplinary STEAM experts that use high‐quality measures which support the development of inferences relating to learning outcomes arising from such interventions. Lay Description: The need to equip students with creative and innovative skills has fuelled the growth of STEAM (Science, Technology, Engineering, Arts and Mathematics) Education worldwide.Technology can play a vital role in STEAM education, offering boundless capabilities and possibilities for creating high‐quality learning experiences that foster students' innovation creativity, communication and collaboration, critical thinking, and problem‐solving skills.Currently, efforts are made by educational researchers and practitioners worldwide to tap the potential of technology to support STEAM learning environments.A need exists for empirical evidence to support assumptions and hypotheses concerning the role of technology in STEAM education.The article offers a systematic review of the literature to identify the application and prevalence of emerging technologies within the landscape of STEAM Education.The systematic review describes a variety of pioneering STEAM education initiatives that utilize emerging technologies in their designs, but also reveals the challenges and obstacles to integrating STEAM disciplines in a meaningful way. [ABSTRACT FROM AUTHOR]

Abstract: This study focuses on the investigation of the effects of computer simulation and animation (CSA) on students' cognitive processes in an undergraduate engineering course. The revised Bloom's taxonomy, which consists of six categories in the cognitive process domain, was employed in this study. Five of the six categories were investigated, including remember, understand, apply, analyze, and evaluate. Data were collected via a think‐aloud protocol involving two groups of student participants: One group learned a worked example problem with a CSA module, and the other group learned the same problem with traditional textbook‐style instruction. A new concept called frequency index was proposed for use in qualitative research that involves the quantitative comparison of the overall popularity of a particular mental activity performed by two groups of students. The results show that as compared to traditional textbook‐style instruction, CSA significantly increases students' activities in the understand category of the revised Bloom's taxonomy during learning and significantly increases students' activities in the understand, apply, analyse, and evaluate categories during subsequent problem‐solving. That learning via CSA has a profound impact on subsequent problem‐solving is attributed to intensive human–computer interactions built in the CSA learning module. [ABSTRACT FROM AUTHOR]

In this study, we investigated the validity of a stealth assessment of physics understanding in an educational game, as well as the effectiveness of different game‐level delivery methods and various in‐game supports on learning. Using a game called Physics Playground, we randomly assigned 263 ninth‐ to eleventh‐grade students into four groups: adaptive, linear, free choice and no‐treatment control. Each condition had access to the same in‐game learning supports during gameplay. Results showed that: (a) the stealth assessment estimates of physics understanding were valid—significantly correlating with the external physics test scores; (b) there was no significant effect of game‐level delivery method on students' learning; and (c) physics animations were the most effective (among eight supports tested) in predicting both learning outcome and in‐game performance (e.g. number of game levels solved). We included student enjoyment, gender and ethnicity in our analyses as moderators to further investigate the research questions. [ABSTRACT FROM AUTHOR]

Abstract A Virtual reality environment with hypermedia was designed to help undergraduates understand the structure and functioning of a turning lathe. Study 1 was carried out with 30 novice students and Study 2 involved 24 students attending a machining course. These studies demonstrated that the virtual lathe can foster the comprehension of some core machining concepts. Further, the studies suggest that novice students benefit most from earlier free navigation of the virtual environment whereas expert students benefit from an analysis of the hypermedia. [ABSTRACT FROM AUTHOR]