Search

Self-direction skill (SDS) is an essential 21st-century skill that can help learners be independent and organized in their quest for knowledge acquisition. While some studies considered learners from higher education levels as the target audience, providing opportunities to start the SDS practice by K12 learners is still rare. Further, practicing such skills requires a concrete context and scaffolding during the skill acquisition. This article introduces the Goal Oriented Active Learner (GOAL) system that facilitates SDS acquisition in learners utilizing daily activities as context. The GOAL architecture integrates learning logs from online environments and physical activity logs from wearable trackers to provide a data-rich environment for the learners to acquire and practice their SDS. The GOAL users follow DAPER, a five-phase process model, to utilize the affordances in the system while practicing SDS. We implemented the GOAL system at a K12 public institution in Japan in 2019. Learners used the online environments for extensive reading and smartwatches for tracking walking and sleeping activities. This study analyzes detailed interaction patterns in GOAL while learners planned and monitored their self-directed actions. The results illustrate the strategies for DAPER behaviors that emerge in different activity contexts. We discuss the potentials and challenges of this technology ecosystem that connects learners’ learning logs and physical activity logs, specifically in the K12 context in Japan and, more generally, from the learning analytics research perspective to provide a context to practice SDS.

This study focuses on the problem that the process of building learning habits has not been clearly described. Therefore, we aim to extract the stages of learning habits from log data. We propose a data model to extract stages of learning habits based on the transtheoretical model and apply the model to the learning logs of self-directed extensive reading to demonstrate the process of building learning habits. We uncover the various stages (i.e., the precontemplation, contemplation, preparation, action, and maintenance stage) that learners underwent, and different proportions of the maximum stage that they achieved during an 11-month, self-directed, extensive reading program—implemented in a Japanese junior high school. This study contributes to realizing a method to evaluate the learning process, by tracing the stages of learning habits in long-term, and continuous learning activities. Further, this study can help guide the development of evidence-based educational interventions to support the building of lifelong learning habits and self-directed learning, using data-driven methods.

Iterative team-based learning (TBL) is a common educational strategy for collaborative learning that involves sequential phases of individual and group learning activities. The advent of digital learning platforms, with the accumulation of learning log data, presents an opportunity to leverage data-driven techniques to enhance TBL practices. However, applying data-driven approaches in iterative TBL scenarios has received limited exploration in existing literature. Through a review of initial studies in this domain, data-driven iterative TBL emerges as a promising area. To explore this topic, we introduce a novel framework, drawing from the GLOBE framework for group learning, aimed at integrating data-driven designs into iterative TBL settings. This framework is proposed to guide data and activity design within iterative TBL, comprising four phases of group learning activity workflow and three essential steps of data flow. Additionally, we present two authentic instances supported by empirical evidence, offering insights into how educators can implement data-driven designs across different phases of TBL. Within the data-driven environment, we also uncover potential impacts and challenges of data-driven iterative TBL, to identify avenues for future research that can further expand our understanding of the possibilities in this domain.

Abstract In inclusive education, students with different needs learn in the same context. With the advancement of artificial intelligence (AI) technologies, it is expected that they will contribute further to an inclusive learning environment that meets the individual needs of diverse learners. However, in Japan, we did not find any studies exploring current needs in an actual special needs context. In this study, we used the learning and evidence analysis framework (LEAF) as a learning analytics-enhanced learning environment and employed Active Reading as an example learning task to investigate the challenges and possibilities of applying AI to inclusive education in the future. Two students who attended a resource room formed the context. We investigated learning logs in the LEAF system while each student executed a given learning task. We detected specific learning behaviors from the logs and explored the challenges and future potential of learning with AI technology, considering human involvement in orchestrating inclusive educational practices.

Data-driven platforms with rich data and learning analytics applications provide immense opportunities to support collaborative learning such as algorithmic group formation systems based on learning logs. However, teachers can still get overwhelmed since they have to manually set the parameters to create groups and it takes time to understand the meaning of each indicator. Therefore, it is imperative to explore predictive indicators for algorithmic group formation to release teachers from the dilemma with explainable group formation indicators and recommended settings based on group work purposes. Employing learning logs of group work from a reading-based university course, this study examines how learner indicators from different dimensions before the group work connect to the subsequent group work processes and consequences attributes through correlation analysis. Results find that the reading engagement and previous peer ratings can reveal individual achievement of the group work, and a homogeneous grouping strategy based on reading annotations and previous group work experience can predict desirable group performance for this learning context. In addition, it also proposes the potential of automatic group formation with recommended parameter settings that leverage the results of predictive indicators.

Abstract The development of technology enables diverse learning experiences nowadays, which shows the importance of learners’ self-regulated skills at the same time. Particularly, the ability to allocate time properly becomes an issue for learners since time is a resource owned by all of them. However, they tend to struggle to manage their time well due to the lack of awareness of its existence. This study, hence, aims to reveal how learners allocate their time and evaluate the effectiveness of the time allocation by examining its effects on learners’ performance. We collect the learning logs of 116 seventh-graders from the online learning system implemented in a Japanese public junior high school. We look at the data in the time window of 34 days before the regular exam. Even though clustering techniques as a Learning Analytics method help identify different groups of learners, it is seldom applied to group students’ learning patterns with different levels of indicators extracted from their learning process data. In this study, we adopt the method to cluster students’ patterns of time allocation and find that better performance can result from the consistency of study time throughout the exam preparation period. Practical suggestions are then proposed for different roles involved in digital learning environments to facilitate students’ time management. Collectively, this study is expected to make contributions to smart learning environments supporting self-regulated learning in the digital era.

Abstract Digitized learning materials are a core part of modern education, and analysis of the use can offer insight into the learning behavior of high and low performing students. The topic of predicting student characteristics has gained a lot of attention in recent years, with applications ranging from affect to performance and at-risk student prediction. In this paper, we examine students reading behavior using a digital textbook system while taking an open-book test from the perspective of engagement and performance to identify the strategies that are used. We create models to predict the performance and engagement of learners before the start of the assessment and extract reading behavior characteristics employed before and after the start of the assessment in a higher education setting. It was found that strategies, such as: revising and previewing are indicators of how a learner will perform in an open ebook assessment. Low performing students take advantage of the open ebook policy of the assessment and employ a strategy of searching for information during the assessment. Also compared to performance, the prediction of overall engagement has a higher accuracy, and therefore could be more appropriate for identifying intervention candidates as an early-warning intervention system.

Abstract With COVID-19 pandemic forcing academic institutions to shift to emergency remote teaching (ERT), teachers worldwide are attempting several strategies to engage their learners. Even though existing research in online learning suggests that effectiveness of the online session is more dependent on pedagogical design rather than technology feature, teachers may still focus on the intricacies of the technology. In this paper, we present the evolution of an active learning pedagogy, supported by technology (eBook reader—BookRoll, Analytics Dashboard—LAViEW), for an undergraduate physics classroom across a semester that was affected by the lockdown due to pandemic. The technology-enhanced pedagogy evolved in three phases—technology used in “Content Focus” mode, technology used in “Problem Focus” mode and technology used in “Learning Dialogue Focus” mode. The entire activities were designed and implemented within the technology-enhanced and evidence-based education and learning (TEEL) ecosystem, which supported integration of learning technologies with analytics system. Comparison of the student’s learning logs indicated that there was a sustained engagement in the learning activities conducted during the blended (before lockdown) and online mode (during lockdown). We had conducted one-way ANOVA to compare the post-test scores for each teaching phase and found statistically significant differences in the latter phases. A preliminary qualitative analysis of the learner artifacts generated as memos in BookRoll during each phase revealed that students were posing conceptual clarifications during the latter phases. These were also having greater alignment with the session agenda and showed construction of new knowledge based on the seed knowledge provided during the instructor–learner interaction sessions. The study provides key insights into how reflection and practice by both learner and teacher improves the acceptance of technology-enabled pedagogy.

Abstract Recent spread of the COVID-19 forces governments around the world to temporarily close educational institutions. In this paper, we evaluated learning engagement, level of satisfaction and anxiety of e-book based remote teaching strategy on an online learning platform. The research involves 358 students at an urban junior-high school in Japan. Learning logs were analyzed to measure student engagement, whereas survey responses indicated their perception regarding the remote learning experience. Log analysis revealed that the average completion rate over 267 learning materials was 67%. We also observed a significant decrease in engagement 3 weeks after remote learning and different subjects and grades. Survey analysis showed students felt both satisfaction and anxiety about remote learning. However, there were significant differences in the level of satisfaction between different grades. The results indicated that (1) maintaining students’ motivation is a challenge to remote learning in secondary schools, and (2) we need to relieve students’ anxiety about their own progress in the class and their classes after the break. This study is the first to report trends in actual teaching–learning engagement, which were recorded during sessions of emergency remote teaching in Japanese schools. The results can inform the future implementation of remote learning in junior-high schools.

While data privacy is a key aspect of Learning Analytics, it often creates difficulty when promoting research into underexplored contexts as it limits data sharing. To overcome this problem, the generation of synthetic data has been proposed and discussed within the LA community. However, there has been little work that has explored the use of synthetic data in real-world situations. This research examines the effectiveness of using synthetic data for training academic performance prediction models, and the challenges and limitations of using the proposed data sharing method. To evaluate the effectiveness of the method, we generate synthetic data from a private dataset, and distribute it to the participants of a data challenge to train prediction models. Participants submitted their models as docker containers for evaluation and ranking on holdout synthetic data. A post-hoc analysis was conducted on the top 10 participant’s models by comparing the evaluation of their performance on synthetic and private validation datasets. Several models trained on synthetic data were found to perform significantly poorer when applied to the non-synthetic private dataset. The main contribution of this research is to understand the challenges and limitations of applying predictive models trained on synthetic data in real-world situations. Due to these challenges, the paper recommends model designs that can inform future successful adoption of synthetic data in real-world educational data systems.

Abstract This study investigates learner’s reading behaviors in a critical reading task in humanities course using learning analytics techniques. A Critical Analysis of Literature and Cinema course was selected as a context. The course activities evolved over 10 years, and for this instance, some face-to-face classroom critical reading activities were migrated to online mode by using BookRoll, a learning analytics enhanced eBook platform. Students (n=22 out of the 50 registered) accessed Hayavadana, an Indian play uploaded on BookRoll, and attempted to identify performative elements and cultural references in the text and highlight them. In this study, we analyze learner’s reading logs gathered in the learning record store linked to BookRoll during that activity. We extend our previous work where we identify four online reading profiles: effortful, strategic, wanderers, and check-out, based on learner’s clickstream interactions and time spent with the content. We validate the profiles with qualitative interview data collected from the learners and illustrate the quantified learning behaviors of each of those profiles based on an engagement metric. Our work aims to initiate further discussion related to the application of learning analytics in humanities courses both to probe into the learning behaviors of the students and thereby enhance the experiences with the use of interactive learning environments and data-driven services.

This paper explores co-design in Japanese education for deploying data-driven educational technology and practice. Although there is a growing emphasis on data to inform educational decision-making and personalize learning experiences, challenges such as data interoperability and inconsistency with teaching goals prevent practitioners from participating. Co-design, characterized by involving various stakeholders, is instrumental in addressing the evolving needs of technology deployment. Japan's educational context aligns with co-design implementation, with a learning and evidence analytics infrastructure facilitating data collection and analysis. From the Japanese co-design practice of educational technologies, the paper highlights a 6-phase co-design framework: motivate, pilot, implement, refine, evaluate, and maintain. The practices focus on data-driven learning strategies, technology interventions, and across-context dashboards, covering assorted learning contexts in Japan. By advocating for a co-design culture and data-driven approaches to enhance education in Japan, we offer insights for education practitioners, policymakers, researchers, and industry developers.

One trending theme within research on learning and teaching is an emphasis on artificial intelligence (AI). While AI offers opportunities in the educational arena, blindly replacing human involvement is not the answer. Instead, current research suggests that the key lies in harnessing the strengths of both humans and AI to create a more effective and beneficial learning and teaching experience. Thus, the importance of 'humans in the loop' is becoming a central tenet of educational AI. As AI technology advances at breakneck speed, every area of society, including education, needs to engage with and explore the implications of this phenomenon. Therefore, this paper aims to assist in this process by examining the impact of AI on education from researchers' and practitioners' perspectives. The authors conducted a Delphi study involving a survey administered to N = 33 international professionals followed by in-depth face-to-face discussions with a panel of international researchers to identify key trends and challenges for deploying AI in education. The results indicate that the three most important and impactful trends were (1) privacy and ethical use of AI; (2) the importance of trustworthy algorithms; and (3) equity and fairness. Unsurprisingly, these were also identified as the three key challenges. Based on these findings, the paper outlines policy recommendations for AI in education and suggests a research agenda for closing identified research gaps.

Abstract Self-direction skill is considered a vital skill for twenty-first-century learners in both the learning context and physical activity context. Analysis skill for self-directed activities requires the students to analyze their own activity data for understanding their status in that activity. It is an important phase that determines whether an appropriate plan can be set or not. This research presents a framework designed to foster students’ analysis skill in self-directed activities, which aims (1) to build a technology-enabled learning system allowing students to practice analyzing data from their own daily contexts, (2) to propose an approach to model student’s analysis skill acquisition level and process, and (3) to provide automated support and feedback for analysis skill development tasks. The analysis module based on the proposed framework was implemented in the GOAL system which synchronized data from learners’ physical and reading activities. A study was conducted with 51 undergraduate students to find reliable indicators for the model to then measure students’ analysis skills. By further analyzing students’ actual usage of the GOAL system, we found the actual activity levels and their preferences regarding analysis varied for the chosen contexts (learning and physical activity). The different context preference groups were almost equal, highlighting the utility of a system that integrates data from multiple contexts. Such a system can potentially respond to students’ individual preferences to execute and acquire self-direction skill.

Abstract Collaborative learning in the form of group work is becoming increasingly significant in education since interpersonal skills count in modern society. However, teachers often get overwhelmed by the logistics involved in conducting any group work. Valid support for executing and managing such activities in a timely and informed manner becomes imperative. This research introduces an intelligent system focusing on group formation which consists of a parameter setting module and the group member visualization panel where the results of the created group are shown to the user and can be graded. The system supports teachers by applying algorithms to actual learning log data thereby simplifying the group formation process and saving time for them. A pilot study in a primary school mathematics class proved to have a positive effect on students’ engagement and affections while participating in group activities based on the system-generated groups, thus providing empirical evidence to the practice of Computer-Supported Collaborative Learning (CSCL) systems.

Abstract Blended learning technique has adapted many new digitized tools to facilitate students with flexible and self-phased learning opportunities. The flipped classroom strategy, one of the blended learning models has been limited by low engagement of students in the online component. In the present study, we augment a Flip and Pair (F&P), an active-learning strategy into the blended learning course. Following the AB type single group quasi-experimental design, we evaluated the effects of F&P strategy on the student’s engagement and learning while orchestrating it for an undergraduate engineering physics course. Our results highlighted that there is a positive correlation between the engagement (computed based on learning logs of TEEL (Technology-enhanced and Evidence-based Education and Learning) platform in the F&P activities with that of the performance score (knowledge quizzes and final exam). F&P strategy had a better contribution compared to Flip and Traditional Teaching (F&TT) strategy with respect to both engagement and performance. Also, students exhibited a positive perception of learning and engagement. Based on our findings, we identified the key instructional measures that an instructor can follow to increase student engagement while using the F&P strategy.

Evidence-based education has become more relevant in the current technology-enhanced teaching-learning era. This paper introduces how Educational BIG data has the potential to generate such evidence. As evidence-based education traditionally hooks on the meta-analysis of the literature, so there are existing platforms that support manual input of evidence as structured information. However, such platforms often focus on researchers as end-users and its design is not aligned to the practitioners’ workflow. In our work, we propose a technology-mediated process of capturing teaching-learning cases (TLCs) using a learning analytics framework. Each case is primarily a single data point regarding the result of an intervention and multiple such cases would generate an evidence of intervention effectiveness. To capture TLCs in our current context, our system automatically conducts statistical modelling of learning logs captured from Learning Management Systems (LMS) and an e-book reader. Indicators from those learning logs are evaluated by the Linear Mixed Effects model to compute whether an intervention had a positive learning effect. We present two case studies to illustrate our approach of extracting case effectiveness from two different learning contexts – one at a junior-high math class where email messages were sent as intervention and another in a blended learning context in a higher education physics class where an active learning strategy was implemented. Our novelty lies in the proposed automated approach of data aggregation, analysis, and case storing using a Learning Analytics framework for supporting evidence-based practice more accessible for practitioners.

UNESCO reported that 90% of students are affected in some way by COVID-19 pandemic. Like many countries, Japan too imposed emergency remote teaching and learning at both school and university level. In this study, we focus on a national university in Japan, and investigate how teaching and learning were facilitated during this pandemic period using an ebook platform, BookRoll, which was linked as an external tool to the university’s learning management system. Such an endeavor also reinforced the Japanese national thrust regarding explorations of e-book-based technologies and using Artificial Intelligence in education. Teachers could upload reading materials for instance their course notes and associate an audio of their lecture. While students who registered in their course accessed the learning materials, the system collected their interaction logs in a learning record store. Across the spring semesters from April - July 2020, BookRoll system collected nearly 1.5 million reading interaction logs from more than 6300 students across 243 courses in 6 domains. The analysis highlighted that during emergency remote teaching and learning BookRoll maintained a weekly average traffic above 1,900 learners creating more than 78,000 reading logs and teachers perceived it as useful for orchestrating their course.

Abstract Early prediction systems have already been applied successfully in various educational contexts. In this study, we investigated developing an early prediction system in the context of eBook-based teaching-learning and used students’ eBook reading data to develop an early warning system for students at-risk of academic failure -students whose academic performance is low. To determine the best performing model and optimum time for possible interventions we created prediction models by using 13 prediction algorithms with the data from different weeks of the course. We also tested effects of data transformation on prediction models. 10-fold cross-validation was used for all prediction models. Accuracy and Kappa metrics were used to compare the performance of the models. Our results revealed that in a sixteen-week long course all models reached their highest performance with the data from the 15th week. On the other hand, starting from the 3rd week, the models classified low and high performing students with an accuracy of over 79%. In terms of algorithms, Random Forest (RF) outperformed other algorithms when raw data were used, however, with the transformed data J48 algorithm performed better. When categorical data were used, Naive Bayes (NB) outperformed other algorithms. Results also indicated that models with transformed data performed lower than the models created using categorical data. However, models with categorical data showed similar performance with models with raw data. The implications of the results presented in this research were also discussed with respect to the field of Learning Analytics.

We aim to identify the engagement strategies that higher education students, engaging in emergency online learning in low-resource settings, perceive to be effective. We conducted a sequential mixed-methods study based on Moore’s interaction framework for distance education. We administered a questionnaire to 313 students engaging in emergency online learning in low-resource settings to examine their perceptions of different engagement strategies. Our results showed that student–content engagement strategies, e.g., screen sharing, summaries, and class recordings, are perceived as the most effective, closely followed by student–teacher strategies, e.g., Q and A sessions and reminders. Student–student strategies, e.g., group chat and collaborative work, are perceived as the least effective. The perceived effectiveness of engagement strategies varies based on the students’ gender and technology access. To support instructors, instructional designers, and researchers, we propose a 10-level guide for engaging students during emergency online classes in low-resource settings.

Self-directed learning (SDL) requires students to take initiative to learn and control their own learning process. Literature highlights the importance of SDL for lifelong learning. Yet, little understanding is known regarding how to support SDL at the school level, specifically for out-of-class learning context. To fill up this gap, this research developed a learning dashboard and integrated SDL process management in GOAL system. It was implemented to provide support for out-of-class online self-directed extensive reading (SDER) at a high school level. A two-group study conducted during a three-week spring vacation found the experimental group (N = 35, with SDL support) achieved significantly more reading outcomes than the control group (N = 12, without SDL support). Detailed GOAL interaction behavior analysis of the experimental group showed that viewing learning dashboard was significantly correlated with reading outcomes as well as interactions related to SDL process management. These findings highlights positive effect of SDL support in GOAL on students' out-of-class SDER outcomes as well as their SDL behaviors. The study provided implications for research related to extensive reading and SDL support for out-of-class learning.

In-class group work activities are found to promote the interpersonal skills of learners. To support the teachers in facilitating such activities, we designed a learning analytics-enhanced technology framework, Group Learning Orchestration Based on Evidence (GLOBE) using data-driven approaches. In this study, we implemented the algorithmic group formation and group work evaluation systems in a Japanese junior high school context. Data from a series of 12 collaborative learning activities were used to validate the difference in the measured heterogeneity of the formed homogeneous and heterogeneous groups compared to random grouping. Further, the peer rating and self-perception of the group work were compared for comparative reading and idea exchange tasks. We found algorithmically formed groups, considering the learner model data, either heterogeneously or homogeneously performed better than random grouping. Specifically, students in groups created by the homogeneous algorithm received higher peer ratings and more positive self-perception of group work in the idea exchange group tasks. We did not find significant differences in the comparative reading tasks. Along with the empirical findings, this work presents a paradigm of continuous data-driven group learning support by incorporating the peer and teacher evaluation scores as an input to the subsequent algorithmic grouping.