Search

Many educational institutions are using predictive models to leverage actionable insights using student data and drive student success. A common task has been predicting students at risk of dropping out for the necessary interventions to be made. However, issues of discrimination by these predictive models based on protected attributes of students have recently been raised. An important question that is constantly asked is: should the protected attributes be excluded from the learning analytics (LA) models in order to ensure fairness? In this article, we aimed at answering questions that if we exclude the protected attributes from the LA models, does the exclusion ensure fairness as it supposedly should? Does the exclusion affect the performance of the LA model? If so, why? We found answers to these questions and went further to explain why. We built machine learning models and performed empirical evaluations using a three-year dropout data for a particular program in a large Australian university. We found that excluding or including the protected attributes had marginal effect on predictive performance and fairness. Perhaps not surprisingly, our findings suggest that the effect of including or excluding protected attributes is a function of how they relate with the prediction outcome. More specifically, if a protected attribute is correlated with the target label and proves to be an important feature, then their inclusion or exclusion would have effect on the performance and fairness and vice versa. Our findings provide insightful information that can be used by relevant stakeholders to make well-informed decisions.

Technological affordances have shown promising potential in advancing the delivery of corporate learning programmes designed for professional leadership development. However, there is a considerable challenge in evaluating learners' skill acquisition, with most of the past research relying on pre- and post-tests or other forms of self-reports to measure leadership development. In that sense, these approaches measure leadership development before and after the programme, while being inefficient for measuring the development during the learning process. This study collected self-reflection answers from a professional development MOOC that allows learners to express their stepwise learning and reflect on their professional experience on leadership fronts. We developed a novel methodology and an automated system for the evaluation of leadership skills' mastery based on the depth of reflection exhibited during the learning process. We identified four groups of learners based on their course content mastery and explored the differences within groups. The results also highlight relevant insights about instructional design and provide promising avenues for future research.

Reference to teachers as agents of change has become commonplace in the education literature, including change toward more inclusive practice in response to the changing demographic of schooling. Yet, little is known about how teacher agency relates to (1) their understanding of, and commitment to any given change agenda and (2) the institutional and social structures through which they are able to access knowledge and resources within and beyond their schools. This study combined social and epistemic network analysis to examine teachers' understanding of change and their sense of agency as they use their social networks to mobilise support for furthering change that matters to them. Our study is the first to apply this learning analytic approach in a real setting context. We used theories of teacher agency and inclusive pedagogy to interpret teachers' social interactions in light of the extent to which they seek to make a difference toward greater inclusion. We collected data with an online log completed by teachers and other staff in two schools in Sweden over 6 months. The findings suggest that teachers understanding of change is embedded in their day-to-day activities such as student support, lesson planning, improvement of programs, and working conditions. Teachers tend to exercise agency toward inclusion when they seek to support student learning and well-being. When teachers act as agents of change, their social networks are bigger, more diverse and more collaborative than in situations in which they act as role implementers. We discuss substantive and methodological implications of these findings.

The gradual shift to online modes of learning in higher education institutions over the past 2 decades accelerated drastically on a global scale between 2020 and 2022. Students and educators, who have initially grappled with the shift, have now become accustomed to online teaching; however, there are concerns about the quality of learning that has resulted. To enable a sustainable and effective online pedagogy, educators may need to learn about fostering higher-order thinking skills, which can be challenging even for experienced educators. To conceptualise effective online pedagogy, the community of inquiry (CoI) framework emphasises cognitive presence (CP), which focuses on the higher-order thinking process. The CoI is the most widely researched framework in online pedagogy, yet contemporary CoI literature lacks collective evidence of factors that influence CP. This scoping review of the CoI literature explores the factors that influence the higher-order thinking that is indicative of CP. Inclusion criteria included evidence of CP in online learning contexts and published between January 2000 and March 2022, providing a total of 121 studies. Results suggest that teaching presence, structure of learning activities and student characteristics all influence CP.

The success and satisfaction of students with online courses is significantly impacted by the sequencing of learning objectives and activities. Equally critical is designing online degree programs and structuring multiple courses to reduce learners' cognitive load and attain maximum learning success. In its current form, the evaluation of program design is complex and done primarily using stakeholders' perceptions and qualitative methods, which are subjective. Emerging technologies produce enormous amounts of educational data that have the potential to lead to evidence-based practice to reform the design of study programs. Although a few studies adopt such data-driven approaches to evaluate and enhance study programs, remarkably little is known about the sequencing of learning objectives and the influence of their mastery in one course on performance across subsequent courses. This study aims to assess the interdependency of learning objectives from multiple courses in a MOOC-based program by drawing on an interdisciplinary approach intersecting analytic techniques and measurement models. Situated in the context of professional leadership skill development, we ground our findings in leadership theory and how learners acquire skills over time and across varying contexts.

The capacity to foster interpersonal interactions in massive open online courses (MOOCs) has frequently been contested, particularly when learner interactions are limited to MOOC forums. The establishment of social presence--a perceived sense of somebody being present and "real"--is among the strategies to tackle the challenges of online learning and could be applied in MOOCs. Thus far, social presence in MOOCs has been under-researched. Studies that previously examined social presence in MOOCs did not account for the peculiar nature of open online learning. In contrast to the existing work, this study seeks to understand how learners perceive social presence, and the different nuances of social presence in diverse MOOC populations. In particular, we compare perceptions of social presence across the groups of learners with different patterns of forum participation in three edX MOOCs. The findings reveal substantial differences in how learners with varying forum activity perceive social presence. Perceptions of social presence also differed in courses with the varying volume of forum interaction and duration. Finally, learners with sustained forum activity generally reported higher social presence scores that included low affectivity and strong group cohesion perceptions. With this in mind, this study is significant because of the insights in brings to the current body of knowledge around social presence in MOOCs. The study's findings also raise questions about the effectiveness of transferring existing socio-constructivist constructs into the MOOC contexts.

Dual-layer MOOCs are an educational framework designed to create customizable modality pathways through a learning experience. The basic premise is to design two framework choices through a course: one that is instructor centered and the other that is student determined and open. Learners have the option to create their own customized pathway by choosing or combining both modalities as they see fit at any given time in the course. This exploratory mixed-methods study sought to understand the patterns that learners engaged in during a course designed with this pathway framework. The results of the quantitative examination of the course activity are presented, as well as the categories and themes that arose from the qualitative research. The results of the analysis indicate that learners value the ability to choose the pathway that they engage the course in. Additional research is needed to improve the technical and design aspects of the framework.

In this article, we empirically validate Tinto's Student Integration model, in particular, the predictions the model makes regarding both students' academic outcomes and their dropout decisions. In doing so, we analyze three decades' worth of student enrollments at an Australian university and present a novel methodological approach using graph embedding techniques to capture both structural and neighborhood-based features of the co-enrollment network. In keeping with Tinto's model, we find that not only do these embedded representations of students' social network predict their final grade point average (GPA), but also are able to successfully classify students who dropout. Our results show that these embedded representations of a student's social network can achieve F1-scores of up to 0.79 when classifying dropout and explain up to 10% of the variance in student's final GPA. When controlling for a small set of covariates and variables common to the literature, this performance increases to 0.83 and 24%, respectively. Furthermore, the performance of these methods is robust to both changes in their parameterization and to corruption of the underlying social networks. Importantly, this implies that hyperparameters may be selected to reduce the computational demands of this method without loss of predictive power. The novelty of this method, and its ability to identify student dropout, merits further investigation to preemptively identify at-risk students.

Research and development in learning analytics has established viable solutions for scaling personalised feedback to all students. However, questions remain regarding how such feedback is perceived, interpreted and acted upon by stakeholders. The present study reports on the analysis of focus group data from four courses to understand students' sensemaking of personalised, learning analytics-based feedback. Deductive analysis and epistemic network analysis were carried out to identify students' perceptions and associated emotional responses to the feedback. The results demonstrate that students mostly held positive perceptions of their feedback -- especially in regard to quality, task-focus and instructor care. While emotions to feedback were often negative, students also noted this increased their motivation to invest more effort in the learning tasks. The results of the study provide further evidence of the interpersonal communication variables influencing students' proactive recipience of feedback. Importantly, the findings show that students were mostly actively engaged with their personalised feedback, and that the instructor-mediated approach to feedback based on learning analytics is aligned to the principles of effective feedback.

Learning in computer-mediated setting represents a complex, multidimensional process. This complexity calls for a comprehensive analytical approach that would allow for understanding of various dimensions of learner generated discourse and the structure of the underlying social interactions. Current research, however, primarily focuses on manual or, more recently, supervised methods for discourse analysis. Moreover, discourse and social structures are typically analyzed separately without the use of computational methods that can offer a holistic perspective. This paper proposes an approach that addresses these two challenges, first, by using an unsupervised machine learning approach to extract speech acts as representations of knowledge construction processes and finds transition probabilities between speech acts across different messages, and second, by integrating the use of discovered speech acts to explain the formation of social ties and predicting course outcomes. We extracted six categories of speech acts from messages exchanged in discussion forums of two MOOCs and each category corresponded to knowledge construction processes from well-established theoretical models. We further showed how measures derived from discourse analysis explained the ways how social ties were created that framed emerging social networks. Multiple regression models showed that the combined use of measures derived from discourse analysis and social ties predicted learning outcomes.

Although technological advances have brought about new opportunities for scaling feedback to students, there remain challenges in how such feedback is presented and interpreted. There is a need to better understand how students make sense of such feedback to adapt self-regulated learning processes. This study examined students' sense-making of learning analytics--based personalised feedback across four courses. Results from a combination of thematic analysis and epistemic network analysis show an association between student perceptions of their personalised feedback and how these map to subsequent self-described self-regulated learning processes. Most notably, the results indicate that personalised feedback, elaborated by personal messages from course instructors, helps students refine or strengthen important forethought processes of goal-setting, as well as to reduce procrastination. The results highlight the need for instructors to increase the dialogic element in personalised feedback in order to reduce defensive reactions from students who hold to their own learning strategies. This approach may prompt reflection on the suitability of students' current learning strategies and achievement of associated learning goals. Implications for practice or policy: (1) Personalised feedback based on learning analytics should be informed by an understanding of students' self-regulated learning; (2) Instructors implementing personalised feedback should align this closely with the course curriculum; (3) Instructors implementing personalised feedback in their courses should consider the relational element of feedback by using a positive tone; and (4) Personalised feedback can be further enhanced by increasing the dialogic element and by including more information about learning strategies.

With widespread adoption of Learning Management Systems (LMS) and other learning technology, large amounts of data--commonly known as trace data--are readily accessible to researchers. Trace data has been extensively used to calculate time that students spend on different learning activities--typically referred to as time-on-task. These measures are used to build predictive models of student learning in order to understand and improve learning processes. While time-on-task measures have been used in Learning Analytics research, the consequences of their use are not fully described or examined. This paper presents findings from two experiments regarding different time-on-task estimation methods and their influence on research findings. Based on modelling different student performance measures with popular statistical methods in two datasets (one online, one blended), our findings indicate that time-on-task estimation methods play an important role in shaping the final study results, particularly in online settings where the amount of interaction with LMS is typically higher. The primary goal of this paper is to raise awareness and initiate debate on the important issue of time-on-task estimation within the broader learning analytics community. Finally, the paper provides an overview of commonly adopted time-on-task estimation methods in educational and related research fields.

This paper reports on the results of an analysis of the research proposals submitted to the MOOC Research Initiative (MRI) funded by the Gates Foundation and administered by Athabasca University. The goal of MRI was to mobilize researchers to engage into critical interrogation of MOOCs. The submissions--266 in Phase 1, out of which 78 was recommended for resubmission in the extended form in Phase 2, and finally, 28 funded--were analyzed by applying conventional and automated content analysis methods as well as citation network analysis methods. The results revealed the main research themes that could form a framework of the future MOOC research: i) student engagement and learning success, ii) MOOC design and curriculum, iii) self-regulated learning and social learning, iv) social network analysis and networked learning, and v) motivation, attitude and success criteria. The theme of social learning received the greatest interest and had the highest success in attracting funding. The submissions that planned on using learning analytics methods were more successful. The use of mixed methods was by far the most popular. Design-based research methods were also suggested commonly, but the questions about their applicability arose regarding the feasibility to perform multiple iterations in the MOOC context and rather a limited focus on technological support for interventions. The submissions were dominated by the researchers from the field of education (75% of the accepted proposals). Not only was this a possible cause of a complete lack of success of the educational technology innovation theme, but it could be a worrying sign of the fragmentation in the research community and the need to increased efforts towards enhancing interdisciplinarity.

Teaching and learning in networked settings has attracted significant attention recently. The central topic of networked learning research is human-human and human-information interactions occurring within a networked learning environment. The nature of these interactions is highly complex and usually requires a multi-dimensional approach to analyze their effects. Therefore, the main goal of this research is the development of a theoretical model of networked learning that allows for a comprehensive and scalable analysis of how and why learners engage in collaboration in networked communities. The proposed research method, anticipated research outcomes, and contributions to the learning analytics field are discussed.

Despite a surge of empirical work on student participation in online learning environments, the causal links between the learning-related factors and processes with the desired learning outcomes remain unexplored. This study presents a systematic literature review of approaches to model learning in Massive Open Online Courses offering an analysis of learning-related constructs used in the prediction and measurement of student engagement and learning outcome. Based on our literature review, we identify current gaps in the research, including a lack of solid frameworks to explain learning in open online setting. Finally, we put forward a novel framework suitable for open online contexts based on a well-established model of student engagement. Our model is intended to guide future work studying the association between contextual factors (i.e., demographic, classroom, and individual needs), student engagement (i.e., academic, behavioral, cognitive, and affective engagement metrics), and learning outcomes (i.e., academic, social, and affective). The proposed model affords further interstudy comparisons as well as comparative studies with more traditional education models.

This article reports on the findings of a study that investigated the effects of instructional conditions and prior experience on students' self-reflection. The study was conducted with the use of a video annotation tool that was used by undergraduate performing arts students to reflect on their video-recorded performances. The study shows a consistent positive effect of previous experience with the video annotation tool for engagement with reflection. Graded instructional conditions with feedback had a positive effect on increasing higher order reflections particularly for students with prior experience with the video annotation tool for reflective purposes. The finding suggests that when including reflection in the curriculum, it is important to consider introducing it at a program or degree level rather than individual courses in order to provide an opportunity for students to gain experience with reflection and any particular tool that is used (e.g., a video annotation tool). Furthermore, reflective tasks should be scaffolded into the curriculum with ample opportunity for formative feedback and summative assessment in order to encourage higher order thinking and foster students' metacognitive awareness and monitoring for increased goal-setting and acknowledgement of the motive or effect of their observed behavior.

In recent years, a wide array of tools have emerged for the purposes of conducting educational data mining (EDM) and/or learning analytics (LA) research. In this article, we hope to highlight some of the most widely used, most accessible, and most powerful tools available for the researcher interested in conducting EDM/LA research. We will highlight the utility that these tools have with respect to common data preprocessing and analysis steps in a typical research project as well as more descriptive information such as price point and user-friendliness. We will also highlight niche tools in the field, such as those used for Bayesian knowledge tracing (BKT), data visualization, text analysis, and social network analysis. Finally, we will discuss the importance of familiarizing oneself with multiple tools--a data analysis toolbox--for the practice of EDM/LA research.

The field of learning analytics was founded with the goal to harness vast amounts of data about learning collected by the extensive use of technology. After the early formation, the field has now entered the next phase of maturation with a growing community who has an evident impact on research, practice, policy, and decision-making. Although learning analytics is a bricolage field borrowing from many related other disciplines, there is still no systematised model that shows how these different disciplines are pieced together. Existing models and frameworks of learning analytics are valuable in identifying elements and processes of learning analytics, but they insufficiently elaborate on the links with foundational disciplines. With this in mind, this paper proposes a consolidated model of the field of research and practice that is composed of three mutually connected dimensions--theory, design, and data science. The paper defines why and how each of the three dimensions along with their mutual relations is critical for research and practice of learning analytics. Finally, the paper stresses the importance of multi-perspective approaches to learning analytics based on its three core dimensions for a healthy development of the field and a sustainable impact on research and practice.

One of the striking differences between massive open online courses (MOOCs) and previous innovations in the education technology field is the unprecedented interest and involvement of the general public. As MOOCs address pressing problems in higher education and the broader educational practice, awareness of the general public debate around MOOCs is essential. Understanding the public discourse around MOOCs can provide insights into important social and public problems, thus enabling the MOOC research community to better focus their research endeavors. While there have been some reports looking at the state of the MOOC-related research, the analysis of the public debate surrounding MOOCs is still largely missing. In this paper, we present the results of a study that looked at the content of the public discourse related to MOOCs. We identified the most important themes and topics in MOOC-related mainstream news reports. Our results indicate that coverage of MOOCs in public media is rapidly decreasing: by the middle of 2014, it decreased by almost 50% from the highest activity during 2013. In addition, the focus of those discussions is also changing. While the majority of discussions during 2012 and 2013 were focused on MOOC providers, the announcements of their partnerships, and million dollar investments, the current focus of MOOC discourse seems to be moving toward more productive topics focused on the overall position of MOOCs in the global educational landscape. Among different topics that this study discovered, government-related issues and the use of data and analytics are some of the topics that seem to be growing in popularity during the first half of 2014.