Search

The gradual shift to online modes of learning in higher education institutions over the past 2 decades accelerated drastically on a global scale between 2020 and 2022. Students and educators, who have initially grappled with the shift, have now become accustomed to online teaching; however, there are concerns about the quality of learning that has resulted. To enable a sustainable and effective online pedagogy, educators may need to learn about fostering higher-order thinking skills, which can be challenging even for experienced educators. To conceptualise effective online pedagogy, the community of inquiry (CoI) framework emphasises cognitive presence (CP), which focuses on the higher-order thinking process. The CoI is the most widely researched framework in online pedagogy, yet contemporary CoI literature lacks collective evidence of factors that influence CP. This scoping review of the CoI literature explores the factors that influence the higher-order thinking that is indicative of CP. Inclusion criteria included evidence of CP in online learning contexts and published between January 2000 and March 2022, providing a total of 121 studies. Results suggest that teaching presence, structure of learning activities and student characteristics all influence CP. Implications for practice or policy: Higher education students enrolled in online courses should be taught how to learn effectively in an online mode. Online course educators must embed learning tasks that foster self-regulation and higher-order skills in students. Online course design should include authentic tasks for students to apply new knowledge to real-life scenarios. Educators must be offered ample professional development activities to build their skills in online pedagogy. Institutions should encourage translation of online educational research to practice.

Refereed/Peer-reviewed The success and satisfaction of students with online courses is significantly impacted by the sequencing of learning objectives and activities. Equally critical is designing online degree programs and structuring multiple courses to reduce learners’ cognitive load and attain maximum learning success. In its current form, the evaluation of program design is complex and done primarily using stakeholders’ perceptions and qualitative methods, which are subjective. Emerging technologies produce enormous amounts of educational data that have the potential to lead to evidence-based practice to reform the design of study programs. Although a few studies adopt such data-driven approaches to evaluate and enhance study programs, remarkably little is known about the sequencing of learning objectives and the influence of their mastery in one course on performance across subsequent courses. This study aims to assess the interdependency of learning objectives from multiple courses in a MOOC-based program by drawing on an interdisciplinary approach intersecting analytic techniques and measurement models. Situated in the context of professional leadership skill development, we ground our findings in leadership theory and how learners acquire skills over time and across varying contexts.

Refereed/Peer-reviewed Many educational institutions are using predictive models to leverage actionable insights using student data and drive student success. A common task has been predicting students at risk of dropping out for the necessary interventions to be made. However, issues of discrimination by these predictive models based on protected attributes of students have recently been raised. An important question that is constantly asked is: should the protected attributes be excluded from the learning analytics (LA) models in order to ensure fairness? In this work, we aimed at answering questions that if we exclude the protected attributes from the LA models, does the exclusion ensure fairness as it should? Does the exclusion affect the performance of the LA model? If so, why? We found answers to these questions and went further to explain why. We built machine learning models and performed empirical evaluations using a 3-year dropout data for a particular program in a large Australian university. We found that excluding or including the protected attributes had marginal effect on predictive performance and fairness. Perhaps not surprisingly, our findings suggest that the effect of including or excluding protected attributes is a function of how they relate with the prediction outcome. More specifically, if a protected attribute is correlated with the target label and proves to be an important feature, then their inclusion or exclusion would have effect on the performance and fairness and vice versa. Our findings provide insightful information that can be used by relevant stakeholders to make well-informed decisions.

Machine Learning (ML) techniques have been increasingly adopted to support various activities in education, including being applied in important contexts such as college admission and scholarship allocation. In addition to being accurate, the application of these techniques has to be fair, i.e., displaying no discrimination towards any group of stakeholders in education (mainly students and instructors) based on their protective attributes (e.g., gender and age). The past few years have witnessed an explosion of attention given to the predictive bias of ML techniques in education. Though certain endeavors have been made to detect and alleviate predictive bias in learning analytics, it is still hard for newcomers to penetrate. To address this, we systematically reviewed existing studies on predictive bias in education, and a total of 49 peer-reviewed empirical papers published after 2010 were included in this study. In particular, these papers were reviewed and summarized from the following three perspectives: (i) protective attributes, (ii) fairness measures and their applications in various educational tasks, and (iii) strategies for enhancing predictive fairness. These findings were summarized into recommendations to guide future endeavors in this strand of research, e.g., collecting and sharing more quality data containing protective attributes, developing fairness-enhancing approaches which do not require the explicit use of protective attributes, validating the effectiveness of fairness-enhancing on students and instructors in real-world settings. Refereed/Peer-reviewed

The advent of technological developments is allowing to gather large amounts of data in several research fields. Learning analytics (LA)/educational data mining has access to big observational unstructured data captured from educational settings and relies mostly on unsupervised machine learning (ML) algorithms to make sense of such type of data. Generalized additive models for location, scale, and shape (GAMLSS) are a supervised statistical learning framework that allows modeling all the parameters of the distribution of the response variable with respect to the explanatory variables. This article overviews the power and flexibility of GAMLSS in relation to some ML techniques. Also, GAMLSS' capability to be tailored toward causality via causal regularization is briefly commented. This overview is illustrated via a data set from the field of LA. This article is categorized under: Application Areas > Education and Learning Algorithmic Development > Statistics Technologies > Machine Learning, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 13 (1)

Refereed/Peer-reviewed Technological affordances have shown promising potential in advancing the delivery of corporate learning programmes designed for professional leadership development. However, there is a considerable challenge in evaluating learners' skill acquisition, with most of the past research relying on pre- and post-tests or other forms of self-reports to measure leadership development. In that sense, these approaches measure leadership development before and after the programme, while being inefficient for measuring the development during the learning process. This study collected self-reflection answers from a professional development MOOC that allows learners to express their stepwise learning and reflect on their professional experience on leadership fronts. We developed a novel methodology and an automated system for the evaluation of leadership skills' mastery based on the depth of reflection exhibited during the learning process. We identified four groups of learners based on their course content mastery and explored the differences within groups. The results also highlight relevant insights about instructional design and provide promising avenues for future research.

Self-reflection and reflective writing have been pivotal for developing a deep understanding of concepts and fostering professional competency in learners. The confluence of the importance of reflective practices within the educational curriculum and the increased proliferation of technology have resulted in numerous studies of how to use automated approaches to analyze broad themes of reflection exhibited by learners in higher educational settings. However, there is a dearth of research in the context of automated analysis of reflective writing demonstrated by professionals within workplace learning. Building on a four-level reflective content analysis model, this article evaluates the depth of reflection exhibited by learners within a professional development MOOC using an automated assessment classifier. Our results identify the varying association of linguistic features across these different levels of reflection. The proposed approach can effectively support the deployment at scale of the labor-intensive evaluation of reflective writing by highly trained professionals. Refereed/Peer-reviewed

Refereed/Peer-reviewed Well-designed online courses enhance learning experiences and allow effective development of learners' skills and knowledge. A critical factor contributing to the design of online courses in the higher education settings are well-defined learning objectives that align with course assessments and learning activities. While there are several introspective instruments to evaluate course designs, with the broader adoption of educational technologies and digital tools, there is a wealth of data that offers insights on the alignment of learning objectives to assessments. Such data has paved the way for evidence-based methods of investigating course effectiveness within higher education. This study outlines a methodology for designing and evaluating the alignment between course learning objectives and assessment activities at scale, utilising a combination of learning analytics and measurement theory approaches, more specificially exploratory multi-dimensional item response theory (MIRT) models. We demonstrate the proposed methodology within a professional development MOOC on leadership skills development, where we evaluate the alignment between course objectives and reflective writing assessments activities. Our results suggested that the alignment of the existing course objectives to assessment activities can be improved, showing the practical value of the proposed approach. The theoretical and practical implications of this research are further illustrated.