Search

The editorial looks back at the journal in 2022 and forward to 2023. For this editorial, we analysed all ‘Notes for Practice’ published in the journal from when they first appeared in issue 5(1) to the end of November, 2022. Our goals were to examine critically the ways in which these notes have been used to foster collaboration between researchers and practitioners, and also to summarise key findings that practitioners can use to inform their work. Our analysis covers 434 Notes for Practice from 130 different papers. The full dataset used for this analysis is provided as a supplementary file.

The creation of machine learning-based predictive models of student success has risen as a central activity in the fields of artificial intelligence in education, learning analytics, and educational data mining. Fueled by the high availability of data on learner interactions with technology, as well as the broad availability of computational methods to deal with these data, predictive modeling in education tends to have a lesser emphasis on producing learning theories and causal interpretations of data and a greater emphasis on predicting accurate coarse-grained learning outcomes for individual students. This chapter presents an overview of the common data sources, discusses the recent developments, and identifies the critical issues in predictive modeling of student success. Open challenges in the area include model explainability, fairness and bias in models, model generalizability and transfer to new circumstances, and the educational actionability of models.

The term Learner Profile has proliferated over the years, and more recently, with the increased advocacy around personalising learning experiences. Learner profiles are at the center of personalised learning, and the characterisation of diversity in classrooms is made possible by profiling learners based on their strengths and weaknesses, backgrounds and other factors influencing learning. In this paper, we discuss three common approaches of profiling learners based on students’ cognitive knowledge, skills and competencies and behavioral patterns, all latter commonly used within Learning Analytics (LA). Although each approach has its strengths and merits, there are also several disadvantages that have impeded adoption at scale. We propose that the broader adoption of learner profiles can benefit from careful combination of the methods and practices of three primary approaches, allowing for scalable implementation of learner profiles across educational systems. In this regard, LA can leverage from other aligned domains to develop valid and rigorous measures of students' learning and propel learner profiles from education research to more mainstream educational practice. LA could provide the scope for monitoring and reporting beyond an individualised context and allow holistic evaluations of progress. There is promise in LA research to leverage the growing momentum surrounding learner profiles and make a substantial impact on the field's core aim - understanding and optimising learning as it occurs. Refereed/Peer-reviewed

One of the major factors affecting student learning is feedback. Although the importance of feedback has been recognized in educational institutions, dramatic changes - such as bigger class sizes and a more diverse student population - challenged the provision of effective feedback. In light of these changes, educators have increasingly been using new digital tools to provide student feedback, given the broader adoption and availability of these new technologies. However, despite these efforts, most educators have limited insight into the recipience of their feedback and wonder which students engage with feedback. This problem is referred to as the "feedback gap," which is the difference between the potential and actual use of feedback, preventing educators and instructional designers from understanding feedback recipience among students. In this study, a set of trackable call-to-action (CTA) links were embedded in feedback messages focused on learning processes and self-regulation of learning in one fully online marketing course and one blended bioscience course. These links helped us examine the association between feedback engagement and course success. We also conducted two focus groups with students from one of the courses to further examine student perceptions of feedback messages. Our results across both courses revealed that early engagement with feedback is positively associated with passing the course and that most students considered feedback messages helpful in their learning. Our study also found some interesting demographic differences between students regarding their engagement with the feedback messages. Such insight enables instructors to ask "why" questions, support students' learning, improve feedback processes, and narrow the gap between potential and actual use of feedback. The practical implications of our findings are further discussed.

Learning in computer-mediated setting represents a complex, multidimensional process. This complexity calls for a comprehensive analytical approach that would allow for understanding of various dimensions of learner generated discourse and the structure of the underlying social interactions. Current research, however, primarily focuses on manual or, more recently, supervised methods for discourse analysis. Moreover, discourse and social structures are typically analyzed separately without the use of computational methods that can offer a holistic perspective. This paper proposes an approach that addresses these two challenges, first, by using an unsupervised machine learning approach to extract speech acts as representations of knowledge construction processes and finds transition probabilities between speech acts across different messages, and second, by integrating the use of discovered speech acts to explain the formation of social ties and predicting course outcomes. We extracted six categories of speech acts from messages exchanged in discussion forums of two MOOCs and each category corresponded to knowledge construction processes from well-established theoretical models. We further showed how measures derived from discourse analysis explained the ways how social ties were created that framed emerging social networks. Multiple regression models showed that the combined use of measures derived from discourse analysis and social ties predicted learning outcomes. Refereed/Peer-reviewed

Refereed/Peer-reviewed Student-facing visualisations have attracted increased attention with recent developments in data-driven tools to support individual and group work. Learning analytics dashboards (LADs), a data-enhanced feedback tool that allows students to make sense of their learning by providing insights into their learning behaviours represents one of the prominent examples of this trend. While these visualisation tools are increasingly used to study and enhance students' learning in academic contexts, current research is limited regarding the effects of the LADs in K-12 environments. There is a missed opportunity to empower teams and allow instructors and researchers to understand how teams use the LAD to regulate their learning. In this study, we developed a K-12 LAD for supporting students' collaborative work and evaluated with respect to students' perceived usefulness of the proposed LAD and the association between its use and course performance. The study followed a mixed-methods approach, combining quantitative analysis of log data from the dashboard and qualitative analysis of students’ perceptions using surveys and focus groups. Our results show that different roles within teams have distinguished engagement patterns with the LAD and that the tool improves the collaborative learning experience. We postulate that the implications of this study will aid future research work when investigating the behaviours of teams and optimising their learning using LADs.

Refereed/Peer-reviewed The development of group cohesion for small groups in face-to-face educational learning settings has been widely studied for decades. The calibre of group cohesion within these contexts is typically associated with increases in performance, resulting in a productive collaboration that helps drive knowledge building and innovation within a learning environment. However, understanding and measuring cohesion in technology-mediated learning experiences still requires additional research. This paper reports on a systematic literature review of small groups in technology-mediated learning settings to explore the definitions of cohesion and the methodologies used for the measurements that have been identified arising in this area. Furthermore, the review provides a characterisation of the factors that impact collaborative learning in computer-supported environments, highlighting two important aspects: i) there is a significant gap in measuring cohesion using unobtrusive methods, such as relying on log or trace data and ii) there is a lack of longitudinal approaches to understand the emergence of cohesion. Finally, we discuss implications for future research in studying group cohesion.

Refereed/Peer-reviewed This paper focuses on the area of success-enablers in learning analytics (LA) adoption from the perspective of senior managers in higher education institutions (HEIs). A significant body of academic literature exists about challenges in LA. However, to date, the success-enablers from the perspectives of institutional senior managers have received limited attention. This research aims to address this gap reporting on the findings of a study that conducted a series of semi-structure interviews with senior managers at 44 European HEIs. A detailed thematic analysis was conducted on the interviews to tease out the main success-enablers. Then, connections of different success-enablers were analyzed using epistemic network analysis (ENA). The analysis showed that the success-enablers in HEIs that had fully adopted LA depended on the involvement of high-level stakeholders, setting an embedded strategy, getting a technology support from the external partnership, or having a strategic analytical culture. The HEIs that were preparing or only partly adopted LA depended on success-enablers such as having a developing analytical culture or a delegation of expertise in LA-related activity. The findings of this study can help HEIs create strategies that can support successful adoption of LA.

Refereed/Peer-reviewed Technology is omnipresent in our rapidly changing digital world. Capitalising on these technologies is paramount for education systems to support a modern workforce. This study presents a review of 10 countries, highlighting how different public education systems have integrated digital technologies for teaching and learning. These education systems were selected based on their high performance and continuous improvement in international assessments. Four main themes emerged as key areas of focus from our analysis: improved and enhanced teaching and learning; digital maturation, including tackling the first and second digital divide; data-driven decision making and evidence-based education; and AI in education. Our findings will help benchmark current approaches and uses of digital teaching and learning, including perceived challenges and opportunities that can be used to inform the development and implementation of digital education policies.

Existing studies have shed light on policies and strategies for learning analytics (LA) adoption, yet there is limited understanding of associations among factors that influence adoption processes or the change in priorities when institutional experience with LA increases. This paper addresses this gap by presenting a study based on interviews with institutional leaders from 27 European higher education institutions. Results showed that experienced institutions demonstrated more interest in exploring learning behaviour and pedagogical reformation than simply measuring a phenomenon. Experienced institutions also paid more attention to methodological approaches to LA than data constraints, and demonstrated a broader involvement of teachers and students. This paper also identifies inter-related connections between prevailing challenges that impede the scaling of LA. Based on the results, we suggest regular evaluations of LA adoption to ensure the alignment of strategy and desired changes. We also identify three areas that require particular attention when forming short-term goals for LA at different phases of adoption Refereed/Peer-reviewed

In this article, we empirically validate Tinto's Student Integration model, in particular, the predictions the model makes regarding both students' academic outcomes and their dropout decisions. In doing so, we analyze three decades' worth of student enrollments at an Australian university and present a novel methodological approach using graph embedding techniques to capture both structural and neighborhood-based features of the co-enrollment network. In keeping with Tinto's model, we find that not only do these embedded representations of students' social network predict their final grade point average (GPA), but also are able to successfully classify students who dropout. Our results show that these embedded representations of a student's social network can achieve $F_1$-scores of up to 0.79 when classifying dropout and explain up to 10% of the variance in student's final GPA. When controlling for a small set of covariates and variables common to the literature, this performance increases to 0.83 and 24%, respectively. Furthermore, the performance of these methods is robust to both changes in their parameterization and to corruption of the underlying social networks. Importantly, this implies that hyperparameters may be selected to reduce the computational demands of this method without loss of predictive power. The novelty of this method, and its ability to identify student dropout, merits further investigation to preemptively identify at-risk students. Refereed/Peer-reviewed

Discussion forums are the primary medium for supporting in-course student interactions in digital learning settings. Despite the significant uptake of discussion forums, questions remain as to how the tool can be used to initiate, maintain, and support interpersonal student connections. Large-scale patterns of student online interactions in forums derived from across the university are under-explored. Most studies have relied on data derived from a single course. This study presents a multi-site analysis of student interactions in online course forums at the university-level. Digital interactions of 14,643 students were analysed across several years in three universities located in the North American, South-East Asian, and Pacific regions. Descriptive results indicate that students with similar grades tend to co-participate in learning discussions. We applied exponential random graph modelling and regression analysis to further understand this observed similarity. Results suggest that this phenomenon can be explained by social processes of selection only to a small extent, and even less so by peer influence mechanisms. The study suggests that occurrence of similarity stems from other factors, such as course and forum designs. The implications of these results raise questions regarding learning designs and the benefits linked to the formation of student connections based on grade similarity Refereed/Peer-reviewed

This paper presents a study that examined automated cross-language classification of online discussion messages for the levels of cognitive presence, a key construct from the widely used Community of Inquiry (CoI) model of online learning. Specifically, we examined the classification of 1,500 Portuguese language discussion messages using a classifier trained on a corpus of the 1,747 English language discussion messages. In the study, a random forest classifier was developed using a small set of 108 validated indicators of psychological processes, linguistic coherence, and online discussion structure. The classifier obtained 67% accuracy and Cohen's κ of 0.32, showing a moderate level of inter-rater agreement above chance and the general viability of the proposed approach. Most importantly, the findings suggest that certain aspects of cognitive presence construct are highly generalizable and transfer across different languages. Finally, the paper also presents a novel method for addressing class imbalance problem using a generic algorithm heuristic technique, which provided substantial improvements over the use of imbalanced dataset. Results and practical implications are further discussed. Refereed/Peer-reviewed

Feedback is a major factor of student success within higher education learning. However, recent changes – such as increased class sizes and socio-economic diversity of the student population – challenged the provision of effective student feedback. Although the use of educational technology for personalised feedback to diverse students has gained traction, the feedback gap still exists: educators wonder which students respond to feedback and which do not.In this study, a set of trackable Call to Action (CTA) links was embedded in two sets of feedback messages focusing on students’time management, with the goal of (1) examining the association between feedback engagement and course success and (2), to predict students’ reaction to provided feedback. We also conducted two focus groups to further examine students’ perception of provided feedback messages. Our results revealed that early engagement with the feedback was associated with higher chances of succeeding in the course. Likewise, previous engagement with feedback was highly predictive of students’ engagement in the future, and also that certain student sub-populations, (e.g., female students),were more likely to engage than others. Such insight enables instructors to ask “why” questions, improve feedback processes and narrow the feedback gap. Practical implications of our findings are further discussed. Refereed/Peer-reviewed

This study presents an evaluation of the Community of Inquiry (CoI) survey instrument developed by Arbaugh et al. (2008) within the context of Massive Open Online Courses (MOOCs). The study reports the results of a reliability analysis and exploratory factor analysis of the CoI survey instrument using the data of 1487 students from five MOOC courses. The findings confirmed the reliability and validity of the CoI survey instrument for the assessment of the key dimensions of the CoI model: teaching presence, social presence, and cognitive presence. Although the CoI survey instrument captured the same latent constructs within the MOOC context as in the Garrison's three-factor model (Garrison et al., 1999), analyses suggested a six-factor model with additional three factors as a better fit to the data. These additional factors were 1) course organization and design (a sub-component of teaching presence), 2) group affectivity (a sub-component of social presence), and 3) resolution phase of inquiry learning (a sub-component of cognitive presence). The emergence of these additional factors revealed that the discrepancies between the dynamics of the traditional online courses and MOOCs affect the student perceptions of the three CoI presences. Based on the results of our analysis, we provide an update to the famous CoI model which captures the distinctive characteristics of the CoI model within the MOOC setting. The results of the study and their implications are further discussed. Refereed/Peer-reviewed

Connectivist pedagogies are geared towards building a network of learners that actively employ technologies to establish interpersonal connections in open online settings. In this context, as course participants increasingly establish interpersonal relationships among peers they have greater opportunity to draw on and leverage the latent social capital that resides in such a distributed learning environment. However, to date there have been a limited number of studies exploring how learners build their social capital in open large-scale courses. To inform the facilitation of learner networks in open online settings and beyond, this study analyzed factors associated with how learners accumulate social capital in the form of learner connections over time. The study was conducted in two massive open online course offerings (Connectivism and Connective Knowledge) that were designed on the principles of connectivist pedagogy and that made use of data about social interaction from Twitter, blogs, and Facebook. For this purpose, linear mixed modeling was used to understand the associations between learner social capital, linguistic and discourse patterns, media used for interaction, as well as the time in the course when interaction took place. The results highlight the association between the language used by the learners and the creation of ties between them. Analyses on the accumulation of connections over time have implications for the pedagogical choices that would be expected to help learners leverage access to potential social capital in a networked context. Refereed/Peer-reviewed

The field of learning analytics was founded with the goal to harness vast amounts of data about learning collected by the extensive use of technology. After the early formation, the field has now entered the next phase of maturation with a growing community who has an evident impact on research, practice, policy, and decision-making. Although learning analytics is a bricolage field borrowing from many related other disciplines, there is still no systematised model that shows how these different disciplines are pieced together. Existing models and frameworks of learning analytics are valuable in identifying elements and processes of learning analytics, but they insufficiently elaborate on the links with foundational disciplines. With this in mind, this paper proposes a consolidated model of the field of research and practice that is composed of three mutually connected dimensions – theory, design, and data science. The paper defines why and how each of the three dimensions along with their mutual relations is critical for research and practice of learning analytics. Finally, the paper stresses the importance of multi-perspective approaches to learning analytics based on its three core dimensions for a healthy development of the field and a sustainable impact on research and practice.

This paper presents an approach to studying relationships between students' social presence and course topics from transcripts of asynchronous discussions in online learning environments. Specifically, the paper uses topic modeling and epistemic network analysis to investigate how students' social presence is expressed across different course topics. Finally, we show how this method can be adopted to examine how students' social presence changed due to an instructional intervention. The results of this study and its implications are further discussed. Refereed/Peer-reviewed

The knowledge required by teachers has long been a focus of public and academic attention. Following a period of intense research interest in teachers’ knowledge in the 1980s and 1990s, many researchers have adopted Shulman’s suggestion that expert teaching practice is based on seven forms of knowledge which collectively are referred to as a knowledge base for teaching. Shulman’s work also offered a decision-making framework known as pedagogical reasoning and action, which allows teachers to use their seven forms of knowledge to make effective pedagogical decisions. Despite the widespread acceptance of these ideas, no empirical evidence exploring the connections between knowledge and decision-making is evident in the research literature.This paper reports on a pilot study in which the connections between knowledge and decisions in science, mathematics and information technology teachers’ lesson plans are quantified and represented using epistemic network analysis. Findings reveal and levels of complexity that have been intimated but, until now, not supported with empirical evidence. Refereed/Peer-reviewed

This paper examines the discrete learning strategies employed within a massive open online course and their relationship to the student learning experience. The theoretical framework centered on the Community of Inquiry model of online education, which outlines the three critical dimensions (presences) of student learning experience: teaching, social, and cognitive presence. The Community of Inquiry survey instrument, administered as the part of the post-course survey, was used to measure student perceived levels of the three presences. Cluster analysis revealed three different groups of students with unique study strategies: limited users, selective users, and broad users. The strategies adopted significantly differed in student use of available tools and resources, final course grade, as well as the perceived levels of cognitive presence. The results also indicate there were significant differences regarding student commitment to learning, motivations and goals for enrolling in a MOOC, as well as goal orientation, approaches to learning, and the use of different study strategies. Implications for research and practice of online learning are further discussed. Refereed/Peer-reviewed

Student engagement is often considered an overarching construct in educational research and practice. Though frequently employed in the learning analytics literature, engagement has been subjected to a variety of interpretations and there is little consensus regarding the very definition of the construct. This raises grave concerns with regards to construct validity: namely, do these varied metrics measure the same thing? To address such concerns, this paper proposes, quantifies, and validates a model of engagement which is both grounded in the theoretical literature and described by common metrics drawn from the field of learning analytics. To identify a latent variable structure in our data we used exploratory factor analysis and validated the derived model on a separate sub-sample of our data using confirmatory factor analysis. To analyze the associations between our latent variables and student outcomes, a structural equation model was fitted, and the validity of this model across different course settings was assessed using MIMIC modeling. Across different domains, the broad consistency of our model with the theoretical literature suggest a mechanism that may be used to inform both interventions and course design. Refereed/Peer-reviewed

Online learning and massive open online courses are widely used in engineering and technology education. Engineering next-generation learning requires overcoming the potential constraints of online learning environments which necessitate higher levels of self-regulation than traditional classroom settings. This particular requirement demands that learners allocate their cognitive, metacognitive, affective and motivational resources to meet this need. Lack of self-regulation can affect learners' engagement with the course content, resulting in sub-optimal learning outcomes or failure to complete the course. This paper reports on the design of a virtual learning assistant and its implementation in online learning activities. This paper outlines the virtual assistant's use as a data collection tool and, further, proposes that the virtual learning assistant has the potential to be used as an assessment tool for self-regulatory skills, and as an intervention tool to support online learners' self-regulation in online learning. Refereed/Peer-reviewed

Massive Open Online Courses have provided researchers with considerable opportunities to assess student learning within technically mediated online environments. However, for most reported studies, analyses are conducted from a single MOOC or over multiple MOOCs with different learner enrolments. Thus, this limits the opportunities to assess the changing behavior of learners over time. In this paper, the behavioral engagement of 175 students was examined, who were enrolled in a professional development study program consisting of four different MOOCs, comprising a complete program of study. A comprehensive analysis was conducted to understand the changing behavior of learners within each course and across the MOOC program. To do this, we used latent class analysis, a clustering technique widely used in other fields, but mostly unexplored within educational technology research. Our results revealed six weekly learning strategies based on student course engagement and three different program-level learning strategies, which not only differed in their learning behavior but also their final academic outcomes. Our study also showed substantial effects of MOOC course design on the level of student engagement in the courses. The results and implications are further discussed. Refereed/Peer-reviewed

In recent years, a wide array of tools have emerged for the purposes of conducting educational data mining (EDM) and/or learning analytics (LA) research. In this article, we hope to highlight some of the most widely used, most accessible, and most powerful tools available for the researcher interested in conducting EDM/LA research. We will highlight the utility that these tools have with respect to common data preprocessing and analysis steps in a typical research project as well as more descriptive information such as price point and user-friendliness. We will also highlight niche tools in the field, such as those used for Bayesian knowledge tracing (BKT), data visualization, text analysis, and social network analysis. Finally, we will discuss the importance of familiarizing oneself with multiple tools—a data analysis toolbox—for the practice of EDM/LA research. Refereed/Peer-reviewed

Dual-layer MOOCs are an educational framework designed to create customizable modality pathways through a learning experience. The basic premise is to design two framework choices through a course: one that is instructor centered and the other that is student determined and open. Learners have the option to create their own customized pathway by choosing or combining both modalities as they see fit at any given time in the course. This exploratory mixed-methods study sought to understand the patterns that learners engaged in during a course designed with this pathway framework. The results of the quantitative examination of the course activity are presented, as well as the categories and themes that arose from the qualitative research. The results of the analysis indicate that learners value the ability to choose the pathway that they engage the course in. Additional research is needed to improve the technical and design aspects of the framework. Refereed/Peer-reviewed

Reflective writing has been widely recognized as one of the most effective activities for fostering students’ reflective and critical thinking. The analysis of students’ reflective writings has been the focus of many research studies. However, to date this has been typically a very labor-intensive manual process involving content analysis of student writings. With recent advancements in the field of learning analytics, there have been several attempts to use text analytics to examine student reflective writings. This paper presents the results of a study examining the use of theoretically-sound linguistic indicators of different psychological processes for the development of an analytics system for assessment of reflective writing. More precisely, we developed a random-forest classification system using linguistic indicators provided by the LIWC and Coh-Metrix tools. We also examined what particular indicators are representative of the different types of student reflective writings Refereed/Peer-reviewed

This paper presents a method for automated content analysis of students’ messages in asynchronous discussions written in Portuguese. In particular, the paper looks at the problem of coding discussion transcripts for the levels of cognitive presence, a key construct in a widely used Community of Inquiry model of online learning. Although there are techniques to coding for cognitive presence in the English language, the literature is still poor in methods for others languages, such as Portuguese. The proposed method uses a set of 87 different features to create a random forest classifier to automatically extract the cognitive phases. The model developed reached Cohen’s \(\kappa \) of .72, which represents a “substantial” agreement, and it is above the Cohen’s \(\kappa \) threshold of .70, commonly used in the literature for determining a reliable quantitative content analysis. This paper also provides some theoretical insights into the nature of cognitive presence by looking at the classification features that were most relevant for distinguishing between the different phases of cognitive presence.

This paper presents a novel method for the evaluation of students’ use of asynchronous discussions in online learning environments. In particular, the paper shows how students’ cognitive development across different course topics can be examined using the combination of natural language processing and graph-based analysis techniques. Drawing on the theoretical foundation of the community of inquiry model, we show how topic modeling and epistemic network analysis can provide qualitatively new insight into students’ development of critical and deep thinking skills. We also show how the same method can be used to investigate the effectiveness of instructional interventions and its effect on student learning. The results of this study and its practical implications are further discussed Refereed/Peer-reviewed

The capacity to foster interpersonal interactions in massive open online courses (MOOCs) has frequently been contested, particularly when learner interactions are limited to MOOC forums. The establishment of social presence-a perceived sense of somebody being present and "real"-is among the strategies to tackle the challenges of online learning and could be applied in MOOCs. Thus far, social presence in MOOCs has been under-researched. Studies that previously examined social presence in MOOCs did not account for the peculiar nature of open online learning. In contrast to the existing work, this study seeks to understand how learners perceive social presence, and the different nuances of social presence in diverse MOOC populations. In particular, we compare perceptions of social presence across the groups of learners with different patterns of forum participation in three edX MOOCs. The findings reveal substantial differences in how learners with varying forum activity perceive social presence. Perceptions of social presence also differed in courses with the varying volume of forum interaction and duration. Finally, learners with sustained forum activity generally reported higher social presence scores that included low affectivity and strong group cohesion perceptions. With this in mind, this study is significant because of the insights into brings to the current body of knowledge around social presence in MOOCs. The study's findings also raise questions about the effectiveness of transferring existing socio-constructivist constructs into the MOOC contexts. Refereed/Peer-reviewed

This paper describes a study that looked at the effects of different technology-use profiles on educational experience within communities of inquiry, and how they are related to the students' levels of cognitive presence in asynchronous online discussions. Through clustering of students (N = 81) in a graduate distance education engineering course, we identified six different profiles: 1) task-focused users, 2) content-focused no-users, 3) no-users, 4) highly intensive users, 5) content-focused intensive users, and 6) socially-focused intensive users. Identified profiles significantly differ in terms of their use of learning platform and their levels of cognitive presence, with large effect sizes of 0.54 and 0.19 multivariate η < sup > 2 < /sup > , respectively. Given that several profiles are associated with higher levels of cognitive presence, our results suggest multiple ways for students to be successful within communities of inquiry. Our results also emphasize a need for a different instructional support and pedagogical interventions for different technology-use profiles. Refereed/Peer-reviewed

Contemporary literature on online and distance education almost unequivocally argues for the importance of interactions in online learning settings. Nevertheless, the relationship between different types of interactions and learning outcomes is rather complex. Analyzing 204 offerings of 29 courses, over the period of six years, this study aimed at expanding the current understanding of the nature of this relationship. Specifically, with the use of trace data about interactions and utilizing the multilevel linear mixed modeling techniques, the study examined whether frequency and duration of student-student, student-instructor, student-system, and student-content interactions had an effect of learning outcomes, measured as final course grades. The findings show that the time spent on student-system interactions had a consistent and positive effect on the learning outcome, while the quantity of student-content interactions was negatively associated with the final course grades. The study also showed the importance of the educational level and the context of individual courses for the interaction types supported. Our findings further confirmed the potential of the use of trace data and learning analytics for studying learning and teaching in online settings. However, further research should account for various qualitative aspects of the interactions used while learning, different pedagogical/media features, as well as for the course design and delivery conditions in order to better explain the association between interaction types and the learning achievement. Finally, the results might imply the need for the development of the institutional and program-level strategies for learning and teaching that would promote effective pedagogical approaches to designing and guiding interactions in online and distance learning settings. We examined the relationship between interaction types and learning out-come.The findings show significant positive effect of student-system interactions.Student-content interactions were negatively associated with the learning outcome.Educational level and course context are important for interaction types supported.

With the steady development of online education and online learning environments, possibilities to support social interactions between students have advanced significantly. This study examined the relationship between indicators of social presence and academic performance. Social presence is defined as students' ability to engage socially with an online learning community. The results of a multiple regression analysis showed that certain indicators of social presence were significant predictors of final grades in a master's level computer science online course. Moreover, the study also revealed that teaching presence moderated the association between social presence and academic performance, indicating that a course design that increased the level of meaningful interactions between students had a significant impact on the development of social presence, and thus could positively affect students' academic performance. This is especially important in situations when discussions are introduced to promote the development of learning outcomes assessed in courses. Another implication of our results is that indicators of social presence can be used for early detection of students at risk of failing a course. Findings inform research and practice in the emerging field of learning analytics by prompting the opportunities to offer actionable insights into the reasons why certain students are lagging behind.

Online learning has become very popular over the last decade. However, there are still many details that remain unknown about the strategies that students follow while studying online. In this study, we focus on the direction of detecting ‘invisible’ collaboration ties between students in online learning environments. Specifically, the paper presents a method developed to detect student ties based on temporal proximity of their assignment submissions. The paper reportson findings of a study that made use of the proposed method to investigate the presence of close submitters in two different massive open online courses. The results show that most of the students (i.e., student user accounts) were grouped as couples, though some bigger communities were also detected. The study also compared the population detected by the algorithm with the rest of user accounts and found that close submitters needed a statistically significantlower amount of activity with the platform to achieve a certificate of completion in a MOOC. These results confirm that the detected close submitters were performing some collaboration or even engaged in unethical behaviors, which facilitates their way into a certificate. However, more work is required in the future to specify various strategies adopted by close submitters and possible associations between the user accounts.

In this poster, we present the results of the study which examined the relationship between student differences in their use of the available technology and their perceived levels of cognitive presence within the MOOC context. The cognitive presence is a construct used to measure the level of practical inquiry in the Communities of Inquiry model. Our results revealed the existence of three clusters based on student technology use. The clusters significantly differed in terms of their levels of cognitive presence, most notably they differed on the levels of problem resolution.

In 2011, the phenomenon of MOOCs had swept the world of education and put online education in the focus of the public discourse around the world. Although researchers were excited with the vast amounts of MOOC data being collected, the benefits of this data did not stand to the expectations due to several challenges. The analyses of MOOC data are very time-consuming and labor-intensive, and require and require a highly advanced set of technical skills, often not available to the education researchers. Because of this MOOC data analyses are rarely done before the courses end, limiting the potential of data to impact the student learning outcomes and experience. In this paper we introduce MOOCito (MOOC intervention tool), a user-friendly software platform for the analysis of MOOC data, that focuses on conducting data-informed instructional interventions and course experimentations. We cover important design principles behind MOOCito and provide an overview of the trends in MOOC research leading to its development. Although a work-inprogress, in this paper, we outline the prototype of MOOCito and the results of a user evaluation study that focused on system's perceived usability and ease-of-use. The results of the study are discussed, as well as their practical implications. Refereed/Peer-reviewed

The field of learning analytics recently attracted attention from educational practitioners and researchers interested in the use of large amounts of learning data for understanding learning processes and improving learning and teaching practices. In this chapter, we introduce content analytics — a particular form of learning analytics focused on the analysis of different forms of educational content. We provide the definition and scope of content analytics and a comprehensive summary of the significant content analytics studies in the published literature to date. Given the early stage of the learning analytics field, the focus of this chapter is on the key problems and challenges for which existing content analytics approaches are suitable and have been successfully used in the past. We also reflect on the current trends in content analytics and their position within a broader domain of educational research.

There is an emerging trend in higher education for the adoption of massive open online courses (MOOCs). However, despite this interest in learning at scale, there has been limited work investigating how MOOC participants have changed over time. In this study, we explore the temporal changes in MOOC learners' language and discourse characteristics. In particular, we demonstrate that there is a clear trend within a course for language in discussion forums to be of both more on-Topic and reflective of deep learning in subsequent offerings of a course. We measure this in two ways, and demonstrate this trend through several repeated analyses of different courses in different domains. While not all courses show an increase beyond statistical significance, the majority do, providing evidence that MOOC learner populations are changing as the educational phenomena matures. Refereed/Peer-reviewed

Massive Open Online Courses represent a fertile ground for examining student behavior. However, due to their openness MOOC attract a diverse body of students, for the most part, unknown to the course instructors. However, a certain number of students enroll in the same course multiple times, and there are records of their previous learning activities which might provide some useful information to course organizers before the start of the course. In this study, we examined how student behavior changes between subsequent course offerings. We identified profiles of returning students and also interesting changes in their behavior between two enrollments to the same course. Results and their implications are further discussed. Refereed/Peer-reviewed

In this paper, we present the results of an exploratory study that examined the problem of automating content analysis of student online discussion transcripts. We looked at the problem of coding discussion transcripts for the levels of cognitive presence, one of the three main constructs in the Community of Inquiry (CoI) model of distance education. Using Coh-Metrix and LIWC features, together with a set of custom features developed to capture discussion context, we developed a random forest classification system that achieved 70.3% classification accuracy and 0.63 Cohen's kappa, which is significantly higher than values reported in the previous studies. Besides improvement in classification accuracy, the developed system is also less sensitive to overfitting as it uses only 205 classification features, which is around 100 times less features than in similar systems based on bag-of-words features. We also provide an overview of the classification features most indicative of the different phases of cognitive presence that gives an additional insights into the nature of cognitive presence learning cycle. Overall, our results show great potential of the proposed approach, with an added benefit of providing further characterization of the cognitive presence coding scheme. Refereed/Peer-reviewed

As the field of learning analytics continues to mature, there is a corresponding evolution and sophistication of the associated analytical methods and techniques. In this regard social network analysis (SNA) has emerged as one of the cornerstones of learning analytics methodologies. However, despite the noted importance of social networks for facilitating the learning process, it remains unclear how and to what extent such network measures are associated with specific learning outcomes. Motivated by Simmel's theory of social interactions and building on the argument that social centrality does not always imply benefits, this study aimed to further contribute to the understanding of the association between students' social centrality and their academic performance. The study reveals that learning analytics research drawing on SNA should incorporate both - descriptive and statistical methods to provide a more comprehensive and holistic understanding of a students' network position. In so doing researchers can undertake more nuanced and contextually salient inferences about learning in network settings. Specifically, we show how differences in the factors framing students' interactions within two instances of a MOOC affect the association between the three social network centrality measures (i.e., degree, closeness, and betweenness) and the final course outcome. Refereed/Peer-reviewed

The goal of this tutorial is to share data mining tools and techniques used by computer scientists with educational social scientists. We broadly define educational social scientists as being made up of people with backgrounds in the learning sciences, cognitive psychology, and educational research. The learning analytics community is heavily populated with researchers of these backgrounds, and we believe those that find themselves at the intersection of research, theory, and practice have a particular interest in expanding their knowledge of datadriven tools and techniques. Refereed/Peer-reviewed

All forms of learning take time. There is a large body of research suggesting that the amount of time spent on learning can improve the quality of learning, as represented by academic performance. The wide-spread adoption of learning technologies such as learning management systems (LMSs), has resulted in large amounts of data about student learning being readily accessible to educational researchers. One common use of this data is to measure time that students have spent on different learning tasks (i.e., time-on-task). Given that LMS systems typically only capture times when students executed various actions, time-on-task measures are estimated based on the recorded trace data. LMS trace data has been extensively used in many studies in the field of learning analytics, yet the problem of time-on-task estimation is rarely described in detail and the consequences tha t it entails are not fully examined. This paper presents the results of a study that examined the effects of different time-on-task estimation methods on the results of commonly adopted analytical models. The primary goal of this paper is to raise awareness of the issue of accuracy and appropriateness surrounding time-estimation within the broader learning analytics community, and to initiate a debate about the challenges of this process. Furthermore, the paper provides an overview of time-on-task estimation methods in educational and related research fields. Refereed/Peer-reviewed

With the widespread adoption of Learning Management Systems (LMS) and other learning technology, large amounts of data – commonly known as trace data – are being recorded and are readily accessible to educational researchers. Among different uses of trace data, it has been extensively used to calculate time that students spent on different learning activities – commonly referred to as student time-on-task. Extracted time-on-task measures are then used to build predictive models of student learning in order to understand and improve learning processes. While time-on-task measures have been extensively used in Learning Analytics research, the details of their estimation are rarely described and the consequences that this process entails are not fully examined.This paper presents findings from two experiments that looked at the different time-on-task estimation methods and how they influence the final research findings. Based on modeling different student performance measures with popular statistical methods in two datasets (one online and one blended), our findings indicate that time-on-task estimation methods play an important role in shaping the final study results. This is particularly true for online setting where the amount of interaction with LMS is typically higher. The primary goal of this paper is to raise awareness and initiate a debate on the important issue of time-on-task estimation within a broader learning analytics community. Finally, the paper provides an overview of commonly adopted time-on-task estimation methods in educational and related research fields.