Search

Background: Learning Analytics (LA) is an emerging field concerned with measuring, collecting, and analysing data about learners and their contexts to gain insights into learning processes. As the technology of Learning Analytics is evolving, many systems are being implemented. In this context, it is essential to understand stakeholders' expectations of LA across Higher Education Institutions (HEIs) for large-scale implementations that take their needs into account. Objectives: This study aims to contribute to knowledge about individual LA expectations of European higher education students. It may facilitate the strategy of stakeholder buy-in, the transfer of LA insights across HEIs, and the development of international best practices and guidelines. Methods: To this end, the study employs a 'Student Expectations of Learning Analytics Questionnaire' (SELAQ) survey of 417 students at the Goethe University Frankfurt (Germany) Based on this data, Multiple Linear Regressions are applied to determine how these students position themselves compared to students from Madrid (Spain), Edinburgh (United Kingdom) and the Netherlands, where SELAQ had already been implemented at HEIs. Results and Conclusions: The results show that students' expectations at Goethe University Frankfurt are rather homogeneous regarding 'LA Ethics and Privacy' and 'LA Service Features'. Furthermore, we found that European students generally show a consistent pattern of expectations of LA with a high degree of similarity across the HEIs examined. European HEIs face challenges more similar than anticipated. The HEI experience with implementing LA can be more easily transferred to other HEIs, suggesting standardized LA rather than tailor-made solutions designed from scratch.

Massive open online courses (MOOCs) pose a challenge for instructors when trying to provide personalised support to learners, due to large numbers of registered participants. Conversational agents can be of help to support learners when working with MOOCs. This article presents an adaptive learning module for JavaPAL, a conversational agent that complements a MOOC on Java programming, helping learners review the key concepts of the MOOC. This adaptive learning module adapts the difficulty of the questions provided to learners considering their level of knowledge using item response theory (IRT) and also provides recommendations of video fragments extracted from the MOOC for when learners fail questions. The adaptive learning module for JavaPAL has been evaluated showing good usability and learnability through the system usability scale (SUS), reasonably suitable video fragments recommendations for learners, and useful visualisations generated as part of the IRT-based adaptation of questions for instructors to better understand what is happening in the course, to design exams, and to redesign the course content.

In Latin American universities, Learning Analytics (LA) has been perceived as a promising opportunity to leverage data to meet the needs of a diverse student cohort. Although universities have been collecting educational data for years, the adoption of LA in this region is still limited due to the lack of expertise and policies for processing and using educational data. In order to get a better picture of how existing data-related practices and policies might affect the incorporation of LA in Latin American institutions, we conducted a mixed methods study in four Latin American universities (two Chilean and two Ecuadorian). In this paper, the qualitative data were based on 37 interviews with managers and 16 focus groups with 51 teaching staff and 45 students; the quantitative data were collected through two surveys answered by 1884 students and 368 teachers, respectively. The findings reveal opportunities to incorporate LA services into existing data practices in the four case studies. However, the lack of reliable information systems and policies to regulate the use of data imposes challenges that need to be overcome for future LA adoption.

To assist higher education institutions in meeting the challenge of limited student engagement in the implementation of Learning Analytics services, the Questionnaire for Student Expectations of Learning Analytics (SELAQ) was developed. This instrument contains 12 items, which are explained by a purported two-factor structure of "Ethical and Privacy Expectations" and "Service Feature Expectations." As it stands, however, the SELAQ has only been validated with students from UK university, which is problematic on account of the interest in Learning Analytics extending beyond this context. Thus, the aim of the current work was to assess whether the translated SELAQ can be validated in three contexts (an Estonian, a Spanish, and a Dutch University). The findings show that the model provided acceptable fits in both the Spanish and Dutch samples, but was not supported in the Estonian student sample. In addition, an assessment of local fit is undertaken for each sample, which provides important points that need to be considered in future work. Finally, a general comparison of expectations across contexts is undertaken, which are discussed in relation to the General Data Protection Regulation (2018).

This paper surveys the state of the art on prediction in MOOCs through a systematic literature review (SLR). The main objectives are: first, to identify the characteristics of the MOOCs used for prediction, second, to describe the prediction outcomes, third, to classify the prediction features, fourth, to determine the techniques used to predict the variables, and, fifth, to identify the metrics used to evaluate the predictive models. Results show there is strong interest in predicting dropouts in MOOCs. A variety of predictive models are used, though regression and support vector machines stand out. There is also wide variety in the choice of prediction features, but clickstream data about platform use stands out. Future research should focus on developing and applying predictive models that can be used in more heterogeneous contexts (in terms of platforms, thematic areas, and course durations), on predicting new outcomes and making connections among them (e.g., predicting learners' expectancies), on enhancing the predictive power of current models by improving algorithms or adding novel higher-order features (e.g., efficiency, constancy, etc.).

One of the reported methods of cheating in online environments in the literature is CAMEO (Copying Answers using Multiple Existences Online), where harvesting accounts are used to obtain correct answers that are later submitted in the master account which gives the student credit to obtain a certificate. In previous research, we developed an algorithm to identify and label submissions that were cheated using the CAMEO method; this algorithm relied on the IP of the submissions. In this study, we use this tagged sample of submissions to i) compare the influence of student and problems characteristics on CAMEO and ii) build a random forest classifier that detects submissions as CAMEO without relying on IP, achieving sensitivity and specificity levels of 0.966 and 0.996, respectively. Finally, we analyze the importance of the different features of the model finding that student features are the most important variables towards the correct classification of CAMEO submissions, concluding also that student features have more influence on CAMEO than problem features.

One of the characteristics of massive open online courses (MOOCs) is that the overall number of social interactions tend to be higher than in traditional courses, hindering the analysis of social learning. Learners typically ask or answer questions using the forum. This makes messages a rich source of information, which can be used to infer learners' behavior and outcomes. It is not feasible for teachers to process all forum messages and automated tools and analysis are required. Although there are some tools for analyzing learners' interactions, there is a need for methodologies and integrated tools that help to interpret the learning process based on social interactions in the forum. This paper presents the 3S (Social, Sentiments, Skills) learning analytics methodology for analyzing forum interactions in MOOCs. This methodology considers a temporal analysis combining the social, sentiments, and skill dimensions that can be extracted from forum data. We also introduce LAT[turned E]S, a learning analytics tool for edX/Open edX related to the three dimensions (3S), which includes visualizations to guide the proposed methodology. We apply the 3S methodology and the tool to an MOOC on Java programming. Results showed, among others, the action-reaction effect produced when learners increase their participation after instructor's events. Moreover, a decrease of positive sentiments over time and before deadlines of open-ended assignments was also observed and that there were certain skills, which caused more troubles (e.g., arrays and loops). These results acknowledge the importance of using a learning analytics methodology to detect problems in MOOCs.

Educational computer-based competition environments need to be designed with a set of features that enhance the learning process. Although recently some frameworks for the design of educational computer-based systems (e.g., educational games) have been proposed, they do not focus on the details of the competition. Therefore, the design of educational computer-based competition environments is still an open issue. We propose the 4NESS framework for the design of such competition environments. This framework classifies the competition features into four dimensions: easiness, fairness, awareness, and adaptiveness. We designed the ISCARE educational computer-based competition environment implementing these four dimensions to evaluate and validate some included features for competition, and conducted experiments during three consecutive years. Students considered the computer-based competition environment with the proposed features, including a modification of the Swiss system, to be very easy to use, fair for pairing students, moderately fair for calculating scores, with good awareness of their state during the competition and that of their peers, and with contents of appropriate level, especially for the group working with adaptive contents. Adaptation of contents made students increase their learning gains in 0.5 sigmas during the competition. Pairings were seen as with students of similar levels. And final scores were moderately fair: while the order tends to be similar as compared to the results in the post-test, students grades with high performance tend to be lower in the competition environment.