1. Learning analytics driven improvements in learning design in higher education: A systematic literature review.
- Author
-
Drugova, Elena, Zhuravleva, Irina, Zakharova, Ulyana, and Latipov, Adel
- Subjects
- *
RESEARCH funding , *UNIVERSITIES & colleges , *ARTIFICIAL intelligence , *TEACHING methods , *DESCRIPTIVE statistics , *ANALYSIS of covariance , *SYSTEMATIC reviews , *ONLINE education , *CURRICULUM planning , *ANALYSIS of variance , *LEARNING strategies , *DATA analysis software , *MACHINE learning - Abstract
Background: Driven by the ongoing need to provide high‐quality learning and teaching, universities recently have shown an increased interest in using learning analytics (LA) for improving learning design (LD). However, the evidence of such improvements is scarce, and the maturity of such research is unclear. Objectives: This study is aimed to evaluate the maturity of research discussing LA‐driven LD improvements in higher education. Methods: The systematic review analyses 49 empirical papers, assesses their quality and suggests further research directions. The review elaborates on methodological (research questions, strategy and methods, LA‐LD integration theoretical backgrounds) and substantial (LA‐driven LD improvements, types of data used, LA software development) features of the papers. Results and Conclusions: The findings demonstrated the lack of theoretical alignment between LA and LD, with research tending towards user experience studies. The most frequently used research strategy was a case study; experiments were very rare. Researchers predominantly used parsing for collecting data and AI methods for analysing it; mostly used data types related to registering learners' engagement with learning activities as well as resources and tools provided in digital learning environments. Takeaways: The research area discussing LA‐driven LD improvements still has a way to go before attaining the level of full maturity. Only a third of the papers reported actual LA‐driven LD improvements; moreover, only three papers measured their effectiveness. The presented LA software was mostly at the beta or implementation stages and did not assess the impact of using this software. Lay Description: What is already known about this topic: Universities show an increased interest in using LA for improving LD.The evidence of such improvements being efficient is scarce.Learning analytics is weakly grounded in learning and teaching. What this paper adds: LA‐LD research presented in the reviewed papers cannot yet be considered fully mature.Almost half of the research was carried out using a case study approach, intrinsically challenging in terms of replication and validity. As few as three out of the reviewed studies applied an experimental approach, capable of stating the cause–effect relationship between LA and LD.All the studied papers mentioned improvements in LD, but only around a third of them reported actual improvements, another one‐third suggested potential improvements, and the rest did not come down to describing any.The LA software presented in the papers was mostly at the testing or implementation stages, and the impact of implemented software on LD has yet to be evaluated. Implications for practice and/or policy: Future LA‐LD research to become more mature may well elaborate on theoretical footing, contribute to the cause–effect relationship between LA and LD utilising experimental research design, ensure sufficient sample size.Practitioners' efforts should be directed to operationalising LD for efficient measurements, addressing pedagogical challenges when applying LA, and assessing LA‐driven improvements introduced into LD. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF