Analytics of student interactions : towards theory-driven, actionable insights

The field of learning analytics arose as a response to the vast quantities of data that are increasingly generated about students, their engagement with learning resources, and their learning and future career outcomes. While the field began as a collage, adopting methods and theories from a variety of disciplines, it has now become a major area of research, and has had a substantial impact on practice, policy, and decision-making. Although the field supports the collection and analysis of a wide array of data, existing work has predominantly focused on the digital traces generated through interactions with technology, learning content, and other students. Yet for any analyses to support students and teachers, the measures derived from these data must (1) offer practical and actionable insight into learning processes and outcomes, and (2) be theoretically grounded. As the field has matured, a number of challenges related to these criteria have become apparent. For instance, concerns have been raised that the literature prioritises predictive modeling over ensuring that these models are capable of informing constructive actions. Furthermore, the methodological validity of much of this work has been challenged, as a swathe of recent research has found many of these models fail to replicate to novel contexts. The work presented in this thesis addresses both of these concerns. In doing so, our research is pervaded by three key concerns: firstly, ensuring that any measures developed are both structurally valid and generalise across contexts; secondly, providing actionable insight with regards to student engagement; and finally, providing representations of student interactions that are predictive of student outcomes, namely, grades and students’ persistence in their studies. This research programme is heavily indebted to the work of Vincent Tinto, who conceptually distinguishes between the interactions students have with the academic and social domains present within their educational institution. This model has been subjected to extensive empirical validation, using a range of methods and data. For instance, while some studies have relied upon survey responses, others have used social network metrics, demographic variables, and students’ time spent in class together to evaluate Tinto’s claims. This model provides a foundation for the thesis, and the work presented may be categorised into two distinct veins aligning with the academic and social aspects of integration that Tinto proposes. These two domains, Tinto argues, continually modify a student’s goals and commitments, resulting in persistence or eventual disengagement and dropout. In the former, academic domain, we present a series of novel methodologies developed for modeling student engagement with academic resources. In doing so, we assessed how an individual student’s behaviour may be modeled using hidden Markov models (HMMs) to provide representations that enable actionable insight. However, in the face of considerable individual differences and cross-course variation, the validity of such methods may be called into question. Accordingly, ensuring that any measurements of student engagement are both structurally valid, and generalise across course contexts and disciplines became a central concern. To address this, we developed our model of student engagement using sticky-HMMs, emphasised the more interpretable insight such an approach provides compared to competing models, demonstrated its cross-course generality, and assessed its structural validity through the successful prediction of student dropout. In the social domain, a critical concern was to ensure any analyses conducted were valid. Accordingly, we assessed how the diversity of social tie definitions may undermine the validity of subsequent modeling practices. We then modeled students’ social integration using graph embedding techniques, and found that not only are student embeddings predictive of their final grades, but also of their persistence in their educational institution. In keeping with Tinto’s model, our research has focused on academic and social interactions separately, but both avenues of investigation have led to the question of student disengagement and dropout, and how this may be represented and remedied through the provision of actionable insight.