Search

There are many claims that gamification (i.e., using game elements outside games) impact decreases over time (i.e., the novelty effect). Most studies analyzing this effect focused on extrinsic game elements, while fictional and collaborative competition have been recently recommended. Additionally, to the best of our knowledge, no long-term research has been carried out with STEM learners from introductory programming courses (CS1), a context that demands encouraging practice and mitigating motivation throughout the semester. Therefore, the main goal of this work is to better understand how the impact of a gamification design, featuring fictional and competitive-collaborative elements, changes over a 14-week period of time, when applied to CS1 courses taken by STEM students (N = 756). In an ecological setting, we followed a 2x7 quasi-experimental design, where Brazilian STEM students completed assignments in either a gamified or non-gamified version of the same system, which provided the measures (number of attempts, usage time, and system access) to assess user behavior at seven points in time. Results indicate changes in gamification's impact that appear to follow a U-shaped pattern. Supporting the novelty effect, the gamification's effect started to decrease after four weeks, decrease that lasted between two to six weeks. Interestingly, the gamification's impact shifted to an uptrend between six and 10 weeks after the start of the intervention, partially recovering its contribution naturally. Thus, we found empirical evidence supporting that gamification likely suffers from the novelty effect, but also benefits from the familiarization effect, which contributes to an overall positive impact on students. These findings may provide some guidelines to inform practitioners about how long the initial contributions of gamification last, and how long they take to recover after some reduction in benefits. It can also help researchers to realize when to apply/evaluate interventions that use gamification by taking into consideration the novelty effect and, thereby, better understand the real impact of gamification on students' behavior in the long run.

Tools for automatic grading programming assignments, also known as Online Judges, have been widely used to support computer science (CS) courses. Nevertheless, few studies have used these tools to acquire and analyse interaction data to better understand the students' performance and behaviours, often due to data availability or inadequate granularity. To address this problem, we propose an Online Judge called CodeBench, which allows for fine-grained data collection of student interactions, at the level of, eg, keystrokes, number of submissions, and grades. We deployed CodeBench for 3 years (2016-18) and collected data from 2058 students from 16 introductory computer science (CS1) courses, on which we have carried out fine-grained learning analytics, towards early detection of effective/ineffective behaviours regarding learning CS concepts. Results extract clear behavioural classes of CS1 students, significantly differentiated both semantically and statistically, enabling us to better explain how student behaviours during programming have influenced learning outcomes. Finally, we also identify behaviours that can guide novice students to improve their learning performance, which can be used for interventions. We believe this work is a step forward towards enhancing Online Judges and helping teachers and students improve their CS1 teaching/learning practices.

Introductory computing courses have a high failure rate worldwide. At the Federal University of Amazonas, this also happens and, since 2016 a group of professors decided to reformulate the course at the institution and some learning analytics initiatives have been adopted. The reformulation included a review of the course program and the use of an online judge. After all these years of research, the group has enough material and data and it is a good moment to summarize what has been done and the achievements so far. In this article, the focus will be the learning analytics in three main areas: student performance prediction, classification of difficulty of programming exercises, and gamification. Also, as a contribution, for the first time in a journal, the whole dataset is available to the community. [ABSTRACT FROM AUTHOR]