Search

A tool is presented that can automatically assess the quality of students' concept maps and provide feedback based on a reference concept map. It is shown that this tool can effectively assess the quality of concept maps, and that it can provide accurate and helpful feedback on a number of specific shortcomings often evident in students' concept maps. However, it was also found that students who had access to feedback often did not request it, or did not take full advantage of the feedback given.

Self-reflection and reflective writing have been pivotal for developing a deep understanding of concepts and fostering professional competency in learners. The confluence of the importance of reflective practices within the educational curriculum and the increased proliferation of technology have resulted in numerous studies of how to use automated approaches to analyze broad themes of reflection exhibited by learners in higher educational settings. However, there is a dearth of research in the context of automated analysis of reflective writing demonstrated by professionals within workplace learning. Building on a four-level reflective content analysis model, this article evaluates the depth of reflection exhibited by learners within a professional development MOOC using an automated assessment classifier. Our results identify the varying association of linguistic features across these different levels of reflection. The proposed approach can effectively support the deployment at scale of the labor-intensive evaluation of reflective writing by highly trained professionals. Refereed/Peer-reviewed

Error messages provided by the programming environments are often cryptic and confusing to learners. This study explored the effectiveness of enhanced programming error messages (EPEMs) in a Python-based introductory programming course. Participants were two groups of middle school students. The control group only received raw programming error messages (RPEMs) and had 35 students. The treatment group received EPEMs and had 33 students. During the class, students used an automated assessment tool called Mulberry to practice their programming skill. Mulberry automatically collected all the solutions students submitted when solving programming problems. Data analysis was based on 6339 student solutions collected by Mulberry. Our results showed that EPEMs did not help to reduce student errors or improve students’ performance in debugging. The ineffectiveness of EPEMs may result from reasons such as the inaccuracy of the interpreter’s error messages or students not reading the EPEMs. However, the viewpoint of productive failure may provide a better explanation of the ineffectiveness of EPEMs. The failures in coding and difficulties in debugging can be resources for learning. We recommend that researchers reconsider the role of errors in code and investigate whether and how failures and debugging contribute to the learning of programming.

As incorporating software testing into programming assignments becomes routine, educators have begun to assess not only the correctness of students' software, but also the adequacy of their tests. In practice, educators rely on code coverage measures, though its shortcomings are widely known. Mutation analysis is a stronger measure of test adequacy, but it is too costly to be applied beyond the small programs developed in introductory programming courses. We demonstrate how to adapt mutation analysis to provide rapid automated feedback on software tests for complex projects in large programming courses. We study a dataset of 1389 student software projects ranging from trivial to complex. We begin by showing that although the state-of-the-art in mutation analysis is practical for providing rapid feedback on projects in introductory courses, it is prohibitively expensive for the more complex projects in subsequent courses. To reduce this cost, we use a statistical procedure to select a subset of mutation operators that maintains accuracy while minimizing cost. We show that with only 2 operators, costs can be reduced by a factor of 2-3 with negligible loss in accuracy. Finally, we evaluate our approach on open-source software and report that our findings may generalize beyond our educational context. (c) 2021 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). National Science Foundation, United States of AmericaNational Science Foundation (NSF) [DUE-1625425, DLR-1740765] We are grateful to the National Science Foundation, United States of America for their support under grants DUE-1625425 and DLR-1740765. We thank the anonymous peer reviewers whose feedback improved this paper.