Search

In educational contexts, automated assessment tools (AAT) are commonly used to provide formative feedback on programming exercises. However, designing exercises for AAT remains a laborious task or imposes limitations on the exercises. Most AAT use either output comparison, where the generated output is compared against an expected output, or unit testing, where the tool has access to the code of the submission under test. While output comparison has the advantage of being programming language independent, the testing capabilities are limited to the output. Conversely, unit testing can generate more granular feedback, but is tightly coupled with the programming language of the submission. In this paper, we introduce TESTed, which enables the best of both worlds: combining the granular feedback of unit testing with the programming language independence of output comparison. Educators can save time by designing exercises that can be used across programming languages. Finally, we report on using TESTed in educational practice.

Self-reflection and reflective writing have been pivotal for developing a deep understanding of concepts and fostering professional competency in learners. The confluence of the importance of reflective practices within the educational curriculum and the increased proliferation of technology have resulted in numerous studies of how to use automated approaches to analyze broad themes of reflection exhibited by learners in higher educational settings. However, there is a dearth of research in the context of automated analysis of reflective writing demonstrated by professionals within workplace learning. Building on a four-level reflective content analysis model, this article evaluates the depth of reflection exhibited by learners within a professional development MOOC using an automated assessment classifier. Our results identify the varying association of linguistic features across these different levels of reflection. The proposed approach can effectively support the deployment at scale of the labor-intensive evaluation of reflective writing by highly trained professionals. [ABSTRACT FROM AUTHOR]

Error messages provided by the programming environments are often cryptic and confusing to learners. This study explored the effectiveness of enhanced programming error messages (EPEMs) in a Python-based introductory programming course. Participants were two groups of middle school students. The control group only received raw programming error messages (RPEMs) and had 35 students. The treatment group received EPEMs and had 33 students. During the class, students used an automated assessment tool called Mulberry to practice their programming skill. Mulberry automatically collected all the solutions students submitted when solving programming problems. Data analysis was based on 6339 student solutions collected by Mulberry. Our results showed that EPEMs did not help to reduce student errors or improve students' performance in debugging. The ineffectiveness of EPEMs may result from reasons such as the inaccuracy of the interpreter's error messages or students not reading the EPEMs. However, the viewpoint of productive failure may provide a better explanation of the ineffectiveness of EPEMs. The failures in coding and difficulties in debugging can be resources for learning. We recommend that researchers reconsider the role of errors in code and investigate whether and how failures and debugging contribute to the learning of programming. [ABSTRACT FROM AUTHOR]

Error messages provided by the programming environments are often cryptic and confusing to learners. This study explored the effectiveness of enhanced programming error messages (EPEMs) in a Python-based introductory programming course. Participants were two groups of middle school students. The control group only received raw programming error messages (RPEMs) and had 35 students. The treatment group received EPEMs and had 33 students. During the class, students used an automated assessment tool called Mulberry to practice their programming skill. Mulberry automatically collected all the solutions students submitted when solving programming problems. Data analysis was based on 6339 student solutions collected by Mulberry. Our results showed that EPEMs did not help to reduce student errors or improve students’ performance in debugging. The ineffectiveness of EPEMs may result from reasons such as the inaccuracy of the interpreter’s error messages or students not reading the EPEMs. However, the viewpoint of productive failure may provide a better explanation of the ineffectiveness of EPEMs. The failures in coding and difficulties in debugging can be resources for learning. We recommend that researchers reconsider the role of errors in code and investigate whether and how failures and debugging contribute to the learning of programming.

A tool is presented that can automatically assess the quality of students’ concept maps and provide feedback based on a reference concept map. It is shown that this tool can effectively assess the quality of concept maps, and that it can provide accurate and helpful feedback on a number of specific shortcomings often evident in students’ concept maps. However, it was also found that students who had access to feedback often did not request it, or did not take full advantage of the feedback given. [ABSTRACT FROM AUTHOR]

This study implemented a data-driven approach to identify Chinese high school students' common errors in a Java-based introductory programming course using the data in an automated assessment tool called the Mulberry. Students' error-related behaviors were also analyzed, and their relationships to success in introductory programming were investigated. This study identified 15 common compilation errors and 6 common test errors. The results showed that these common errors accounted for a large proportion of all errors, so identifying the common errors is important to help students succeed in introductory programming courses. Based on these common errors, five underlying student difficulties were identified and are discussed. In addition, after analyzing existing measures of students' error-related behaviors, we developed a measure called improvement rate to quantify students' success in fixing errors. The results of our study suggest that students' competence of improving code is important to their success in introductory programming. We recommend researchers design and develop automated assessment tools that provide feedback messages for common student errors and instructors who explicitly teach knowledge and skills of improving code in class. [ABSTRACT FROM AUTHOR]

Abstract: A simulation-based framework is discussed in this paper for quantification/evaluation of risk and development of automated risk assessment tools, focusing on applications to offshore wind turbines under extreme environmental conditions. The framework is founded on a probabilistic characterization of the uncertainty in the models for the excitation, the turbine and its performance. Risk is then quantified as the expected value of some risk consequence measure over the probability distributions considered for the uncertain model parameters. Stochastic simulation is proposed for the risk assessment, corresponding to the evaluation of some associated probabilistic integral quantifying risk, as it allows for the adoption of comprehensive computational models for describing the dynamic turbine behavior. For improvement of the computational efficiency, a surrogate modeling approach is introduced based on moving least squares response surface approximations. The assessment is also extended to a probabilistic sensitivity analysis that identifies the importance of each of the uncertain model parameters, i.e. risk factors, towards the total risk as well as towards each of the failure modes contributing to this risk. The versatility and computational efficiency of the advocated approaches is finally exploited to support the development of standalone risk assessment applets for automated implementation of the probabilistic risk quantification/assessment. [Copyright &y& Elsevier]