Search

Artificial Intelligence (AI) can foster education but can also be misused to breach academic integrity. Large language models like ChatGPT are able to generate solutions for individual assessments that are expected to be completed independently. There are a number of automated detectors for AI assisted work. However, most of them are not dedicated to programming and/or they rely on existing student submissions (i.e., the learning approach). This paper presents a straightforward detector for AI assisted code, relying on code anomaly. No existing student submissions are needed. The detector employs 34 features covering constants, data structures, branches, loops, functions, and others. According to our evaluation on three data sets, the detector and its normalized variation are effective with 89% top-K precision. However, allowing discussion among colleagues and access to the internet might reduce the effectiveness by 25%. The effectiveness is further reduced by about the same amount when AI assistance is only used on some tasks, not the whole submissions. Although our detectors should be used with caution due to the limitations, it sufficiently shows that code anomaly can be distinctive for identifying AI assisted work. Instructors can start looking for the code anomaly among the submissions for such identification. [ABSTRACT FROM AUTHOR]

The importance of accurately predicting student performance in education, especially in the challenging curricular unit of Introductory Programming, cannot be overstated. As institutions struggle with high failure rates and look for solutions to improve the learning experience, the need for effective prediction methods becomes critical. This study aims to conduct a systematic review of the literature on methods for predicting student performance in higher education, specifically in Introductory Programming, focusing on machine learning algorithms. Through this study, we not only present different applicable algorithms but also evaluate their performance, using identified metrics and considering the applicability in the educational context, specifically in higher education and in Introductory Programming. The results obtained through this study allowed us to identify trends in the literature, such as which machine learning algorithms were most applied in the context of predicting students' performance in Introductory Programming in higher education, as well as which evaluation metrics and datasets are usually used. [ABSTRACT FROM AUTHOR]

Programming is acknowledged widely as a cornerstone skill in Computer Science education. Despite significant efforts to refine teaching methodologies, a segment of students is still at risk of failing programming courses. It is crucial to identify potentially struggling students at risk of underperforming or academic failure. This study explores the predictive potential of students' problem-solving skills through dynamic, domain-independent, complex problem-solving assessment. To evaluate the predictive potential of complex problem-solving empirically, a case study with 122 participants was conducted in the undergraduate Introductory Programming Course at the University of Maribor, Slovenia. A latent variable approach was employed to examine the associations. The study results showed that complex problem-solving has a strong positive effect on performance in Introductory Programming Courses. According to the results of structural equation modeling, 64% of the variance in programming performance is explained by complex problem-solving ability. Our findings indicate that complex problem-solving performance could serve as a significant, cognitive, dynamic predictor, applicable to the Introductory Programming Course. Moreover, we present evidence that the demonstrated approach could also be used to predict success in the broader computing education community, including K-12, and the wider education landscape. Apart from predictive potential, our results suggest that valid and reliable instruments for assessing complex problem-solving could also be used for assessing general-purpose, domain-independent problem-solving skills in computing education. Likewise, the results confirmed the positive effect of previous programming experience on programming performance. On the other hand, there was no significant direct effect of performance in High School mathematics on Introductory Programming. [ABSTRACT FROM AUTHOR]

Learning computer programming is typically difficult for newcomers. Demotivation and learned helplessness have received much attention. Besides the subject's intricacy, low in-class participation has been associated with poor student achievement. This paper presents a follow-up, stage 2 study on the novel instructional technique, Student-Driven Probe Instruction (SDPI), to address low in-class participation in programming courses. Instead of the teacher lecturing/explaining content to the class and asking questions, students were shown a snippet of code or other relevant material and given the option to ask questions beforehand. The study was conducted in two stages: stage 1 pilot and stage 2. This paper presents the results of stage 2, while stage 1 operations and results are discussed briefly. The number of questions asked in class, real-time Trello board postings, and emails/Slack conversations with the instructor were used to track participation. In-class participation showed significant improvement. Average quiz and in-class activity scores showed marginal gains. Results from the endof-course survey show that students preferred SDPI over the traditional lecture style since it stirred their interest in the content and provided them the confidence to ask questions in class. The study is purely exploratory in nature, and no conclusions can be drawn due to the extremely small sample size of the student population. [ABSTRACT FROM AUTHOR]

Cryptic and ambiguous error messages provided by programming environments are crucial barriers to beginners when learning to program. This study designed a coding manual to explain common Python programming errors to students using a data-driven approach, following guidelines suggested by prior studies. A quasi-experiment with two groups of middle school students was conducted to examine the effectiveness of the coding manual in reducing student errors, increasing confidence, and promoting learning performance. By analyzing 6015 erroneous student programs collected by the automated assessment tool (AAT) Mulberry, we found that the experimental group who used the coding manual during learning did not make fewer language specification errors (LSEs) in general. However, for LSEs indicating the actual errors in the program, the coding manual showed significant effects in reducing error frequencies. We also found that the coding manual failed to increase students’ confidence in programming and promote learning performance. Possible causes of the ineffectiveness may include high cognitive loads during programming, the productivity of learning from the debugging process, and the incompleteness of the explanations and examples in the coding manual. We recommend that computer programming instructors use AATs or similar tools to collect learning data and identify students’ common errors, directly explain identified common LSEs in class, and explicitly teach debugging methods and strategies. Future research should focus on students’ self-regulation during programming, better methods of explaining common errors to novices, and the long-term effects of using a coding manual on students’ learning in introductory programming.

The importance of accurately predicting student performance in education, especially in the challenging curricular unit of Introductory Programming, cannot be overstated. As institutions struggle with high failure rates and look for solutions to improve the learning experience, the need for effective prediction methods becomes critical. This study aims to conduct a systematic review of the literature on methods for predicting student performance in higher education, specifically in Introductory Programming, focusing on machine learning algorithms. Through this study, we not only present different applicable algorithms but also evaluate their performance, using identified metrics and considering the applicability in the educational context, specifically in higher education and in Introductory Programming. The results obtained through this study allowed us to identify trends in the literature, such as which machine learning algorithms were most applied in the context of predicting students’ performance in Introductory Programming in higher education, as well as which evaluation metrics and datasets are usually used.

The COVID-19 pandemic restrained the academic environment and changed the rules of the educational game; contact classes were restricted, and online assessments had to be introduced. This situation opened an opportunity for some students to use source-code plagiarism to pass coding assessments in the Introduction to Programming subject module. The focus of this paper is on making sense of this environment to establish a process to ensure that students obtain the skills they need to build on in subsequent modules. This is necessary to reach the outcomes of a computing course. Four aspects were used in establishing this source-code plagiarism awareness process in focusing on one class of students. Qualitative data were gathered by firstly requesting the class to supply feedback on their understanding of source-code plagiarism, and secondly inviting students identified as guilty of Python source-code plagiarism to start a conversation with the lecturer, which was triangulated with quantitative data regarding the success of the latter group in terms of their pass rate. Although the Measure of Software Similarity tool was instrumental in establishing a source-code plagiarism detection process, it is cumbersome and time consuming. Hence fourthly, it was compared to other available tools to determine their suitability in comparison. A refined source-code plagiarism awareness process is the resultant finding of this paper. [ABSTRACT FROM AUTHOR]

Copyright of MLS Educational Research is the property of MLS Journals S.L. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Novice programmers struggle to comprehend specific programming constructs such as arrays, recursion, and loops. One way to address this challenge is to provide practice problems for students in these topics that are considered difficult to comprehend — such as nested loops. It is proven that practice aids program comprehension, hence, since it is time consuming to manually create many practice problems; synthesising these problems is an Expert Artificial Intelligence Task that is worth investigating. In this paper we present the synthesis of nested loop exercises in Python for practice using a context-free grammar. We defined the grammar rules for modeling program templates. Algorithms were designed and implemented for the generation of structurally different nested loop exercises based on the defined production rules of the context-free grammar. Each program generated consists of two or more looping statements, with a nesting, and the context-free grammar rules also allow for the spawning of infinitely many sub-loops for every loop. We checked for the correctness of the synthesised nested loop programs with an experimental procedure of iteratively supplying these programs to the Python interpreter. The first 120,000 iterations returned no syntactic bugs, hence, showing that these programs do compile and have outputs. Furthermore, an evaluation of the programs was carried out in a survey with 210 participants (novice and expert programmers). The results of the evaluation show that over 82% of the participants agreed that the synthesised exercises were correctly generated, solvable and can be used as practice exercises for novice programmers in introductory programming modules. 120,000 iterations of synthesised loop programs can be found at: https://tinyurl.com/nestedloops2021.

Control-flow statements often cause misunderstandings among novice computer science students. To better address these problems, teachers need to know the misconceptions that are typical at this stage. In this paper, we present the results of studying students' misconceptions about control-flow statements. We compiled 181 questions, each containing an algorithm written in pseudocode and the execution trace of that algorithm. Some of the traces were correct; others contained highlighted errors. The students were asked to explain in their own words why the selected line of the trace was correct or erroneous. We collected and processed 10,799 answers from 67 CS1 students. Among the 24 misconceptions we found, 6 coincided with misconceptions from other studies, and 7 were narrower cases of known misconceptions. We did not find previous research regarding 11 of the misconceptions we identified. [ABSTRACT FROM AUTHOR]

Perceptual Learning Modules (PLMs) is a variation of Perceptual Learning based on multiple-choice questionnaires. There exists successful research of the use of PLMs in math and flight training. The possibility of designing and adopting PLMs in Introductory Programming Courses (CS1) is still an open area of study. The goal of this study is to test whether students that received a PLM training on recognising segments of programs will perform better at writing programs. Two PLM interventions were administered to students. The first intervention was a nonrandom controlled experiment, in which students opted to answer the PLM questionnaire (N=40), while the control group consisted of students that did not answer it (N=629). The second intervention was a randomized controlled experiment with a placebo, in which students were randomly assigned to perform either the PLM questionnaire (N=51) or another a placebo activity (N=51). The different forms of analysis of the first experiment results yielded Cohen's d ranging from 0.23 to 0.34 in favor of the PLM intervention. For the second experiment, the effect size was d = -0.11 against the PLM intervention, but the two results were significant. We believe that the cautious conclusion is that there is a null effect in using a PLM activity as part of a CS1 course. The paper is also of interest because of the methodological decisions and techniques used. [ABSTRACT FROM AUTHOR]

Programming is acknowledged widely as a cornerstone skill in Computer Science education. Despite significant efforts to refine teaching methodologies, a segment of students is still at risk of failing programming courses. It is crucial to identify potentially struggling students at risk of underperforming or academic failure. This study explores the predictive potential of students’ problem-solving skills through dynamic, domain-independent, complex problem-solving assessment. To evaluate the predictive potential of complex problem-solving empirically, a case study with 122 participants was conducted in the undergraduate Introductory Programming Course at the University of Maribor, Slovenia. A latent variable approach was employed to examine the associations. The study results showed that complex problem-solving has a strong positive effect on performance in Introductory Programming Courses. According to the results of structural equation modeling, 64% of the variance in programming performance is explained by complex problem-solving ability. Our findings indicate that complex problem-solving performance could serve as a significant, cognitive, dynamic predictor, applicable to the Introductory Programming Course. Moreover, we present evidence that the demonstrated approach could also be used to predict success in the broader computing education community, including K-12, and the wider education landscape. Apart from predictive potential, our results suggest that valid and reliable instruments for assessing complex problem-solving could also be used for assessing general-purpose, domain-independent problem-solving skills in computing education. Likewise, the results confirmed the positive effect of previous programming experience on programming performance. On the other hand, there was no significant direct effect of performance in High School mathematics on Introductory Programming.

Novice programmers struggle to comprehend specific programming constructs such as arrays, recursion, and loops. One way to address this challenge is to provide practice problems for students in these topics that are considered difficult to comprehend — such as nested loops. It is proven that practice aids program comprehension, hence, since it is time consuming to manually create many practice problems; synthesising these problems is an Expert Artificial Intelligence Task that is worth investigating. In this paper we present the synthesis of nested loop exercises in Python for practice using a context-free grammar. We defined the grammar rules for modeling program templates. Algorithms were designed and implemented for the generation of structurally different nested loop exercises based on the defined production rules of the context-free grammar. Each program generated consists of two or more looping statements, with a nesting, and the context-free grammar rules also allow for the spawning of infinitely many sub-loops for every loop. We checked for the correctness of the synthesised nested loop programs with an experimental procedure of iteratively supplying these programs to the Python interpreter. The first 120,000 iterations returned no syntactic bugs, hence, showing that these programs do compile and have outputs. Furthermore, an evaluation of the programs was carried out in a survey with 210 participants (novice and expert programmers). The results of the evaluation show that over 82% of the participants agreed that the synthesised exercises were correctly generated, solvable and can be used as practice exercises for novice programmers in introductory programming modules. 120,000 iterations of synthesised loop programs can be found at: https://tinyurl.com/nestedloops2021. [ABSTRACT FROM AUTHOR]

Especially in large courses, feedback is often given only on the final results; less attention is paid to the programming process. Today, however, some programming environments, e.g., Thonny, log activities during programming and have the functionality of replaying the programming process. This information can be used to provide feedback, and this knowledge can be integrated into practical sessions in the classroom. This study aimed to analyse how feedback based on logs affects exam results, task completion time, the number of runs, error messages, and pastes (of the whole group and beginners and non-beginners separately). An experiment was conducted in 2020 and 2021 in the course “Introduction to Programming”. Some groups received additional feedback on homework throughout the course based on log information; the remaining groups worked as usual. Based on the information received from the logs, general recommendations were also offered in the practical sessions. Our study showed that feedback based on logs improved mainly exam test results and programming task solving time among beginners. Therefore, it would be a good method to use, especially in beginner groups. [ABSTRACT FROM AUTHOR]

This article discusses the correlation between first-year students' prior knowledge of programming and their academic achievements. Many students who enroll in the Faculty of Computer Science and Information Technology at Riga Technical University drop out during their first year. Therefore, it is crucial to support first-year students in acquiring programming knowledge successfully. The Faculty of Computer Science and Information Technology provides an additional course called "Algorithmization Practice" for students without prior programming experience. The authors of this paper conducted research to investigate whether this course helps bridge the gap between students with and without prior programming knowledge and whether it improves the academic performance of students without prior knowledge in their first year. The collected data was summarized using descriptive statistics, and the final scores of students from different groups were compared to identify any differences between those who attended the "Algorithmization Practice" course and those who did not. Based on a sample of 1744 introductory programming course students, our analysis indicates that students who attended the additional course demonstrated better academic performance. [ABSTRACT FROM AUTHOR]

Teaching an introductory programming course to first-year students has long been a challenge for many college instructors. The COVID-19 pandemic, which caused unprecedented shifts in learning modality across the globe, has worsened the learning experience of novice programmers. Instructors have to find innovative ways to keep students engaged and learning. Blended or hybrid learning has become a new preferred way of learning during the COVID-19 pandemic. Blended learning is viewed as a combination of both in-person and online instructions. Such a learning environment offers instructors the flexibility to provide learners with an engaging face-to-face learning experience while promoting the well-being and safety of students. Starting Fall 2020, York College (and other CUNY colleges) has since offered several courses in hybrid mode. Two years have passed since the abrupt transition. There were several lessons learned from the experiences. In this paper, I discussed evidence-based pedagogical approaches that were used to teach students in an introductory computer programming class at York College, CUNY, where blended learning was used. Student perceptions of learning experience and obtaining coding skills in both online and in-person environments are also presented. The findings from the survey suggested that students benefited from face-to-face interactions and feedback, while those who preferred an online environment liked the flexibility that online components offer. Through careful design and implementation of pedagogical approaches used in the class, novice programmers could potentially benefit from both face-to-face and online components of blended learning. [ABSTRACT FROM AUTHOR]

Automated grading systems (autograders) assist the process of teaching in introductory programming courses (CS1). However, the sole focus on correctness can obfuscate the assessment of other characteristics present in code. In this work, we investigated if code, deemed correct by an autograder, were developed with characteristics that indicated potential misunderstandings of the concepts taught in CS1. These characteristics were named Misconceptions in Correct Code (MC³). By analyzing 2,441 codes developed by CS1 students, we curated an initial list of 45 MC³. This list was assessed by CS1 instructors, resulting in the identification of MC³ that should be addressed in classes. We selected the 15 most severe MC³ for further investigation, including a semi-structured observation in a CS1 course and an automated detection software using static code analysis. The results suggested that students develop these MC³ either due to an incomplete comprehension of the concepts taught in CS1 course or a lack of attention while elaborating their code, with correctness being their primary goal. We believe our results can contribute to: (1) the research field of misconceptions in CS1; (2) promoting alternative approaches to complement the use of autograders in CS1 classes; and (3) providing insights that can serve as the foundation for teaching interventions involving MC³ in CS1. [ABSTRACT FROM AUTHOR]

The design and administration of methodological interventions is a possible way to address dropout and failure rates in undergraduate introductory programming courses (CS1). However, to implement such strategies, it is required to identify how CS1 courses are organized and offered to students. In this work, we analyzed 225 syllabi from CS1 courses distributed in 95 Brazilian public universities. We collected these syllabi from Brazilian undergraduate programs with focus on Computer Science aspects. We report context information regarding the most covered topics, most common CS1 course names, when undergraduates take the course, total class hours, and the programming paradigms and languages taught. The results indicate that the Brazilian scenario has its own characteristics that differs from those presented in related work conducted in other countries or world regions. In Brazil, we identified 90% of the analyzed CS1 courses teaches the procedural paradigm, with C as the most common programming language (53% of the total). These results differ from those conducted in other countries where Java and object-oriented are the most common languages and paradigm taught in CS1 courses. We believe that our results can be used to: (1) provide an update to those interested in the Brazilian scenario of CS1 courses; (2) support future interventions in teaching and learning of CS1; and (3) support the Brazilian community in the development of future CS1 syllabi. [ABSTRACT FROM AUTHOR]

Teaching programming is a challenging activity nowadays, especially in introductory programming courses, which are typically massively attended. Writing functional programs is a cognitive skill, which many students, novices in programming, find it difficult to master. It is equally challenging to assess this ability of students. Research has shown that students need to learn how to read and understand, before they can learn to write programs. Code tracing questions are a suitable way to assess the knowledge of semantics of programming constructs (understanding what programs do). However, when designing these types of questions, teachers need to be aware of the complexity of the code and must always try to create questions with code snippets of same or similar complexity, especially when preparing different versions of the same test (for large student groups). In this paper we propose a new model for smart automatic generation of code tracing questions. The model uses a suitable source code metric to measure the code complexity, and to enable generation of questions containing code snippets of consistent complexity. We also introduce CodeCPP: a newly developed Moodle plugin, which is an implementation of the proposed model. The evaluation of CodeCPP, through a year‐long use of the system in teaching programming as well as feedback from selected teachers with many years of experience in teaching programming courses, shows positive and promising results. The system utilization confirms that the application of the proposed model enables a fair and objective knowledge assessment. Furthermore, the feedback provided by teachers and students is highly positive. [ABSTRACT FROM AUTHOR]

Control-flow statements often cause misunderstandings among novice computer science students. To better address these problems, teachers need to know the misconceptions that are typical at this stage. In this paper, we present the results of studying students’ misconceptions about control-flow statements. We compiled 181 questions, each containing an algorithm written in pseudocode and the execution trace of that algorithm. Some of the traces were correct; others contained highlighted errors. The students were asked to explain in their own words why the selected line of the trace was correct or erroneous. We collected and processed 10,799 answers from 67 CS1 students. Among the 24 misconceptions we found, 6 coincided with misconceptions from other studies, and 7 were narrower cases of known misconceptions. We did not find previous research regarding 11 of the misconceptions we identified.

Students struggle to get good grades in Computer Programming courses. One of the most important ways to improve the pass rate of a difficult subject, such as programming, is to implement a learning support system based on Artificial Intelligence (AI) technology. This system can provide students with a customised learning experience. This paper proposed a Revision-Bot (abbreviated as called RevBot), an intelligent interactive system that assists students in practicing past exam questions in a Python programming course. RevBot was created and implemented using the Snatchbot Chatbot API. An assessment was conducted to establish the usefulness of RevBot, and the results show that RevBot can help students improve their performance in the Python programming course. [ABSTRACT FROM AUTHOR]