Search

Providing timely feedback is crucial in promoting academic achievement and student success. However, for multifarious reasons (e.g., limited teaching resources), feedback often arrives too late for learners to act on the feedback and improve learning. Thus, automated feedback systems have emerged to tackle educational tasks in various domains, including novice programming, short-essay writing, and open-ended questions. However, to the best of our knowledge, no previous study has investigated automated feedback generation on students' project reports. In this paper, we present a novel data-driven system, named "Insta-Reviewer," for automatically generating instant feedback on students' project reports, using state-of-the-art natural language processing (NLP) models. We also propose a framework for manually evaluating system-generated feedback. Experimental results show that feedback generated by Insta-Reviewer on real students' project reports can achieve near-human performance. Our work demonstrates the feasibility of automatic feedback generation for students' project reports while highlighting several prominent challenges for future research. [For the full proceedings, see ED623995.]

An online peer-assessment system typically allows students to give textual feedback to their peers, with the goal of helping the peers improve their work. The amount of help that students receive is highly dependent on the quality of the reviews. Previous studies have investigated using machine learning to detect characteristics of reviews (e.g., Do they mention a problem, make a suggestion, or tell the student where to make a change?). Machine-learning approaches to peer-assessment evaluation are heavily reliant on labeled data to learn how to identify review characteristics. However, attaining reliable labels for those characteristics is always time-consuming and labor-intensive. In this study, we propose to apply pseudo-labeling, a semi-supervised learning-based strategy, to improve the recognition of reviews that detect problems in the reviewed work. This is done by utilizing a small, reliably labeled dataset along with a large unlabeled dataset to train a text classifier. The ultimate goal of this research is to show that for peer assessment evaluation, we can utilize both unlabeled and labeled datasets to obtain a robust auto-labeling system and thereby save much effort in labeling the data. [For the full proceedings, see ED623995.]

Peer assessment can be a more effective pedagogical method when reviewers provide quality feedback. But what makes feedback helpful to reviewees? Other studies have identified quality feedback as focusing on detecting problems, providing suggestions, or pointing out where changes need to be made. However, it is important to seek students' perspectives on what makes a review helpful to a reviewee. This study explores the helpfulness of feedback from students' perspectives when the feedback contained suggestions or mentioned problems or both. We applied natural language processing techniques to identify suggestions and problems mentioned in peer reviews. We also analyzed important text features that are associated with suggestions or problems detected by the peer feedback. The result showed that students are likely to find a review helpful if a suggestion is provided along with the problem mentioned in the feedback rather than simply identifying the problem. [For the full proceedings, see ED623995.]

Peer assessment adds value when students provide "helpful" feedback to their peers. But, this begs the question of how we determine "helpfulness." One important aspect is whether the review detects problems in the submitted work. To recognize problem detection, researchers have employed NLP and machine-learning text classification methods. Past studies have used datasets that were narrowly focused on a small number of classes in specific academic fields. This paper reports on how well models trained on one dataset or field perform on data from classes that are unlike the classes whose data they have been trained on. Specifically we took a model developed with data from a computer science class with several programming assignments, and tried to transfer it onto an education class focused more on writing research papers. We have attempted to perform such a task on a few models including logistic regression classifier, random forest classifier, multinomial naive bayes classifier and support vector machine. We made several attempts to raise the accuracy of classification, including lemmatizing to deduct variation in data input, and active learning strategies. [For the full proceedings, see ED607784.]

Imagine a student using an intelligent tutoring system. A researcher records the correctness and time of each of your attempts at solving a math problem, nothing more. With no names, no birth dates, no connections to the school, you would think it impossible to track the answers back to the class. Yet, class sections have been identified with no more data than this. This paper recounts shocking episodes where educational data was used to re-identify individual students, build profiles on students, and commit fraud. We look at the ethical principles that underlie privacy as it relates to research data, and discuss ethical issues in data mining relating to social networks and big data. We explore four major types of data used in EDM [educational data mining]: (i) clickstream data, (ii) student-interaction data, (iii) evaluative data, and (iv) demographic data. Each type of data can be harmful if disclosed in particular contexts, even if all personally identifiable information is removed. We consider laws and legal precedents controlling access to student data in the United States and the European Union. This paper concludes by describing some practical situations in EDM and suggesting privacy policies that satisfy the ethical concerns raised earlier in the paper. [For the full proceedings, see ED607784.]

Peer assessment has proven to be a useful strategy for increasing the timeliness and quantity of formative feedback, as well as for promoting metacognitive thinking among students. Previous research has determined that reviews that contain suggestions can motivate students to revise and improve their work. This paper describes a method for automatically detecting suggestions in review text. The quantity of suggestions can be treated as a metric for the helpfulness of review text. Even before a review is submitted, the system can tell a reviewer when a review is lacking in suggestions and consequently advise that they be added. This paper presents several neural-network approaches for detecting suggestions and compares them against traditional natural language processing (NLP) methods such as rule-based techniques, as well as past machine-learning approaches. Our network-based classifiers outperformed rule-based classifiers in every experiment. Our neural-network classifiers attained F1- scores in the low 90% range, outperforming the support vector machine (SVM) classifier whose F1-score was 88%. The naïve Bayes (NB) classifier had an F1-score of 84% and the rule-based classifier had an F1-score of 80%. As in other domains such as determining sentiment, we found that neural-network models perform better than the likes of naïve Bayes and support vector machines when classifying suggestions in text. [For the full proceedings, see ED599096.]

In recent years, two trends have combined to cause engineering faculty to look for better ways to assign homework. The first is the Science of Teaching and Learning (SoTL) and its emphasis on the role of metacognition in learning. The second is the growth of online answer sites like Chegg and CourseHero, which allow students to get answers to homework without actually doing the problems. Over the past decade, several approaches have been devised to deal with the fallout. They typically have students submit homework twice: the first time to provide their answers to problems, and the second time to engage in some reflective activity comparing their approach or their answers with solutions provided by the instructor. This study identifies 14 such approaches, looks at what they have in common and how they differ, and summarizes their research findings. [ABSTRACT FROM AUTHOR]