Search

Bayesian Knowledge Tracing, a model used for cognitive mastery estimation, has been a hallmark of adaptive learning research and an integral component of deployed intelligent tutoring systems (ITS). In this paper, we provide a brief history of knowledge tracing model research and introduce pyBKT, an accessible and computationally efficient library of model extensions from the literature. The library provides data generation, fitting, prediction, and cross-validation routines, as well as a simple to use data helper interface to ingest typical tutor log dataset formats. We evaluate the runtime with various dataset sizes and compare to past implementations. Additionally, we conduct sanity checks of the model using experiments with simulated data to evaluate the accuracy of its EM parameter learning and use real-world data to validate its predictions, comparing pyBKT's supported model variants with results from the papers in which they were originally introduced. The library is open source and open license for the purpose of making knowledge tracing more accessible to communities of research and practice and to facilitate progress in the field through easier replication of past approaches. [For the full proceedings, see ED615472.]

The Indian Institute of Science is proud to host the fully in-person sixteenth iteration of the International Conference on Educational Data Mining (EDM) during July 11-14, 2023. EDM is the annual flagship conference of the International Educational Data Mining Society. The theme of this year's conference is "Educational data mining for amplifying human potential." Not all students or seekers of knowledge receive the education necessary to help them realize their full potential, be it due to a lack of resources or lack of access to high quality teaching. The dearth in high-quality educational content, teaching aids, and methodologies, and non-availability of objective feedback on how they could become better teachers, deprive our teachers from achieving their full potential. The administrators and policy makers lack tools for making optimal decisions such as optimal class sizes, class composition, and course sequencing. All these handicap the nations, particularly the economically emergent ones, who recognize the centrality of education for their growth. EDM-2023 has striven to focus on concepts, principles, and techniques mined from educational data for amplifying the potential of all the stakeholders in the education system. The spotlights of EDM-2023 include: (1) Five keynote talks by outstanding researchers of eminence; (2) A plenary Test of Time award talk and a Banquet talk; (3) Five tutorials (foundational as well as advanced); (4) Four thought provoking panels on contemporary themes; (5) Peer reviewed technical paper and poster presentations; (6) Doctoral students consortium; and (7) An enchanting cultural programme. [Individual papers are indexed in ERIC.]

With the increased number of MOOC offerings, it is unclear how these courses are related. Previous work has focused on capturing the prerequisite relationships between courses, lectures, and concepts. However, it is also essential to model the content structure of MOOC courses. Constructing a precedence graph that models the similarities and variations of learning paths followed by similar MOOCs would help both students and instructors. Students can personalize their learning by choosing the desired learning path and lectures across several courses guided by the precedence graph. Similarly, by examining the precedence graph, instructors can 1) identify knowledge gaps in their MOOC offerings, and 2) find alternative course plans. In this paper, we propose an unsupervised approach to build the precedence graph of similar MOOCs, where nodes are clusters of lectures with similar content, and edges depict alternative precedence relationships. Our approach to cluster similar lectures based on PCK-Means clustering algorithm that incorporates pairwise constraints: Must-Link and Cannot-Link with the standard K-Means algorithm. To build the precedence graph, we link the clusters according to the precedence relations mined from current MOOCs. Experiments over real-world MOOC data show that PCK-Means with our proposed pairwise constraints outperform the K-Means algorithm in both Adjusted Mutual Information (AMI) and Fowlkes-Mallows scores (FMI). [For the full proceedings, see ED607784.]

One of the essential problems, in educational data mining, is to predict students' performance on future learning materials, such as problems, assignments, and quizzes. Pioneer algorithms for predicting student performance mostly rely on two sources of information: students' past performance, and learning materials' domain knowledge model. The domain knowledge model, traditionally curated by domain experts, maps learning materials to concepts, topics, or knowledge components that are presented in them. However, creating a domain model by manually labeling the learning material can be a difficult and time-consuming task. In this paper, we propose a tensor factorization model for student performance prediction that does not rely on a predefined domain model. Our proposed algorithm models student knowledge as a soft membership of latent concepts. It also represents the knowledge acquisition process with an added rank-based constraint in the tensor factorization objective function. Our experiments show that the proposed model outperforms state-of-the-art algorithms in predicting student performance in two real-world datasets, and is robust to hyper-parameters. [For the full proceedings, see ED599096.]

One significant challenge in the field of measuring ability is measuring the current ability of a learner while they are learning. Many forms of inference become computationally complex in the presence of time-dependent learner ability, and are not feasible to implement in an online context. In this paper, we demonstrate an approach which can estimate learner skill over time even in the presence of large data sets. We use a rating system derived from the Elo rating system and its relatives, which are commonly used in chess and sports tournaments. A learner's submission of a course assignment is interpreted as a single match. We apply this approach to Coursera's online learning platform, which includes millions of learners who have submitted assignments tens of millions of times in over 3000 courses. We demonstrate that this provides reliable estimates of item difficulty and learner ability. Finally, we address how this scoring framework may be used as a basis for various applications that account for a learner's ability, such as adaptive diagnostic tests and personalized recommendations. [For the full proceedings, see ED599096.]

In this paper we consider the problem of modelling when students end their session in an online mathematics educational system. Being able to model this accurately will help us optimize the way content is presented and consumed. This is done by modelling the probability of an action being the last in a session, which we denote as the End-of-Session probability. We use log data from a system where students can learn mathematics through various kinds of learning materials, as well as multiple types of exercises, such that a student session can consist of many different activities. We model the End-of-Session probability by a deep recurrent neural network in order to utilize the long term temporal aspect, which we experimentally show is central for this task. Using a large scale dataset of more than 70 million student actions, we obtain an AUC of 0.81 on an unseen collection of students. Through a detailed error analysis, we observe that our model is robust across different session structures and across varying session lengths. [For the full proceedings, see ED599096.]

Clustering of educational data allows similar students to be grouped, in either crisp or fuzzy sets, based on their similarities. Standard approaches are well suited to identifying common student behaviors; however, by design, they put much less emphasis on less common behaviors or outliers. The approach presented in this paper employs fuzzing clustering in the identification of these outlier behaviors. The algorithm is an iterative one, where clustering is applied, outliers identified, the data restricted to the outliers, and the process repeated. This approach produces a clustering that is crisp between each iteration and fuzzy within. It arose as a consequence of trying to cluster student progress trajectories in an adaptive learning platform. Included are results from applying the repeated fuzzy clustering algorithm to data from multiple courses and semesters at the University of Central Florida, (N=5,044). [For the full proceedings, see ED599096.]

Personalized education systems recommend learning contents to students based on their capacity to accelerate their learning. This paper proposes a personalized exercise recommendation system for online self-directed learning. We first improve the performance of knowledge tracing models. Existing deep knowledge tracing models, such as Dynamic Key-Value Memory Network (DKVMN), ignore exercises' concept tags, which are usually available in tutoring systems. We modify DKVMN to design its memory structure based on the course's concept list, and explicitly consider the exercise-concept mapping relationship during students' knowledge tracing. We evaluated the model on the 5th grade students' math exercising dataset in TAL, one of the biggest education groups in China, and found that our model has higher performance than existing models. We also enhance the DKVMN model to support more input features and obtain higher performance. Second, we use the model to build a student simulator, and use it to train an exercise recommendation policy with deep reinforcement learning. Experimental results show that our policy achieves better performance than existing heuristic policy in terms of maximizing the students' knowledge level. To the best of our knowledge, this is the first time that deep reinforcement learning has been applied to personalized mathematic exercise recommendation. [For the full proceedings, see ED599096.]

Short forms of psychometric scales have been commonly used in educational and psychological research to reduce the burden of test administration. However, it is challenging to select items for a short form that preserve the validity and reliability of the scores of the original scale. This paper presents and evaluates multiple automated methods for scale short form creation based on metaheuristic optimization algorithms that incorporate validity criteria based on internal structure and relationships with other variables. The ant colony optimization (ACO) algorithm, tabu search (TS), simulated annealing (SA) and genetic algorithm (GA) are examined using confirmatory factor analysis (CFA) of scales with one factor, three factor, and bi-factor factorial structure. The results indicate that SA created short forms with best model fit for scales with one and three factor structures, but ACO was able to obtain highest reliability. For scales with bi-factor structure, SA provide short forms with best model fit, but TS obtained highest reliability. Overall, the SA algorithm is recommended because it produced consistently best model fit and reliability that was only slightly lower than the ACO or TS algorithms. [For the full proceedings, see ED599096.]

This paper describes a multiple criteria approach based on a hybrid method of Collaborative Filtering (CF) and ContentBased Filtering (CBF) for discovering the most relevant criteria which could affect the elective course recommendation for university students. In order to determine which factors are the most important, it is proposed a genetic algorithm which automatically discovers the importance of the different criteria assigning weights to each one of them. We have carried out an in-depth study using a real data set with more than 1700 ratings of Computer Science graduates at University of Cordoba. We have used different proposals and different weights for each criterion in order to discover what is the combination of multiple criteria which provides better results. [For the full proceedings, see ED593090.]

Over the past decade, machine learning has become an integral part of educational technologies. With more and more applications such as students' performance prediction, course recommendation, dropout prediction and knowledge tracing relying upon machine learning models, there is increasing evidence and concerns about bias and unfairness of these models. Unfair models can lead to inequitable outcomes for some groups of students and negatively impact their learning. We show by real-world examples that educational data has embedded bias that leads to biased student modeling, which urges the development of fairness formalizations and fair algorithms for educational applications. Several formalizations of fairness have been proposed that can be classified into two types: (i) group fairness and (ii) individual fairness. Group fairness guarantees that groups are treated fairly as a whole, which might not be fair to some individuals. Thus individual fairness has been proposed to make sure fairness is achieved on individual level. In this work, we focus on developing an individually fair model for identifying students at-risk of underperforming. We propose a model which is based on the idea that the prediction for a student (identifying at-risk students) should not be influenced by his/her sensitive attributes. The proposed model is shown to effectively remove bias from these predictions and hence, making them useful in aiding all students. [For the full proceedings, see ED607784.]

Digital educational technologies offer the potential to customize students' experiences and learn what works for which students, enhancing the technology as more students interact with it. We consider whether and when attempting to discover how to personalize has a cost, such as if the adaptation to personal information can delay the adoption of policies that benefit all students. We explore these issues in the context of using multi-armed bandit (MAB) algorithms to learn a policy for what version of an educational technology to present to each student, varying the relation between student characteristics and outcomes and also whether the algorithm is aware of these characteristics. Through simulations, we demonstrate that the inclusion of student characteristics for personalization can be beneficial when those characteristics are needed to learn the optimal action. In other scenarios, this inclusion decreases performance and increases variation in student experiences. Moreover, including unneeded student characteristics can systematically disadvantage students with less common values for these characteristics. Our simulations do however suggest that real-time personalization will be helpful in particular real-world scenarios, and we illustrate this through case studies using existing experimental results in ASSISTments. Overall, our simulations show that adaptive personalization in educational technologies can be a double-edged sword: real-time adaptation improves student experiences in some contexts, but the slower adaptation and increased variability mean that a more personalized model is not always beneficial. [For the full proceedings, see ED607784.]

Selecting courses in an open-curriculum education program is a difficult task for students and academic advisors. Course recommendation systems nowadays can be used to reduce the complexity of this task. To control the recommendation error, we argue that course recommendations need to be provided together with "statistical" confidence. The latter can be used for computing a statistically valid set of recommended courses that contains courses a student is likely to take with a probability of at least 1-[epsilon] for a user-specified significance level [epilsilon]. For that purpose, we introduce a generic algorithm for course recommendation based on the conformal prediction framework. The algorithm is used for implementing two conformal course recommender systems. Through experimentation, we show that these systems accurately suggest courses to students while maintaining statistically valid sets of courses recommended. [For the full proceedings, see ED607784.]

Recommending courses to students is a fundamental and also challenging issue in the traditional university environment. Not exactly like course recommendation in MOOCs, the selection and recommendation for higher education is a non-trivial task as it depends on many factors that students need to consider. Although many studies on this topic have been proposed, most of them only focus either on historical course enrollment data or on models of predicting course outcomes to give recommendation results, regardless of multiple reasons behind course selection behavior. To address such a challenge, we first conduct a survey to show the underlying characteristic of the course selection of university students. According to the survey results, we propose a hybrid course recommendation framework based on multiple features. Our experimental result illustrates that our method outperforms other approaches. Also, our framework is easier to interpret, scrutinize, and explain than conventional black-box methods for course recommendation. [For the full proceedings, see ED607784.]

The analysis of log data generated by online educational systems is an important task for improving the systems, and furthering our knowledge of how students learn. This paper uses previously unseen log data from Edulab, the largest provider of digital learning for mathematics in Denmark, to analyse the sessions of its users, where 1.08 million student sessions are extracted from a subset of their data. We propose to model students as a distribution of different underlying student behaviours, where the sequence of actions from each session belongs to an underlying student behaviour. We model student behaviour as Markov chains, such that a student is modelled as a distribution of Markov chains, which are estimated using a modified k-means clustering algorithm. The resulting Markov chains are readily interpretable, and in a qualitative analysis around 125,000 student sessions are identified as exhibiting unproductive student behaviour. Based on our results this student representation is promising, especially for educational systems offering many different learning usages, and offers an alternative to common approaches like modelling student behaviour as a single Markov chain often done in the literature. [For the full proceedings, see ED596512.]

In this paper we present a novel, data-driven algorithm for generating feedback for students on open-ended programming problems. The feedback goes beyond next-step hints, annotating a student's whole program with suggested edits, including code that should be moved or reordered. We also build on existing work to design a methodology for evaluating this feedback in comparison to human tutor feedback, using a dataset of real student help requests. Our results suggest that our algorithm is capable of reproducing ideal human tutor edits almost as frequently as another human tutor. However, our algorithm also suggests many edits that are not supported by human tutors, indicating the need for better feedback selection. [For the full proceedings, see ED596512.]

The role of affective states in learning has recently attracted considerable attention in education research. The accurate prediction of affective states can help increase the learning gain by incorporating targeted interventions that are capable of adjusting to changes in the individual affective states of students. Until recently, most work on the prediction of affective states has relied on expensive and stationary lab devices that are not well suited for classrooms and everyday use. Here, we present an automated pipeline capable of accurately predicting (AUC up to 0.86) the affective states of participants solving tablet-based math tasks using signals from low-cost mobile bio-sensors. In addition, we show that we can achieve a similar classification performance (AUC up to 0.84) by only using handwriting data recorded from a stylus while students solved the math tasks. Given the emerging digitization of classrooms and increased reliance on tablets as teaching tools, stylus data may be a viable alternative to bio-sensors for the prediction of affective states. [For the full proceedings, see ED599096.]

Intelligent Tutoring Systems (ITSs) have shown success in the domain of programming, in part by providing customized hints and feedback to students. However, many popular novice programming environments still lack these intelligent features. This is due in part to their use of open-ended programming assignments, which are difficult to support with existing hint generation techniques. In this paper, we present a new data-driven algorithm, based on the Hint Factory, to generate hints for these open-ended assignments. We evaluate our algorithm on historical student data and show that it can provide hints that successfully lead students to solutions from any state, help students achieve assignment objectives, and align with the student's future solution. [For the full proceedings, see ED592609.]

Whether teaching in a classroom or a Massive Online Open Course it is crucial to present the material in a way that benefits the audience as a whole. We identify two important tasks to solve towards this objective; (1) group students so that they can maximally benefit from peer interaction and (2) find an optimal schedule of the educational material for each group. Thus, in this paper we solve the problem of team formation and content scheduling for education. Given a time frame "d," a set of students S with their required need to learn different activities T and given "k" as the number of desired groups, we study the problem of finding "k" group of students. The goal is to teach students within time frame "d" such that their potential for learning is maximized and find the best schedule for each group. We show this problem to be NP-hard and develop a polynomial algorithm for it. We show our algorithm to be effective both on synthetic as well as a real data set. For our experiments we use real data on students' grades in a Computer Science department. As part of our contribution we release a semi-synthetic dataset that mimics the properties of the real data. [For complete proceedings, see ED560503.]

In this paper we consider discrimination-aware classification of educational data. Mining and using rules that distinguish groups of students based on sensitive attributes such as gender and nationality may lead to discrimination. It is desirable to keep the sensitive attributes during the training of a classifier to avoid information loss but decrease the undesirable correlation between the sensitive attributes and the class attribute when building the classifier. We illustrate, motivate, and solve the problem, and present a case study for predicting student exam performance based on enrollment information and assessment results during the semester. We evaluate the performance of two discrimination-aware classifiers and compare them with their non-discrimination-aware counterparts. The results show that the discrimination-aware classifiers are able to reduce discrimination with trivial loss in accuracy. The proposed method can help teachers to predict student performance accurately without discrimination. [For complete proceedings, see ED560503.]

Predicting the success or failure of a student in a course or program is a problem that has recently been addressed using data mining techniques. In this paper we evaluate some of the most popular classification and regression algorithms on this problem. We address two problems: prediction of approval/failure and prediction of grade. The former is tackled as a classification task while the latter as a regression task. Separate models are trained for each course. The experiments were carried out using administrate data from the University of Porto, concerning approximately 700 courses. The algorithms with best results overall in classification were decision trees and SVM while in regression they were SVM, Random Forest, and AdaBoost.R2. However, in the classification setting, the algorithms are finding useful patterns, while, in regression, the models obtained are not able to beat a simple baseline. [This work was partially funded by projects financed by the North Portugal Regional Operational Programme (ON.2--O Novo Norte), under the National Strategic Reference Framework (NSRF), through the European Regional Development Fund (ERDF).] [For complete proceedings, see ED560503.]

Student modeling can help guide the behavior of a cognitive tutor system and provide insight to researchers on understanding how students learn. In this context, Bayesian Knowledge Tracing (BKT) is one of the most popular knowledge inference models due to its predictive accuracy, interpretability and ability to infer student knowledge. However, the most popular methods for training the parameters of BKT have some problems, such as identifiability, local minima, degenerate parameters and computational cost during fitting. In this paper we address some of the issues of one of these training models, BKT Brute Force. Instead of finding the parameter values that provide the lowest Residual Sum of Squares (RSS), we estimate this minimum RSS value from some a priori known values of the skill. From there we perform some preliminary analysis to improve our knowledge of the relationship between the RSS, from BKT-BF, and the four BKT parameters. [For complete proceedings, see ED560503.]

Expertise in a domain of knowledge is characterized by a greater fluency for solving problems within that domain and a greater facility for transferring the structure of that knowledge to other domains. Deliberate practice and the feedback that takes place during practice activities serve as gateways for developing domain expertise. However, there is a difficulty in consistently aligning feedback about a learner's practice performance with the intended learning outcomes of those activities -- especially in situations where the person providing feedback is unfamiliar with the intention of those activities. To address this problem, we propose an intelligent model to automatically label opportunities for practice (assessment questions) according to the learning outcomes intended by the course designers. As a proof of concept, we used a reduced version of Bloom's Taxonomy to define the intended learning outcomes. Using a factorial design, we employed term frequency-inverse document frequency (TF-IDF) and latent Dirichlet allocation (LDA) to transform questions from text to word weightages with support vector machine (SVM) and extreme learning machine (ELM) to train and automatically label the questions. We trained our models with 120 questions labeled by the subject matter expert of an undergraduate engineering course. Compared to existing works which create models based on a selfgenerated dataset, our proposed approach uses 30 untrained questions from online/textbook sources to validate the performance of our models. Exhaustive comparison analysis of the testing set showed that TF-IDF with ELM outperformed the other combinations by yielding 0.86 reliability (F1 measure) with the subject matter expert. [For the full proceedings, see ED596512.]

Since MOOC is suffering high dropout rate, researchers try to explore the reasons and mitigate it. Focusing on this task, we employ a composite model to infer behaviors of learners in the coming weeks based on his/her history log of learning activities, including interaction with video lectures, participation in discussion forum, and performance of assignments, etc. The prediction accuracy of our proposed model outperforms related methods. Besides, we try combining the model with suggested interventions, such as sending reminder emails to at-risk learners. Future work, which is currently underway, will evaluate its influence on mitigating dropout rate. [For the full proceedings, see ED596512.]

Augmented Graph Grammars are a graph-based rule formalism that supports rich relational structures. They can be used to represent complex social networks, chemical structures, and student-produced argument diagrams for automated analysis or grading. In prior work we have shown that Evolutionary Computation (EC) can be applied to induce empirically-valid grammars for student-produced argument diagrams based upon fitness selection. However this research has shown that while the traditional EC algorithm does converge to an optimal fitness, premature convergence can lead to it getting stuck in local maxima, which may lead to undiscovered rules. In this work, we augmented the standard EC algorithm to induce more heterogeneous Augmented Graph Grammars by replacing the fitness selection with a novelty-based selection mechanism every ten generations. Our results show that this novelty selection increases the diversity of the population and produces better, and more heterogeneous, grammars. [For the full proceedings, see ED596512.]

This study examined how machine learning and natural language processing (NLP) techniques can be leveraged to assess the interpretive behavior that is required for successful literary text comprehension. We compared the accuracy of seven different machine learning classification algorithms in predicting human ratings of student essays about literary works. Three types of NLP feature sets: unigrams (single content words), elaborative (new) n-grams, and linguistic features were used to classify idea units (paraphrase, text-based inference, interpretive inference). The most accurate classifications emerged using all three NLP features sets in combination, with accuracy ranging from 0.61 to 0.94 (F=0.18 to 0.81). Random Forests, which employs multiple decision trees and a bagging approach, was the most accurate classifier for these data. In contrast, the single classifier, Trees, which tends to "overfit" the data during training, was the least accurate. Ensemble classifiers were generally more accurate than single classifiers. However, Support Vector Machines accuracy was comparable to that of the ensemble classifiers. This is likely due to Support Vector Machines' unique ability to support high dimension feature spaces. The findings suggest that combining the power of NLP and machine learning is an effective means of automating literary text comprehension assessment. [For the full proceedings, see ED596512. For the corresponding grantee submission, see ED577127.]

The 10th International Conference on Educational Data Mining (EDM 2017) is held under the auspices of the International Educational Data Mining Society at the Optics Velley Kingdom Plaza Hotel, Wuhan, Hubei Province, in China. This years conference features two invited talks by: Dr. Jie Tang, Associate Professor with the Department of Computer Science and Technology at Tsinghua University; and Dr. Ron Cole, President of Boulder Learning Inc. The main conference invited contributions to the Research Track and Industry Track. 122 submissions were received (71 full, 47 short, 4 industry). 18 full papers papers were accepted (25% acceptance rate) and 32 short papers for oral presentation (42% acceptance rate) and an additional 39 for poster presentations, 3 demonstrations. The industry track includes all 4 submitted industry papers and 1 paper initially submitted as a full paper. The EDM conference provides opportunities for young researchers, and particularly Ph.D. students, to present their research ideas and receive feedback from the peers and more senior researchers. This year, the Doctoral Consortium features 6 such presentations. In addition to the main program, the conference includes 3 workshops: (1) Graph-based Educational Data Mining (G-EDM 2017); (2) Sharing and Reusing Data & Analytics Methods with LearnSphere; and (3) Deep Learning with Educational Data; and 2 tutorials: (1) Why Data Standards are Critical for EDM and AIED; and (2) Principal Stratification for EDM Experiments. [For the 2016 proceedings, see ED592609.]

We present an analysis of log data from a semester's use of the OpenDSA eTextbook system with the goal of determining the most difficult course topics in a data structures course. While experienced instructors can identify which topics students most struggle with, this often comes only after much time and effort, and does not provide real-time analysis that might benefit an intelligent tutoring system. Our factors included the fraction of wrong answers given by student, results from Item Response Theory, and the rate of model answer and hint use by students. We grouped exercises by topic covered to yield a list of topics associated with the harder exercises. We found that a majority of these exercises were related to algorithm analysis topics. We compared our results to responses given by a sample of experienced instructors, and found that the automated results match the expert opinions reasonably well. We investigated reasons that might explain the over-representation of algorithm analysis among the difficult topics, and hypothesize that visualizations might help to better present this material. [For the full proceedings, see ED592609.]

In recent years, substantial improvements were obtained in the effectiveness of data driven algorithms to validate the mapping of items to skills, or the Q-matrix. In the current study we use ensemble algorithms on top of existing Q-matrix refinement algorithms to improve their performance. We combine the boosting technique with a decision tree. The results show that the improvements from both the decision tree and Adaboost combined are better than the decision tree alone and yield substantial gains over the best performance of individual Q-matrix refinement algorithm. [For the full proceedings, see ED592609.]

User clustering algorithms have been introduced to analyze users' learning behaviors and help to provide personalized learning guides in traditional Web-based learning systems. However, the explicit and implicit coupled interactions, which means the correlations between user attributes generated from learning actions, are not considered in these algorithms. Much significant and useful information which can positively affect clustering accuracy is neglected. To solve the above issue, we proposed a coupled user clustering algorithm for Wed-based learning systems. It respectively takes into account intra-coupled and inter-coupled relationships of learning data, and utilizes Taylor-like expansion to represent their integrated coupling correlations. The experiment result demonstrates the outperformance of the algorithm in terms of efficiently capturing correlations of learning data and improving clustering accuracy. [For the full proceedings, see ED592609.]

The 9th International Conference on Educational Data Mining (EDM 2016) is held under the auspices of the International Educational Data Mining Society at the Sheraton Raleigh Hotel, in downtown Raleigh, North Carolina, in the USA. The conference, held June 29-July 2, 2016, follows the eight previous editions (Madrid 2015, London 2014, Memphis 2013, Chania 2012, Eindhoven 2011, Pittsburgh 2010, Cordoba 2009 and Montreal 2008). The EDM conference is the leading international forum for high-quality research that leverages educational data, learning analytics, and machine learning to answer research questions that shed light on the learning processes. This year's conference features three invited talks by: Rakesh Agrawal, President and Founder of Data Insights Laboratories; Marcia C. Linn, Professor of the University of California at Berkeley; and Judy Kay, Professor of the University of Sydney. Judy Kay's invited paper entitled "Enabling people to harness and control EDM for lifelong, life-wide learning" is also presented in the proceedings. Together with the "Journal of Educational Data Mining" ("JEDM"), the EDM 2016 conference supports a "JEDM" Track that provides researchers a venue to deliver more substantial mature work than is possible in a conference proceedings and to present their work to a live audience. The papers submitted to this track followed the "JEDM" peer review process; three papers have been accepted to the track and were presented at the conference. The abstracts of the invited talks, panels and accepted "JEDM" Track papers can be found in these proceedings. [For the 2015 proceedings, see ED560503.]

As adaptive tutoring systems grow increasingly popular for the completion of classwork and homework, it is crucial to assess the manner in which students are scored within these platforms. The majority of systems, including ASSISTments, return the binary correctness of a student's first attempt at solving each problem. Yet for many teachers, partial credit is a valuable practice when common wrong answers, especially in the presence of effort, deserve acknowledgement. We present a grid search to analyze 441 partial credit models within ASSISTments in an attempt to optimize per unit penalization weights for hints and attempts. For each model, algorithmically determined partial credit scores are used to bin problem performance, using partial credit to predict binary correctness on the next question. An optimal range for penalization is discussed and limitations are considered. [For complete proceedings, see ED560503.]

Classification evaluation metrics are often used to evaluate adaptive tutoring systems-- programs that teach and adapt to humans. Unfortunately, it is not clear how intuitive these metrics are for practitioners with little machine learning background. Moreover, our experiments suggest that existing convention for evaluating tutoring systems may lead to suboptimal decisions. We propose the Learner Effort-Outcomes Paradigm (Leopard), a new framework to evaluate adaptive tutoring. We introduce Teal and White, novel automatic metrics that apply Leopard and quantify the amount of effort required to achieve a learning outcome. Our experiments suggest that our metrics are a better alternative for evaluating adaptive tutoring. [For complete proceedings, see ED560503.]

The 7th International Conference on Education Data Mining held on July 4th-7th, 2014, at the Institute of Education, London, UK is the leading international forum for high-quality research that mines large data sets in order to answer educational research questions that shed light on the learning process. These data sets may come from the traces that students leave when they interact, either individually or collaboratively, with learning management systems, interactive learning environments, intelligent tutoring systems, educational games or when they participate in a data-rich learning context. The types of data therefore range from raw log files to eyetracking devices and other sensor data. Being hosted in London, UK the theme of the conference is "Big Data--Big Ben--Education Data Mining for Big Impact in Teaching and Learning". In this seventh year of EDM conferences, it is clear that the field is continuing to grow at a rapid pace. With renewed focus on education driven by big data learning analytics has put the EDM field in the center of growing interest. Traditional educational technologies, intelligent tutoring systems, educational games, and learning management systems all continue to generate growing amounts of data that are becoming available for analysis. The new interest in MOOCs and their promise to reach thousands or even hundreds of thousands of students per class requires techniques for feedback and grading that are being researched in the EDM domain. The conference submissions this year also continue to grow. A tremendous amount of work has gone into bringing this conference together, and the following are presented: (1) The Field of EDM: Where We Came from and Where We're Going (Joseph Beck); (2) Generative Adaptivity for Optimization of the Learning Ecosystem (Zoran Popovic; (3) 150K+ Online Students at a Time: How to Understand What's Happening in Online 4 Learning (Daniel Russell); (4) Adaptive Practice of Facts in Domains with Varied Prior Knowledge (Jan Papoušek, Radek Pelánek and Vít Stanislav); (5) Alternating Recursive Method for Q-Matrix Learning (Yuan Sun, Shiwei Ye, Shunya Inoue and Yi Sun); (6) Application of Time Decay Functions and the Elo System in Student Modeling (Radek Pelánek); (7) Causal Discovery with Models: Behavior, Affect, and Learning in Cognitive Tutor Algebra (Stephen Fancsali); (8) Choice-Based Assessment: Can Choices Made in Digital Games Predict 6th-Grade Students' Math Test Scores? (Min Chi, Daniel Schwartz, Kristen Pilner Blair and Doris B. Chin); (9) Comparing Expert and Metric-Based Assessments of Association Rule Interestingness (Diego Luna Bazaldua, Ryan Baker and Maria Ofelia San Pedro); (10) Different Parameters - Same Prediction: An Analysis of Learning Curves (Tanja Käser, Kenneth Koedinger and Markus Gross); (11) Discovering Gender-Specific Knowledge from Finnish Basic Education Using PISA Scale Indices (Mirka Saarela and Tommi Kärkkäinen); (12) EduRank: A Collaborative Filtering Approach to Personalization in E-Learning (Avi Segal, Ziv Katzir, Kobi Gal, Guy Shani and Bracha Shapira); (13) Exploring Differences in Problem Solving with Data-Driven Approach Maps (Michael Eagle and Tiffany Barnes); (14) General Features in Knowledge Tracing: Applications to Model Multiple Subskills, Temporal Item Response Theory, and Expert Knowledge (José González-Brenes, Yun Huang and Peter Brusilovsky); (15) Generating Hints for Programming Problems Using Intermediate Output ( Barry Peddycord III, Andrew Hicks and Tiffany Barnes); (16) Integrating Latent-Factor and Knowledge-Tracing Models to Predict Individual Differences in Learning (Mohammad Khajah, Rowan Wing, Robert Lindsey and Michael Mozer); (17) Interpreting Model Discovery and Testing Generalization to a New Dataset (Ran Liu, Elizabeth A. McLaughlin and Kenneth R. Koedinger); (18) Learning Individual Behavior in an Educational Game: A Data-Driven Approach (Seong Jae Lee, Yun-En Liu and Zoran Popovic); (19) Predicting Learning and Affect from Multimodal Data Streams in Task-Oriented Tutorial Dialogue (Joseph Grafsgaard, Joseph Wiggins, Kristy Elizabeth Boyer, Eric Wiebe and James Lester); (20) Sentiment Analysis in MOOC Discussion Forums: What does It Tell Us? (Miaomiao Wen, Diyi Yang and Carolyn Rose); (21) The Effect of Mutual Gaze Perception on Students' Verbal Coordination (Bertrand Schneider and Roy Pea); (22) The Opportunities and Limitations of Scaling Up Sensor-Free Affect Detection (Michael Wixon, Ivon Arroyo, Kasia Muldner, Winslow Burleson, Cecil Lozano and Beverly Woolf); (23) The Problem Solving Genome: Analyzing Sequential Patterns of Student Work with Parameterized Exercises (Julio Guerra, Shaghayegh Sahebi, Peter Brusilovsky and Yu-Ru Lin); (24) Trading Off Scientific Knowledge and User Learning with Multi-Armed Bandits (Yun-En Liu, Travis Mandel, Emma Brunskill and Zoran Popovic); (25) Vertical and Stationary Scales for Progress Maps (Russell Almond, Ilya Goldin, Yuhua Guo and Nan Wang); (26) Visualization and Confirmatory Clustering of Sequence Data from a Simulation- Based Assessment Task (Yoav Bergner, Zhan Shu and Alina von Davier); (27) Who's in Control?: Categorizing Nuanced Patterns of Behaviors within a Game- Based Intelligent Tutoring System (Erica Snow, Laura Allen, Devin Russell and Danielle McNamara); (28) Acquisition of Triples of Knowledge from Lecture Notes: A Natural Language Processing Approach (Thushari Atapattu, Katrina Falkner and Nickolas Falkner); (29) Towards Assessing Students' Prior Knowledge from Tutorial Dialogues (Dan Stefanescu, Vasile Rus and Art Graesser); (30) Assigning Educational Videos at Appropriate Locations in Textbooks (Marios Kokkodis, Anitha Kannan and Krishnaram Kenthapadi) (31) Better Data Beats Big Data (Michael Yudelson, Stephen Fancsali, Steven Ritter, Susan Berman, Tristan Nixon and Ambarish Joshi); (32) Building a Student At-Risk Model: An End-to-End Perspective (Lalitha Agnihotri and Alexander Ott); (33) Can Engagement be Compared? Measuring Academic Engagement for Comparison (Ling Tan, Xiaoxun Sun and Siek Toon Khoo); (34) Comparison of Algorithms for Automatically Building Example-Tracing Tutor Models (Rohit Kumar, Matthew Roy, Bruce Roberts and John Makhoul); (35) Computer-Based Adaptive Speed Tests (Daniel Bengs and Ulf Brefeld); (36) Discovering Students' Complex Problem Solving Strategies in Educational Assessment (Krisztina Tóth, Heiko Rölke, Samuel Greiff and Sascha Wüstenberg); (37) Discovering Theoretically Grounded Predictors of Shallow vs. Deep-Level Learning (Carol Forsyth, Arthur Graesser, Philip I. Pavlik Jr., Keith Millis and Borhan Samei); (38) Domain Independent Assessment of Dialogic Properties of Classroom Discourse (Borhan Samei, Andrew Olney, Sean Kelly, Martin Nystrand, Sidney D'Mello, Nathan Blanchard, Xiaoyi Sun, Marci Glaus and Art Graesser); (39) Empirically Valid Rules for Ill-Defined Domains (Collin Lynch and Kevin Ashley); (40) Entropy: A Stealth Measure of Agency in Learning Environments (Erica Snow, Matthew Jacovina, Laura Allen, Jianmin Dai and Danielle McNamara); (41) Error Analysis as a Validation of Learning Progressions (Brent Morgan, William Baggett and Vasile Rus); (42) Exploration of Student's Use of Rule Application References in a Propositional Logic Tutor (Michael Eagle, Vinaya Polamreddi, Behrooz Mostafavi and Tiffany Barnes); (43) Exploring Real-Time Student Models Based on Natural-Language Tutoring Sessions (Benjamin Nye, Mustafa Hajeer, Carolyn Forsyth, Borhan Samei, Xiangen Hu and Keith Millis); (44) Forum Thread Recommendation for Massive Open Online Courses (Diyi Yang, Mario Piergallini, Iris Howley and Carolyn Rose); (45) Investigating Automated Student Modeling in a Java MOOC (Michael Yudelson, Roya Hosseini, Arto Vihavainen and Peter Brusilovsky); (46) Mining Gap-Fill Questions from Tutorial Dialogues (Nobal B. Niraula, Vasile Rus, Dan Stefanescu and Arthur C. Graesser); (47) Online Optimization of Teaching Sequences with Multi-Armed Bandits (Benjamin Clement, Pierre-Yves Oudeyer, Didier Roy and Manuel Lopes); (48) Predicting MOOC Performance with Week 1 Behavior (Suhang Jiang, Adrienne Williams, Katerina Schenke, Mark Warschauer and Diane O'Dowd); (49) Predicting STEM and Non-STEM College Major Enrollment from Middle School Interaction with Mathematics Educational Software (Maria Ofelia San Pedro, Jaclyn Ocumpaugh, Ryan Baker and Neil Heffernan); (50) Quantized Matrix Completion for Personalized Learning (Andrew Lan, Christoph Studer and Richard Baraniuk); (51) Reengineering the Feature Distillation Process: A Case Study in Detection of Gaming the System (Luc Paquette, Adriana de Carvahlo, Ryan Baker and Jaclyn Ocumpaugh); (52) SKETCHMINER: Mining Learner-Generated Science Drawings with Topological Abstraction (Andy Smith, Eric N. Wiebe, Bradford W. Mott and James C. Lester); (53) Teachers and Students Learn Cyber Security: Comparing Software Quality, Security (Shlomi Boutnaru and Arnon Hershkovitz); (54) Testing the Multimedia Principle in the Real World: A Comparison of Video vs. Text Feedback in Authentic Middle School Math Assignments (Korinn Ostrow and Neil Heffernan); (55) The Importance of Grammar and Mechanics in Writing Assessment and Instruction: Evidence from Data Mining (Scott Crossley, Kris Kyle, Laura Allen and Danielle McNamara); (56) The Long and Winding Road: Investigating the Differential Writing Patterns of High and Low Skilled Writers (Laura Allen, Erica Snow and Danielle McNamara); (57) The Refinement of a Q-Matrix: Assessing Methods to Validate Tasks to Skills Mapping (Michel Desmarais, Behzad Beheshti and Peng Xu); (58) Tracing Knowledge and Engagement in Parallel in an Intelligent Tutoring System (Sarah Schultz and Ivon Arroyo); (59) Tracking Choices: Computational Analysis of Learning Trajectories (Erica Snow, Laura Allen, G.Tanner Jackson and Danielle McNamara); (60) Unraveling Students' Interaction Around a Tangible Interface Using Gesture Recognition (Bertrand Schneider and Paulo Blikstein); (61) A Predictive Model for Video Lectures Classification (Priscylla Silva, Roberth Pinheiro and Evandro Costa); (62) Accepting or Rejecting Students' Self-Grading in their Final Marks by using Data Mining (Javier Fuentes, Cristobal Romero, Carlos García-Martínez and Sebastián Ventura); (63) Analysis and extraction of behaviors by students in lectures 329 Eiji Watanabe, Takashi Ozeki and Takeshi Kohama (64) Analysis of Student Retention and Drop-Out Using Visual Analytics (Jan Géryk and Lubomír Popelínský); (65) Automatic Assessment of Student Reading Comprehension from Short Summaries (Lisa Mintz, Dan Stefanescu, Shi Feng, Sidney D'Mello and Arthur Graesser); (66) Building an Intelligent PAL from the Tutor.com Session Database Phase 1: Data Mining (Donald Morrison, Benjamin Nye, Borhan Samei, Vivek Varma Datla, Craig Kelly and Vasile Rus); (67) Building Automated Detectors of Gameplay Strategies to Measure Implicit Science Learning (Elizabeth Rowe, Ryan Baker, Jodi Asbell-Clarke, Emily Kasman and William Hawkins); (68) Challenges on Applying BKT to Model Student Knowledge in Multi-Context Online Learning Environment (Wolney Mello Neto and Eduardo Barbosa); (69) Combination of Statistical and Semantic Data Sources for the Improvement of Software Engineering Courses (Michael Koch, Markus Ring, Florian Otto and Dieter Landes); (70) Comparing Learning in a MOOC and a Blended On-Campus Course (Kimberly Colvin, John Champaign, Alwina Liu, Colin Fredericks and David Pritchard); (71) Cost-Effective, Actionable Engagement Detection at Scale (Ryan Baker and Jaclyn Ocumpaugh); (72) Data Mining of Undergraduate Course Evaluations (Sohail Javaad Syed, Yuheng Helen Jiang and Lukasz Golab); (73) Data Sharing: Low-Cost Sensors for Affect and Cognition (Keith Brawner) (74) Diagnosing Algebra Understanding via Inverse Planning (Anna Rafferty and Thomas Griffiths); (75) Discovering and Describing Types of Mathematical Errors (Thomas McTavish and Johann Larusson); (76) Discovering Prerequisite Relationships Among Knowledge Components (Richard Scheines, Elizabeth Silver and Ilya Goldin); (77) Dynamic Re-Composition of Learning Groups Using PSO-Based Algorithms (Zhilin Zheng and Niels Pinkwart); (78) Educational Data Mining and Analyzing of Student Learning Outcomes from the Perspective of Learning Experience (Zhongmei Shu, Qiong-Fei Qu and Lu-Qi Feng); (79) Using EEG in Knowledge Tracing (Yanbo Xu, Kai-Min Chang, Yueran Yuan and Jack Mostow); (80) Exploring Engaging Dialogues in Video Discussions (I-Han Hsiao, Hui Soo Chae, Manav Malhotra, Ryan Baker and Gary Natriello); (81) Exploring Indicators from Keyboard and Mouse Interactions to Predict the User Affective State (Sergio Salmeron-Majadas, Olga C. Santos and Jesus G. Boticario); (82) Extracting Latent Skills from Time Series of Asynchronous and Incomplete Examinations (Shinichi Oeda, Yu Ito and Kenji Yamanishi); (83) Generalizing and Extending a Predictive Model for Standardized Test Scores Based On Cognitive Tutor Interactions (Ambarish Joshi, Stephen Fancsali, Steven Ritter, Tristan Nixon and Susan Berman); (84) How Patterns in Source Codes of Students Can Help in Detection of Their Programming Skills? (Štefan Pero and Tomáš Horváth); (85) A Preliminary Investigation of Learner Characteristics for Unsupervised Dialogue Act Classification (Aysu Ezen-Can and Kristy Elizabeth Boyer); (86) Improving Retention Performance Prediction with Prerequisite Skill Features (Xiaolu Xiong, Seth Adjei and Neil Heffernan); (87) Indicator Visualization for Adaptive Exploratory Learning Environments (Sergio Gutierrez Santos, Manolis Mavrikis, Alex Poulovassilis and Zheng Zhu); (88) Learning Aid Use Patterns and Their Impact on Exam Performance in Online Developmental Mathematics (Nicole Forsgren Velasquez, Ilya Goldin, Taylor Martin and Jason Maughan); (89) Learning to Teach like a Bandit (Mykola Pechenizkiy and Pedro A. Toledo); (90) Matching Hypothesis Text in Diagrams and Essays (Collin Lynch, Mohammad Falakmasir and Kevin Ashley); (91) Matrix Factorization Feasibility for Sequencing and Adaptive Support in Intelligent Tutoring Systems (Carlotta Schatten, Ruth Janning, Manolis Mavrikis and Lars Schmidt-Thieme); (92) Microgenetic Designs for Educational Data Mining Research (Taylor Martin, Nicole Forsgren Velasquez, Ani Aghababyan, Jason Maughan and Philip Janisiewicz); (93) Mining and Identifying Relationships among Sequential Patterns in Multi-Feature, Hierarchical Learning Activity Data (Cheng Ye, John Kinnebrew and Gautam Biswas); (94) Mining Coherent Evolution Patterns in Education through Biclustering (André Vale, Sara C. Madeira and Claudia Antunes); (95) Mining Multi-Dimensional Patterns for Student Modelling (Andreia Silva and Claudia Antunes); (96) Mining Reading Comprehension Within Educational Objective Frameworks (Terry Peckham and Gordon McCalla); (97) Mining Students' Strategies to enable Collaborative Learning (Sergio Gutierrez-Santos, Manolis Mavrikis and Alexandra Poulovassilis); (98) Modeling Student Socioaffective Responses to Group Interactions in a Collaborative Online Chat Environment (Whitney Cade, Nia Dowell, Art Graesser, Yla Tausczik and James Pennebaker); (99) Now We're Talkin': Leveraging the Power of Natural Language Processing to Inform ITS Development (Laura Allen, Erica Snow and Danielle McNamara); (100) Peer Assessment in the First French MOOC: Analyzing Assessors' Behavior (Matthieu Cisel, Rémi Bachelet and Eric Bruillard); (101) Peer Influence on Attrition in Massively Open Online Courses (Diyi Yang, Miaomiao Wen and Carolyn Rose); (102) Predicting Students' Learning Achievement by Using Online Learning Patterns in Blended Learning Environments: Comparison of Two Cases on Linear and Non-Linear Model (Jeong Hyun Kim, Yeonjeong Park, Jongwoo Song and Il-Hyun Jo); (103) Predictive Performance of Prevailing Approaches to Skills Assessment Techniques: Insights from Real vs. Synthetic Data Sets (Behzad Beheshti and Michel Desmarais); (104) Recent-Performance Factors Analysis (April Galyardt and Ilya Goldin); (105) Refining Learning Maps with Data Fitting Techniques: Searching for Better Fitting Learning Maps (Seth Adjei, Douglas Selent, Neil Heffernan, Zach Pardos, Angela Broaddus and Neal Kingston); (106) Relevancy Prediction of Micro-Blog Questions in an Educational Setting (Mariheida Cordova-Sanchez, Parameswaran Raman, Luo Si and Jason Fish); (107) Singular Value Decomposition in Education: A Case Study on Recommending Courses (Fábio Carballo and Claudia Antunes); (108) The Predictive Power of SNA Metrics in Education (Diego García-Saiz, Camilo Palazuelos and Marta Zorrilla); (109) Data-Driven Curriculum Design: Mining the Web to Make Better Teaching Decisions (Antonio Moretti, Jose Gonzalez-Brenes and Katherine McKnight); (110) Towards IRT-Based Student Modeling from Problem Solving Steps (Manuel Hernando, Eduardo Guzmán, Sergey Sosnovsky, Eric Andres and Susanne Narciss); (111) Towards Uncovering the Mysterious World of Math Homework (Mingyu Feng); (112) Towards Using Similarity Measure for Automatic Detection of Significant Behaviors from Continuous Data (Ben-Manson Toussaint, Vanda Luengo and Jérôme Tonetti); (113) Using Data Mining to Automate ADDIE (Fritz Ray, Keith Brawner and Robby Robson); (114) Using Multimodal Learning Analytics to Study Learning Mechanisms in Hands-on Environments (Marcelo Worsley and Paulo Blikstein); (115) Using Problem Solving Times and Expert Opinion to Detect Skills (Juraj Nižnan, Radek Pelánek and Jirí Rihák); (116) Toward Collaboration Sensing: Multimodal Detection of the Chameleon Effect in Collaborative Learning Settings (Bertrand Schneider); (117) The Use of Student Confidence for Prediction & Resolving Individual Student Knowledge Structure (Charles Lang); (118) Nonverbal Communication and Teaching Performance (Roghayeh Barmaki); (119) Data-Driven Feedback Beyond Next-Step Hints (Michael Eagle and Tiffany Barnes); (120) E3: Emotions, Engagement and Educational Games (Ani Aghababyan); (121) MOOC Leaner Motivation and Learning Pattern Discovery--A Research Prospectus Paper (Yuan Wang); and (122) Personalization and Incentive Design in E-learning Systems (Avi Segal). Workshops presented include: (1) Graph-based Educational Data Mining (G-EDM) (Collin F. Lynch, Tiffany Barnes); (2) Non-Cognitive Factors & Personalization for Adaptive Learning (NCFPAL@EDM) (Steven Ritter, Stephen E. Fancsali); (3) Approaching Twenty Years of Knowledge Tracing: Lessons Learned, Open Challenges, and Promising Developments (Michael Yudelson, José P. González-Brenes, Michael Mozer); and (4) Feedback from Multimodal Interaction in Learning Management Systems (Lars Schmidt-Thieme, Arvid Kappas, Carles Sierra, Emanuele Ruffaldi). References are included in each presentation.