Search

Massive online Open Courses (MOOCs) make extensive use of videos. Students interact with them by pausing, seeking forward or backward, replaying segments, etc. We can reasonably assume that students have different patterns of video interactions, but it remains hard to compare student video interactions. Some methods were developed, such as Markov Chain and Edit Distance. However, these methods have caveats as we show with prototypical examples. This paper proposes a new methodology of comparing video sequences of interaction based both on time spent in each state and the succession of states by computing the distance between the transition matrices of the video interaction sequences. Results show the proposed methodology can better characterize video interaction in a task to discriminate which student is interacting with a video, or which video a student is interacting with. [For the full proceedings, see ED607784.]

Online "Peer Instruction" has become prevalent in many "flipped classroom" settings, yet little work has been done to examine the content students generate in such a learning environment. This study characterizes a dataset generated by an open-source, web-based homework system that prompts students to first answer questions, and then provide explanations of their reasoning. Of particular interest in this dataset, is that students are also prompted to evaluate a subset of peer explanations based on how convincing they are, as part of the Peer Instruction learning script. Since these student "votes" are then used in the selection of what is shown to future learners, we cast this as an instance of "learnersourcing," a paradigm that presents new research opportunities for the Learning Analytics community. This study characterizes a dataset from one "Peer Instruction" tool, that includes not only the student generated answers and explanations, but this novel "vote" attribute, which aims to capture how convincing each explanation is to other learners. The dataset includes longitudinal observations of student responses over the course of a semester, following groups from three STEM disciplines. The data is made available to interested researchers. [For the full proceedings, see ED607784.]

The 12th iteration of the International Conference on Educational Data Mining (EDM 2019) is organized under the auspices of the International Educational Data Mining Society in Montreal, Canada. The theme of this year's conference is EDM in Open-Ended Domains. As EDM has matured it has increasingly been applied to open-ended and ill-defined tasks such as writing, design, and collaborative problem solving, and it has been used in new informal contexts where student actions are at best semi-structured. This iteration of the conference includes a range of work in these and other areas. This year's conference features three invited talks: Julita Vassileva, Professor at the Department of Computer Science, University of Saskatchewan, Canada; Steve Ritter, Co-Founder and Chief Scientist, Carnegie Learning Inc., Pittsburgh; and Michael Mozer, Professor Department of Computer Science and Institute of Cognitive Science University of Colorado. The number of accepted papers include 22 full papers and 42 short papers. An additional 47 papers were accepted to the poster track. The poster and demo track itself accepted 14 contributions out of 34 submissions. Together with the "Journal of Educational Data Mining" ("JEDM"), the EDM 2019 conference held a "JEDM" Track that provides researchers a venue to deliver more substantial mature work than is possible in a conference proceeding and to present their work to a live audience. The papers submitted to this track followed the "JEDM" peer review process. Two such papers are featured in the conference's program. Additionally this year, papers that were regularly published in the journal in 2018 were invited for presentation at the conference. Two authors accepted this invitation. The main conference invited contributions to an Industry Track in addition to the main track. The EDM 2019 Industry Track received eleven submissions of which six were accepted. The EDM conference continues its tradition of providing opportunities for young researchers to present their work and receive feedback from their peers and senior researchers. The doctoral consortium this year features eight such presentations. This year's conference includes also an invited talk by the authors of the 2018 winner of the EDM Test of Time Award. This year's talk is delivered by Mykola Pechenizkiy. In addition to the main program, there are three workshops: (1) Learning Analytics: Building bridges between the Education and the Computing communities; (2) Reinforcement Learning for Educational Data Mining; and (3) Workshop on EDM & Games: Leveling Up Engaged Learning with Data-Rich Analytics. Three tutorials were presented as well: (1) Sharing and Reusing Data and Analytic Methods with LearnSphere; (2) Causal Discovery with Tetrad in LearnSphere's Tigris and Designing and Developing Open; and (3) Pedagogically-Based Predictive Models using the Moodle Analytics API.

In most contexts of student skills assessment, whether the test material is administered by the teacher or within a learning environment, there is a strong incentive to minimize the number of questions or exercises administered in order to get an accurate assessment. This minimization objective can be framed as a Q-matrix design problem: given a set of skills to assess and a fixed number of question items, determine the optimal set of items, out of a potentially large pool, that will yield the most accurate assessment. In recent years, the Q-matrix identifiability under DINA/DINO models has been proposed as a guiding principle for that purpose. We empirically investigate the extent to which identifiability can serve that purpose. Identifiability of Q-matrices is studied throughout a range of conditions in an effort to measure and understand its relation to student skills assessment. The investigation relies on simulation studies of skills assessment with synthetic data. Results show that identifiability is an important factor that determines the capacity of a Q-matrix to lead to accurate skills assessment with the least number of questions. [For the full proceedings, see ED593090.]

In recent years, substantial improvements were obtained in the effectiveness of data driven algorithms to validate the mapping of items to skills, or the Q-matrix. In the current study we use ensemble algorithms on top of existing Q-matrix refinement algorithms to improve their performance. We combine the boosting technique with a decision tree. The results show that the improvements from both the decision tree and Adaboost combined are better than the decision tree alone and yield substantial gains over the best performance of individual Q-matrix refinement algorithm. [For the full proceedings, see ED592609.]

This paper reports on an analyis of data from a novel "Peer Instruction" application, named DALITE. The Peer Instruction paradigm is well suited to take advantage of peer-input in web-based learning environments. DALITE implements an asynchronous instantiation of peer instruction: after submitting their answer to a multiple-choice question, students are asked to write a rationale for their choice. Then, they can compare their answer to other students' answers, and are asked to choose the best peer-submitted rationale among those displayed. We engaged in an analysis of student behaviour and learning outcomes in the DALITE learning environment. Specifically, we focus our investigation on the relationship between student proficiency, how students change their answers after reading each others' writings, and the peer-votes they earn in DALITE. Key results include: i) peer-votes earned is a significant predictor of success in the course; ii) there are no significant differences between strong and weak students in how often they switch from the correct answer to a wrong answer after consulting peer-rationales, or vice versa; iii) even though males outscore females in conceptual physics questions, females earn as many votes from their peers as males do for the content they produce when justifying their answer choices. [This work has been funded through the Programme d'Aide la Recherche sur l'Education et l'Apprentissage (PAREA), administered by the Ministere d'Education et Loisirs de Quebec.] [For complete proceedings, see ED560503.]

This study investigates the issue of the goodness of fit of different skills assessment models using both synthetic and real data. Synthetic data is generated from the different skills assessment models. The results show wide differences of performances between the skills assessment models over synthetic data sets. The set of relative performances for the different models create a kind of "signature" for each specific data. We conjecture that if this signature is unique, it is a good indicator that the corresponding model is a good fit to the data. [For complete proceedings, see ED560503.]

The problem of mapping items to skills is gaining interest with the emergence of recent techniques that can use data for both defining this mapping, and for refining mappings given by experts. We investigate the problem of refining mapping from an expert by combining the output of different techniques. The combination is based on a partition tree that combines the suggested refinements of three known techniques from the literature. Each technique is given as input a Q-matrix, that maps items to skills, and student test outcome data, and outputs a modified Q-matrix that constitutes suggested improvements. We test the accuracy of the partition tree combination techniques over both synthetic and real data. The results over synthetic data show a high improvement over the best single technique with a 86% error reduction on average for four different Q-matrices. For real data, the error reduction is 55%. In addition to the substantial error reduction, the partition tree refinements provide a much more stable performance than any single technique. These results suggest that the partition tree is a valuable refinement combination approach that can effectively take advantage of the complementarity of the Q-matrix refinement techniques. It brings the goal of using a data driven approach to refine the item to skill mapping closer to real applications, although challenges remain and are discussed. [For complete proceedings, see ED560503.]

The Second International Conference on Educational Data Mining (EDM2009) was held at the University of Cordoba, Spain, on July 1-3, 2009. EDM brings together researchers from computer science, education, psychology, psychometrics, and statistics to analyze large data sets to answer educational research questions. The increase in instrumented educational software and databases of student test scores, has created large repositories of data reflecting how students learn. The EDM conference focuses on computational approaches for using those data to address important educational questions. The broad collection of research disciplines ensures cross fertilization of ideas, with the central questions of educational research serving as a unifying focus. This publication presents the following papers: (1) A Comparison of Student Skill Knowledge Estimates (Elizabeth Ayers, Rebecca Nugent, Nema Dean); (2) Differences Between Intelligent Tutor Lessons, and the Choice to Go Off-Task (Ryan S.J.d. Baker); (3) A User-Driven and Data-Driven Approach for Supporting Teachers in Reflection and Adaptation of Adaptive Tutorials (Dror Ben-Naim, Michael Bain, and Nadine Marcus); (4) Detecting Symptoms of Low Performance Using Production Rules (Javier Bravo and Alvaro Ortigosa); (5) Predicting Students Drop Out: A Case Study (Gerben W. Dekker, Mykola Pechenizkiy and Jan M. Vleeshouwers); (6) Using Learning Decomposition and Bootstrapping with Randomization to Compare the Impact of Different Educational Interventions on Learning (Mingyu Feng, Joseph E. Beck and Neil T. Heffernan); (7) Does Self-Discipline impact students' knowledge and learning? (Yue Gong, Dovan Rai, Joseph E. Beck, and Neil T. Heffernan); (8) Consistency of Students' Pace in Online Learning (Arnon Hershkovitz and Rafi Nachmias); (9) Student Consistency and Implications for Feedback in Online Assessment Systems (Tara M. Madhyastha and Steven Tanimoto); (10) Edu-mining for Book Recommendation for Pupils (Ryo Nagata, Keigo Takeda, Koji Suda, Junichi Kakegawa, and Koichiro Morihiro); (11) Conditional Subspace Clustering of Skill Mastery: Identifying Skills that Separate Students (Rebecca Nugent, Elizabeth Ayers, and Nema Dean); (12) Determining the Significance of Item Order In Randomized Problem Sets (Zachary A. Pardos and Neil T. Heffernan); (13) Learning Factors Transfer Analysis: Using Learning Curve Analysis to Automatically Generate Domain Models (Philip I. Pavlik Jr., Hao Cen, Kenneth R. Koedinger); (14) Detecting and Understanding the Impact of Cognitive and Interpersonal Conflict in Computer Supported Collaborative Learning Environments (David Nadler Prata, Ryan S.J.d. Baker, Evandro d.B. Costa, Carolyn P. Rose, Yue Cui, Adriana M.J.B. de Carvalho); (15) Using Dirichlet priors to improve model parameter plausibility (Dovan Rai, Yue Gong, Joseph E. Beck); (16) Reducing the Knowledge Tracing Space (Steven Ritter, Thomas K. Harris, Tristan Nixon, Daniel Dickison, R. Charles Murray, and Brendon Towle); (17) Automatic Detection of Student Mental Models During Prior Knowledge Activation in MetaTutor (Vasile Rus, Mihai Lintean, and Roger Azevedo); (18) Automatic Concept Relationships Discovery for an Adaptive E-course (Marian Simko, Maria Bielikova); (19) Unsupervised MDP Value Selection for Automating ITS Capabilities (John Stamper and Tiffany Barnes); (20) Recommendation in Higher Education Using Data Mining Techniques (Cesar Vialardi, Javier Bravo Agapito, Leila Shafti, Alvaro and Ortigosa); (21) Developing an Argument Learning Environment Using Agent-Based ITS (ALES) (Safia Abbas and Hajime Sawamura); (22) A Data Mining Approach to Reveal Representative Collaboration Indicators in Open Collaboration Frameworks (Antonio R. Anaya and Jesus G. Boticario); (23) Dimensions of Difficulty in Translating Natural Language into First-Order Logic (Dave Barker-Plummer, Richard Cox, and Robert Dale); (24) Predicting Correctness of Problem Solving from Low-level Log Data in Intelligent Tutoring Systems (Suleyman Cetintas, Luo Si, Yan Ping Xin, and Casey Hord); (25) Back to the future: a non-automated method of constructing transfer models (Ming Feng and Joseph Beck); (26) How do Students Organize Personal Information Spaces? (Sharon Hardof-Jaffe, Arnon Hershkovitz, Hama Abu-Kishk, Ofer Bergman, and Rafi Nachmias); (27) Improving Student Question Classification (Cecily Heiner and Joseph L. Zachary); (28) Why, What, and How to Log? Lessons from LISTEN (Jack Mostow and Joseph E. Beck); (29) Process Mining Online Assessment Data (Mykola Pechenizkiy, Nikola Trcka, Ekaterina Vasilyeva, Wil van der Aalst, and Paul De Bra); (30) Obtaining Rubric Weights For Assessments By More Than One Lecturer Using A Pairwise Learning Model (J. R. Quevedo and E. Montanes); (31) Collaborative Data Mining Tool for Education (Enrique Garcia, Cristobal Romero, Sebastian Ventura, Miguel Gea, and Carlos de Castro); (32) Predicting Student Grades in Learning Management Systems with Multiple Instance Genetic Programming (Amelia Zafra and Sebastian Ventura); and (33) Visualization of Differences in Data Measuring Mathematical Skills (Lukas Zoubek and Michal Burda). Individual papers contain tables, figures, footnotes, references and appendices.