Search

Data papers, as one of the channels to encourage researchers to open up research data under the open science movement, are expected to provide strong incentives through formal citations. However, few studies have investigated the drivers of this emerging type of publication. This study examines researchers' motivations, and considerations for data paper submission, as well as their perspectives on this scholarly publication. Through an in‐depth interview approach with ten data paper authors, our preliminary results found that, researchers are often driven by extrinsic factors to increase their publications, and data papers are sometimes viewed as territory claims before further research. Although the academic community widely recognizes the benefits of publishing data papers, some still cast a doubtful eye on its academic value and impact. We anticipate such insights on the driving forces and point of views of data papers could provide opportunities for stakeholders to fill gaps and strengthen the open science ecosystem. [ABSTRACT FROM AUTHOR]

We propose an innovative, effective, and data-agnostic method to train a deep-neural network model with an extremely small training dataset, called VELR (Voting-based Ensemble Learning with Rejection). In educational research and practice, providing valid labels for a sufficient amount of data to be used for supervised learning can be very costly and often impractical. The shortage of training data often results in deep neural networks being overfitting. There are many methods to avoid overfitting such as data augmentation and regularization. Though, data augmentation is considerably data dependent and does not usually work well for natural language processing tasks. Moreover, regularization is often quite task specific and costly. To address this issue, we propose an ensemble of overfitting models with uncertainty-based rejection. We hypothesize that misclassification can be identified by estimating the distribution of the class-posterior probability P(y|x) as a random variable. The proposed VELR method is data independent, and it does not require changes to the model structure or the re-training of the model. Empirical studies demonstrated that VELR achieved classification accuracy of 0.7 with only 200 samples per class on the CIFAR-10 dataset, but 75% of input samples were rejected. VELR was also applied to a question generation task using a BERT language model with only 350 training data points, which resulted in generating questions that are indistinguishable from human-generated questions. The paper concludes that VELR has potential applications to a broad range of real-world problems where misclassification is very costly, which is quite common in the educational domain. [For the complete proceedings, see ED630829.]

The intangible concept of data, as part of statistical literacy, can be complex for young children to grasp. Inquiry as a pedagogy has potential for supporting student development of statistical literacy as the investigation process is driven by the inquiry question. The aim of this paper is to gain insight into how a teacher's communication processes with her students supported their emerging understandings about the abstract concept of data. In this exploratory case study, we present data from a Year 4 classroom (age 9) in a guided mathematical inquiry within the STEM context of agricultural science. The inquiry question the students addressed was, "How big is a leaf?" The inquiry focused on linking data to the real-life context the data represented.

The university admission test comes after the high school graduation exam - the baccalaureate. The baccalaureate results of each candidate must be known by the university admissions committee. They provide information on the degree of intelligence, the skills acquired up to this date, but also the presence of inclinations and skills indispensable to the fulfillment of professional aspirations. The university entrance exam should not be focused only on quantity and quality of knowledge. Besides that, one of the objectives of this exam must be to test the interest in completing the studies through the university level for which he opts, but also the candidate's skills that "offer" him the productive and satisfying course of the entire cycle of higher education, even the perspective of future achievements. To realize that three psychological investigation tools of candidates (tests) can be used, necessary to highlight: (1) personality profile of the candidate; (2) interest profile, motivational; and (3) aptitude profile. The paper proposes a new admission methodology: the data collected through the proposed tests and correlated with the high school graduation data can accept the candidates, as admitted to the profile they opted for, or can redirect them to choose the right path. [For the full proceedings, see ED630948.]

Despite calls to increase the focus on explainability and interpretability in EDM and, in particular, student success prediction, so that it becomes useful for personalized intervention systems, only few efforts have been undertaken in that direction so far. In this paper, we argue that this is mainly due to the limitations of current Explainable Artificial Intelligence (XAI) approaches regarding interpretability. We further argue that the issue, thus, calls for a a combination of AI and social science methods utilizing the strengths of both. For this, we introduce a step-wise model of interpretability where the first step constitutes of knowing important features, the second step of understanding counterfactuals regarding a particular person's prediction, and the third step of uncovering causal relations relevant for a set of similar students. We show that LIME, a current XAI method, reaches the first but not subsequent steps. To reach step two, we propose an extension to LIME, Minimal Counterfactual-LIME, finding the smallest number of changes necessary to change a prediction. Reaching step three, however, is more involved and additionally requires theoretical and causal reasoning - to this end, we construct an easily applicable framework. Using artificial data, we showcase that our methods can recover connections among features; additionally, we demonstrate its applicability on real-life data. Limitations of our methods are discussed and collaborations with social scientists encouraged. [For the full proceedings, see ED623995.]

This work explores how population-based engagement prediction can address cold-start at scale in large learning resource collections. The paper introduces: (1) VLE, a novel dataset that consists of content and video based features extracted from publicly available scientific video lectures coupled with implicit and explicit signals related to learner engagement; (2) two standard tasks related to predicting and ranking context-agnostic engagement in video lectures with preliminary baselines; and (3) a set of experiments that validate the usefulness of the proposed dataset. Our experimental results indicate that the newly proposed VLE dataset leads to building context-agnostic engagement prediction models that are significantly performant than ones based on previous datasets, mainly attributing to the increase of training examples. VLE dataset's suitability in building models towards Computer Science/ Artificial Intelligence education focused on e-learning/MOOC use-cases is also evidenced. Further experiments in combining the built model with a personalising algorithm show promising improvements in addressing the cold-start problem encountered in educational recommenders. This is the largest and most diverse publicly available dataset to our knowledge that deals with learner engagement prediction tasks. The dataset, helper tools, descriptive statistics and example code snippets are available publicly. [For the full proceedings, see ED623995.]

The automatic code generation is the process of generating source code snippets from a program, i.e., code for generating code. Its importance lies in facilitating software development, particularly important is helping in the implementation of software designs such as engineering diagrams, in such a case, automatic code generation copes with the problem of how to obtain code from a graphic representation, for instance an UML diagram or a Relational Diagram. Some advantages of automatic code generation are: a) to obtain the source code more quickly and to do it with lower margins of error; b) it is promising to be applied in teaching contexts, whilst provide instructors with a tool to teach, the expected results of assignments can be assessed by comparing the results of students and the automatic generated code. Furthermore, one of the most frequently tasks in classrooms when teaching relational databases is the design of Entity-Relationship Diagrams which eventually become SQL code. The manual transition from an Entity-Relationship Diagram to SQL code is a time-consuming process and requires of a skilled eye to be successfully performed. In this paper, we present "DerSql," an extension of the DIA Diagrammer, a well-known free software engineering tool, to automatically generate SQL code from an Entity-Relationship Diagrams. The results are tested for the case of 1 -- 1 and 1 -- n arities relationships. We consider that "DerSql" represents a remarkable tool for teaching while it is a promising advance in developing DIA as a 4th Generation software engineering application. [For the full proceedings, see ED638044.]

In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of nonignorable missing data in the VLE log le data, and this is expected to negatively impact IRT item parameter estimation accuracy, which then negatively impacts any future ability estimates utilized in the VLE. In the psychometric literature, methods for handling missing data are mostly centered around conditions in which the data and the amount of missing data are not as large as those that come from VLEs. In this paper, we introduce a semi-supervised learning method to deal with a large proportion of missingness contained in VLE data from which one needs to obtain unbiased item parameter estimates. The proposed framework showed its potential for obtaining unbiased item parameter estimates that can then be fixed in the VLE in order to obtain ongoing ability estimates for operational purposes. [This paper was published in: V. Cavalli-Sforza, A. N. Rafferty, C. Romero, & J. Whitehill (Eds.), "Proceedings of The 13th International Conference on Educational Data Mining (EDM 2020)," (pp. 715-719).]

ITSs have the requirement to be adaptive to the student with AI. The classical ITS architecture defines three components to split the data and to keep it flexible and thus adaptive. However, there is a lack of abstract descriptions how to put adaptive behavior into practice. This paper defines how you can structure your data for case based systems in a way that adaptivity is easier to achieve while maintaining the classical splitting of the system and reducing the data footprint. Building a case based system from a collection of exchangeable steps is also possible with this approach. Two variants of adaptivity based on the data structure are explored and both can be used in conjunction. [For the full proceedings, see ED621108.]

Typical data science instruction uses generic datasets like survival rates on the Titanic, which may not be motivating for students. Will introducing real-life data science problems fill this motivational deficit? To analyze this question, we contrasted learning with generic datasets and artificial problems (Phase 1) with a community-sourced dataset and authentic problems (Phase 2) in the context of an 8-week virtual internship. Retrospective survey questions indicated interns experienced increased motivation in Phase 2. Additionally, analysis of intern discourse using Linguistic Inquiry and Word Count (LIWC) indicated a significant difference in linguistic measures between the two phases. Phase 1 had significantly greater measures of pronouns with a small-medium effect size, 2nd person words with a medium-large effect size, positive emotion with a medium effect size, inter-rogations with a medium-large effect size, question marks with a medium-large effect size, risk with a medium-large effect size, and causal words with a medium effect size. These results in conjunction with a retrospective survey suggest that phase 1 had more questions asked, more causal relationships defined, and included linguistic features of success and failure. Results from Phase 2 indicated that community-sourced data and problems may increase motivation for learning data science. [For the full proceedings, see ED623995.]

The application of item response theory (IRT) is almost universal in the development, implementation, and maintenance of large-scale assessments. Therefore, establishing the fit of IRT models to data is essential as the viability of calibration and equating implementations depend on it. In a typical test administration situation, measurement disturbances that influence model data fit are expected. Unfortunately, test administrations nationwide experienced new measurement disturbances because of the COVID-19 pandemic. Given the substantial disruption in education, did the response patterns of test takers change enough that model data fit is threatened and the degree of confidence in applying IRT analyses diminished? Using data from a large-scale state assessment system's 2019 and 2021 administration of the same test forms, model and data fit statistics for items and test takers were evaluated. The summary item fit index Q[subscript 1] (Yen, 1993) and the person fit statistic l[subscript z] (Choi, 2010; Drasgow et. al., 1985) were used for the analyses. Results from the study provide evidence that there wasn't a greater risk to the use of IRT models in 2021 than in previous years, despite the measurement disturbances introduced by the COVID-19 pandemic.

Big data (BD) is the buzz phrase these days. Everyone is talking about its potential, its volume, its variety, and its velocity. Knowledge management (KM) has been around since the mid-1990s. The goals of KM have been to collect, store, categorize, mine, and process data into knowledge. The methods of knowledge acquisition varied from organizational culture to the next. Typical processes converted data into information through traditional databases and then applied business intelligence and data mining methodologies to extract knowledge. With the recent arrival of big data as a disruptive technology and the center of big data, this paper attempts to combine KM and BD fields together. Both areas could help each other tremendously. KM historically, when applied correctly, has helped managers to make decisions faster and better, prevented reinventing the wheel, preserved some talented processes through keeping track of best practices, and prompted innovation due to knowledge sharing and dissemination. BD deals with massive amount of data and does not require a traditional database to be effective. BD has its tools and requirement that can be enhanced through KM. The final aim of this paper is to recreate a model where both big data and knowledge management coexist. The author hopes with a better understanding of both fields to develop a new course where the focus is a productive intersection of knowledge management and big data. To keep up with changing times, this paper will bring the needed awareness of these fields for information systems and business students. [For the full proceedings, see ED575713.]

Accountability seems forever engrained into the K-12 environment, as has been the expectation of delivering quality education to school aged children and adolescents. Yet, repeated failure of this expectation has focused the public's and policy maker's attention on the limitations of major accountability systems. This paper explores applications of machine learning, predictive analytics, and data visualization to student information available to educational decision makers. In particular, we demonstrate how to use individual academic performance histories to identify "at-risk" students in real time for advising, academic coaching, and other support services and how to aggregate longitudinal data at the school or district level for system modeling, profiling, comparison, and intervention evaluation.

In higher education, predictive analytics can provide actionable insights to diverse stakeholders such as administrators, instructors, and students. Separate feature sets are typically used for different prediction tasks, e.g., student activity logs for predicting in-course performance and registrar data for predicting long-term college success. However, little is known about the overall utility of different data sources across prediction tasks and the fairness of their predictions with respect to different subpopulations. Using data from over 2,000 college students at a large public university, we examined the utility of institutional data, learning management system (LMS) data, and survey data for accurately and fairly predicting short-term and long-term student success. We found that institutional data and LMS data both have decent predictive power, but survey data shows very little predictive utility. Combining institutional data with LMS data leads to even higher accuracy than using either alone. In terms of fairness, using institutional data consistently underestimates historically disadvantaged student subpopulations more than their peers, whereas LMS data tend to overestimate some of these groups more often. Combining the two data sources does not fully neutralize the biases and still leads to high rates of underestimation among disadvantaged groups. Moreover, algorithmic biases affect not only demographic minorities but also students with acquired disadvantages. These analyses serve to inform more cost-effective and equitable use of student data for predictive analytics applications in higher education. [For the full proceedings, see ED607784.]

Supervised machine learning has become one of the most important methods for developing educational and intelligent tutoring software; it is the backbone of many educational data mining methods for estimating knowledge, emotion, and other aspects of learning. Hence, in order to ensure optimal utilization of computing resources and effective analysis of models, it is essential that researchers know which evaluation metrics are best suited to educational data. In this article, we focus on the problem of wrapper feature selection, where predictors are added to models based on how much they improve model accuracy in terms of a given metric. We compared commonly-used machine learning algorithms including naive Bayes, support vector machines, logistic regression, and random forests on 11 diverse learning-related datasets. We optimized feature selection based on nine different metrics, then evaluated each to address research questions about how effective each metric was in terms of the others (e.g., does optimizing for precision also result in good F1?) as well as calibration (i.e., are predictions produced by models accurate probabilities of correctness?). We provide empirical evidence that the Matthews correlation coefficient (MCC) produced the overall best results across the other metrics, but that root mean squared error (RMSE) selected the best-calibrated models. Finally, we also discuss issues related to the number of features selected when optimizing for each metric, as well as the types of datasets for which certain metrics were more effective. [For the full proceedings, see ED607784.]

Twelve scholarly papers and eighteen abstracts comprise the content of the nineteenth annual Brick & Click Libraries Conference, held annually at Northwest Missouri State University in Maryville, Missouri. The proceedings, authored by academic librarians and presented at the conference, portray the contemporary and future face of librarianship. The 2019 paper and abstract titles include: (1) Universally Accessible: Practical Methods for Digital Accessibility for Ordinary Librarians (Rebeca Peacock and Sean McCann); (2) Developing Student Agency through Personalized Learning (Elizabeth Stephan and Shevell Thibou); (3) Students of the Stacks: The Fellowship of Experience (Christina Prucha and Sabrina Davis); (4) Library Assessment: How High the Mountain! (Cheryl L. Blevens, Shelley Arvin, and Valentine K. Muyumba); (5) The Tool We Never Teach: The Good, the Bad and the End (Point) of Google's Usefulness in First-Year Experience Assignments (Lindsay Brownfield); (6) Mentoring and You: Providing Meaningful Experiences for Student Employees (Ruth Harries); (7) Spending Miss Alice's Money: Practical Applications for Evolving Libraries (Phillip Jones); (8) Using Data Visualization to Analyze Topic Development by Business Communication Students in a One-Shot Setting (Heidi Blackburn and Jason Heppler); (9) Collaborate for Student Success: Long Night against Procrastination (Carolyn Johnson); (10) Using Acquisitions: A Library's Steps for Ordering Books to Tracking Funds (Natasha Young); (11) Smart Spending -- Affordable Programming (Amber Carr and Kayla Reed); (12) Be a Leader, Not a Boss: Creating a Unified and Fulfilled Student Employment Team (Courtney Gard and Adrianna Bennett); (13) Revitalization of the Liaison Program (Amber Carr, Kayla Reed, and Nancy Crabtree); (14) Maintaining Training (Kayla Reed); (15) Understanding the Basics of Serials...and Beyond (Carol Doms and Stephanie Spratt); (16) Off the Web and Into the Fishing Hole (Carmen Orth-Alfie and Natalie Mahan); (17) Open Educational Resources: Build a Lesson Using Free Government Resources (Stephanie Hallam, Pat Willingham, and Kris Baranovic); (18) Old Acquaintance -- New Friend...Library and IT Partnerships (Martha Allen); (19) Display and Exhibit Design: Maximizing Existing Spaces for Outreach and Stakeholder Engagement (Alyssa Denneler); (20) Cross-Training Student Workers in a Learning Commons Environment (Meredith Knoff); (21) Deciding, Documenting, and Disseminating Library Policies and Practices: A Case Study from a Newly-Created Library Unit (Alea Henle and Rob Withers); (22) Reference Remodeling: Adapting and Promoting Reference Services (Cyndi Landis); (23) The Reference Interview Beyond Reference: Putting it to the E-Resource Troubleshooting Test (Angela Rathmel and Greta Valentine); (24) Town-Town-Gown Relations: Managing a County-Wide Collaborative Digital Community Archive (Christopher Jones, Allison Haack, Monique Shore, and Julia Bauder); (25) "The Library is Home for a Lot of Us Engineers:" How Engineering Majors Experience Library Space (Noël Kopriva); (26) Creating Library FAQ Guidelines Using Query Spy (Danielle M. Drummond); (27) A Reprieve for Sisyphus: How a Rotating ILL Task Schedule Provides Full Task Coverage and Variety for Staff (Gail Williams); (28) Hosting Comic Book Club in an Academic Setting (Mara Inge); (29) We Need Diverse Digital Resources (Jo Monahan); and (30) Accessible & Active eLearning Game Plan (Monica Maher). [For the 2018 proceedings, see ED590389.]

The field of education is undergoing fundamental change with the growing use of data. Fine-scale data collection at the item-response level is now possible. Xerox has developed a system that bridges the paper-to-digital divide by providing the well-established and easy-to-use paper interface to students, but digitizes the responses for scoring, validating, reporting, and managing data using a range of digital technologies. The Ignite™ system supports written responses, shading, connecting lines, multiple choice selections, and other question types. For some users, monitoring is at a very fine-grain level in both time and skill, while for others the data is used for more summative evaluations and strategic planning; one users' details may be another user's overview. All the reports presented in this document use the same basic atomic data elements and associated meta-data. The hierarchical nature of the organization of users requires that these atomic elements be combined in different ways for specialized visual representations, dependent upon the needs of the user. [For complete proceedings, see ED560503.]

In this paper, the authors attempted to evaluate design of leaning units with use of Learning Analytics technique on page flip information. Traditional formative assessment has been carried out by giving assignments and evaluating their results. However, the information that teacher can get from the evaluation is limited and coarse-grained. The authors set a research question that whether one can evaluate relation between learning objectives of learning units and the learners' actual activities in the units from page flip histories. The experimental result showed that the intensity of relation between the assignments and learning materials were different for each unit. Quantitatively, the correlation coefficients between "p-value of chi-square tests between range of page flip count and grade of assignment" and "proportion of number of questions in an assessment with reference page" was -0.889. With use of this relation, the authors attempted to evaluate design of units, and found out that the thing that may be critical to get high grade in an assignment of a certain unit was how many times they refer to their text book, though this assignment was designed for assessment of learners' skills to use or apply the learned knowledge. If teacher can get such suggestion as feedback, it is possible to utilize for formative assessment. [For full proceedings, see ED571332.]

In this paper, we present a dataset consisting of data generated from 22 previously and currently running randomized controlled experiments inside the ASSISTments online learning platform. This dataset provides data mining opportunities for researchers to analyze ASSISTments data in a convenient format across multiple experiments at the same time. The data preprocessing steps are explained in detail to inform researchers about how this dataset was generated. A list of column descriptions is provided to define the columns in the dataset and a set of summary statistics are presented to briefly describe the dataset.

Intelligent Tutoring Systems (ITSs) have shown success in the domain of programming, in part by providing customized hints and feedback to students. However, many popular novice programming environments still lack these intelligent features. This is due in part to their use of open-ended programming assignments, which are difficult to support with existing hint generation techniques. In this paper, we present a new data-driven algorithm, based on the Hint Factory, to generate hints for these open-ended assignments. We evaluate our algorithm on historical student data and show that it can provide hints that successfully lead students to solutions from any state, help students achieve assignment objectives, and align with the student's future solution. [For the full proceedings, see ED592609.]

One of the largest challenges in educator preparation in higher education is the effective use of data for actionable and meaningful decision making about programs for current and future educators, administrators, and educational leaders. Within programs there are many types of students. These may include elementary, secondary, special education, foreign language, social studies, science, literacy, health, school level administrators, district administrators, curriculum leaders, and many others based on school and program. However, in all instances the program (i.e. college, university, school, etc.) is charged with using data in a meaningful manner to guide program decisions, make revisions to curriculum and clinical experiences, and determine the overall quality of the program. Visual Scaling provides an effective solution, and is a process whereby goals and expectations are clearly articulated, and baseline and performance assessments are compared against the expected level of performance. A bibliography is included.

Course selection can be a daunting task, especially for first year students. Sub-optimal selection can lead to bad performance of students and increase the dropout rate. Given the availability of historic data about student performances, it is possible to aid students in the selection of appropriate courses. Here, we propose a method to compose a personalized curriculum for a given student. We develop a modular approach that combines a context-aware grade prediction with statistical information on the useful temporal ordering of courses. This allows for meaningful course recommendations, both for fresh and senior students. We demonstrate the approach using the data of the computer science Bachelor students at Saarland University. [For the full proceedings, see ED593090.]

Lawson (1983a) offered that an individuals' experiences as students in schools are significant in shaping their views of teaching. Pajares (1992) argued that a teacher's perceptions of the classroom are a product of their experiences as students. The purpose of this study was to examine the relationship between physical education (PE) pre-service teachers': conceptions of assessment in their role as students, knowledge of classroom assessment (CA), data self-efficacy and intention to engage in data-driven decision making upon entering the profession. Results indicate PE pre-service teachers with more positive conceptions of assessment in their student role show greater knowledge about CA, have better opinions of data and greater intention to use data and CAs as evidence for decision-making.

Data-driven learning (DDL) is an inductive approach to language learning in which students study examples of authentic language and use them to find patterns of language use. This inductive approach to learning has the advantages of being learner-centered, encouraging hypothesis testing and learner autonomy, and helping develop learning skills. The approach has grown out of corpus linguistics, and it has been used to help students learn grammar and vocabulary usage. It is also possible to use it for teaching speech acts. In this paper, we contrast inductive and deductive learning and discuss an example of an exercise using data-driven learning to learn about the expressions and strategies used in apologies. [For full proceedings, see ED565044.]

Corpora have multiple affordances, not least for use by teachers and learners of a foreign language (L2) in what has come to be known as "data-driven learning" or DDL. The corpus and concordance interface were originally conceived by and for linguists, so other users need to adopt the role of "language researcher" to make the most of them. Despite the alleged advantages of this, it does create a potential barrier for occasional or non-specialist users in particular. While researchers debate the status of the "web-as-corpus", the Internet represents a vast bank of data already familiar to most people; less discussed is the status of "Google-as-concordancer"--another familiar tool. This paper discusses some of the advantages and disadvantages of this approach from a pedagogical perspective. (Contains 4 footnotes.)

Data-driven methods have been a successful approach to generating hints for programming problems. However, the majority of previous studies are focused on procedural hints that aim at moving students to the next closest state to the solution. In this paper, I propose a data-driven method to generate remedy hints for BOTS, a game that teaches programming through a block-moving puzzle. Remedy hints aim to help students out of dead-end states, which are states in the problem from where no student has ever derived a solution. To address this, my proposed work includes designing debugging activities and generating remedy hints from students' solutions to debugging activities. [For complete proceedings, see ED560503.]

We present an analysis of log data from a semester's use of the OpenDSA eTextbook system with the goal of determining the most difficult course topics in a data structures course. While experienced instructors can identify which topics students most struggle with, this often comes only after much time and effort, and does not provide real-time analysis that might benefit an intelligent tutoring system. Our factors included the fraction of wrong answers given by student, results from Item Response Theory, and the rate of model answer and hint use by students. We grouped exercises by topic covered to yield a list of topics associated with the harder exercises. We found that a majority of these exercises were related to algorithm analysis topics. We compared our results to responses given by a sample of experienced instructors, and found that the automated results match the expert opinions reasonably well. We investigated reasons that might explain the over-representation of algorithm analysis among the difficult topics, and hypothesize that visualizations might help to better present this material. [For the full proceedings, see ED592609.]

With the growing popularity of MOOCs and sharp trend of digitalizing education, there is a huge amount of free digital educational material on the web along with the activity logs of large number of participating students. However, this data is largely unstructured and there is hardly any information about the relationship between material from different sources. We propose a generic algorithm to use educational material and student activity data from heterogeneous sources to create a Prerequisite Structure Graph (PSG). A PSG is a directed acyclic graph, where the nodes are educational units and the edges specify the pairwise ordering of the units in effective teaching by instructors or for effective learning by students. We propose an unsupervised approach utilizing both text content and student data, which outperforms to supervised methods (utilizing only text content) on the task of estimating a PSG. [For the full proceedings, see ED592609.]

We have been incrementally adding data-driven methods into the Deep Thought logic tutor for the purpose of creating a fully data-driven intelligent tutoring system. Our previous research has shown that the addition of data-driven hints, worked examples, and problem assignment can improve student performance and retention in the tutor. In this study, we investigate how the addition of these methods affects students' demonstrative knowledge of logic proof solving using their post-tutor examination scores. We have used data collected from three test conditions with different combinations of our data-driven additions to determine which methods are most beneficial to students who demonstrate higher or lower knowledge of the subject matter. Our results show that students who are assigned problems based on profiling proficiency compared to prior exemplary students with similar problem-solving behavior show higher examination scores overall, and the use of proficiency profiling increases retention and reduces the amount of time taken in-tutor for lower performing students in particular. The results from this study also helps differentiate the behavior of higher and lower performing students in tutor, which can allow quicker interventions for lower proficiency students. [For the full proceedings, see ED592609.]

The first intelligent tutoring systems for computer programming have been proposed more than 30 years ago, mostly focusing on well defined programming tasks e.g. in the context of logic programming. Recent systems also teach complex programs, where explicit modelling of every possible program and mistake is no longer possible. Such systems are based on data-driven approaches, which focus on the syntax of a program or consider the output for example cases. However, the system's understanding of student programs could be enriched by a deeper focus on the actual execution of a program. This requires a suitable data representation which encodes information of programming style as well as its functionality in a suitable way, thus offering entry points for automated feedback generation. In this contribution we propose a representation of computer programs via execution traces for example input and demonstrate the power of this representation in three key challenges for intelligent tutoring systems: identifying the underlying solution strategy, identifying erroneous solutions and locating the errors in erroneous programs for feedback display. [For the full proceedings, see ED592609.]

The increasing use of educational technologies in classrooms is producing vast amounts of process data that capture rich information about learning as it unfolds. The field of educational data mining has made great progress in using log data to build models that improve instruction and advance the science of learning. Thus far, however, the predictive and explanatory power of such models has often been limited to the actions that educational technologies can log. A major challenge in incorporating more contextually rich data streams into models of learning is collecting and integrating data from different sources and at different grain sizes. We present our methodological advances in automating the integration of log data with additional multi-modal (e.g., audio, screen video, webcam video) data streams. We also demonstrate several examples of how integrating multiple streams of data into the knowledge component (KC) model refinement process improves the predictive fit of student models and yields important pedagogical implications. This work represents an important advancement in facilitating the integration of rich qualitative details of students' learning contexts into the quantitative approaches characteristic of EDM research. [For the full proceedings, see ED592609.]

Sustainable Development Goals call on Members States to "ensure inclusive and equitable quality education and promote lifelong learning opportunities for all" and sets a number of targets related to technical and vocational education and training (TVET). In order to collect input from the global TVET community, UNESCO-UNEVOC organized a virtual conference from May 9-16, 2016 on the UNEVOC TVeT Forum. Moderated by Phil Loveder and John Stanwick from the National Centre for Vocational Education Research in Australia, a UNEVOC Centre, this virtual conference sought to inform the wider TVET community about a current research project currently undertaken by the NCVER. The collaborative research project aims to identify and highlight key issues in the measurement of the return on investment in TVET, and the virtual conference was an opportunity for the community to share their ideas, expertise and experiences with the research team. This virtual conference was the fifteenth in a series of moderator-driven discussions introduced by UNESCO-UNEVOC in 2011. Conducted on the UNEVOC TVeT Forum--a global online community of over 4,500 members--and guided by an expert, these discussions provide a platform for sharing of experiences, expertise and feedback. A list of participants is included.

This field report is the ninth in a series produced by the Center on Innovations in Learning's League of Innovators. The series describes, discusses, and analyzes policies and practices that enable personalization in education. This report introduces sessions from the "Conversations with Innovators" event held at Temple University, June 22-23, 2016. In each of three sessions, pairs of experts (each of whom had written a chapter in the "Handbook on Personalized Learning for States, Districts, and Schools") made brief 5-7 minute presentations on the designated topic, after which the floor was then opened for participants' questions and discussion. This issue of "Connect" presents Session 3, which addressed "Information Technologies to Advance Teaching and Learning" with Janet S. Twyman and Ryan Baker as presenters' respondents. [For the "Handbook on Personalized Learning for States, Districts, and Schools," see ED568173; for the seventh issue of "Connect," see ED568179; and for the eighth issue of "Connect," see ED568180.]

This study investigates the issue of the goodness of fit of different skills assessment models using both synthetic and real data. Synthetic data is generated from the different skills assessment models. The results show wide differences of performances between the skills assessment models over synthetic data sets. The set of relative performances for the different models create a kind of "signature" for each specific data. We conjecture that if this signature is unique, it is a good indicator that the corresponding model is a good fit to the data. [For complete proceedings, see ED560503.]

How can we automatically determine which skills must be mastered for the successful completion of an online course? Large-scale online courses (e.g., MOOCs) often contain a broad range of contents frequently intended to be a semester's worth of materials; this breadth often makes it difficult to articulate an accurate set of skills and knowledge (i.e., a skill model, or the QMatrix). We have developed an innovative method to discover skill models from the data of online courses. Our method assumes that online courses have a pre-defined skill map for which skills are associated with formative assessment items embedded throughout the online course. Our method carefully exploits correlations between various parts of student performance, as well as in the text of assessment items, to build a superior statistical model that even outperforms human experts. To evaluate our method, we compare our method with existing methods (LFA) and human engineered skill models on three Open Learning Initiative (OLI) courses at Carnegie Mellon University. The results show that (1) our method outperforms human-engineered skill models, (2) skill models discovered by our method are interpretable, and (3) our method is remarkably faster than existing methods. These results suggest that our method provides a significant contribution to the evidence-based, iterative refinement of online courses with a promising scalability. [For complete proceedings, see ED560503.]

The NEAIR 2013 Conference Proceedings is a compilation of papers presented at the Newport, Rhode Island, conference. Papers in this document include: (1) Aspiring to the Role of "Data Badass:" Some Thoughts on the Political Context of IR (Mark Freeman); (2) Data-Driven Internal Benchmarks and Successful Learning Outcomes (Mamta Saxena and Douglas L. Flor); (3) Going Test-Optional: Gathering Evidence and Making the Decision at Ithaca College (Yuko Mulugetta); (4) Influence of Factors on Cumulative GPA of Freshman and Transfer Students (Tania Das); (5) An Institutional Model for Degree Completion: A "Moneyball" Approach (Alexander Yin and Lecitia Oseguera);(6) The Ripple Effect: Declining Student Interest in Humanities amidst Growing Demand for Professional Education (Elissa Lu); (7) Timing is Everything: What we can Learn from "Survey Procrastinators" (Lauren Conoscenti); (8) Understanding the Leaky STEM Pipeline by Taking a Close Look at Factors Influencing Retention and Graduation Rates (Di Chen and Heather Kelly);(9) Using National Benchmarking Assessment to Inform Student Learning Outcomes (Mark Rotondo and Darlena Jones). [Individual papers contain references.]

The IADIS CELDA 2012 Conference intention was to address the main issues concerned with evolving learning processes and supporting pedagogies and applications in the digital age. There had been advances in both cognitive psychology and computing that have affected the educational arena. The convergence of these two disciplines is increasing at a fast pace and affecting academia and professional practice in many ways. Paradigms such as just-in-time learning, constructivism, student-centered learning and collaborative approaches have emerged and are being supported by technological advancements such as simulations, virtual reality and multi-agents systems. These developments have created both opportunities and areas of serious concerns. This conference aimed to cover both technological as well as pedagogical issues related to these developments. The IADIS CELDA 2012 Conference received 98 submissions from more than 24 countries. Out of the papers submitted, 29 were accepted as full papers. In addition to the presentation of full papers, short papers and reflection papers, the conference also includes a keynote presentation from internationally distinguished researchers. Individual papers contain figures, tables, and references.

One of the common tasks of inferential statistics is to compare two data sets. Long before formal statistical procedures, however, students can be encouraged to make comparisons between data sets and therefore build up intuitive statistical reasoning. Such tasks also give meaning to the data collection students may do. This study describes the answers given by beginning university students to tasks involving comparing data sets in graphical form, originally designed for students between Grades 3 to 9. The results show that whereas all the students had successfully completed either pre-tertiary mathematics or a bridging mathematics course many had similar difficulties to students of a younger age. In particular, they did not use a measure of centre or proportional reasoning when appropriate.

The 5th International Conference on Educational Data Mining (EDM 2012) is held in picturesque Chania on the beautiful Crete island in Greece, under the auspices of the International Educational Data Mining Society (IEDMS). The EDM 2012 conference is a leading international forum for high quality research that mines large data sets of educational data to answer educational research questions. These data sets may come from learning management systems, interactive learning environments, intelligent tutoring systems, or any system used in a learning context. The following papers are presented at the conference: (1) Stream Mining in Education? Dealing with Evolution (Myra Spiliopoulou); (2) From Text to Feedback: Leveraging Data Mining to Build Educational Technologies (Danielle S. McNamara); (3) Five Aspirations for Educational Data Mining (Bob Dolan and John Behrens); (4) Assisting Instructional Assessment of Undergraduate Collaborative Wiki and SVN Activities (Jihie Kim, Erin Shaw, Hao Xu and Adarsh G V); (5) Automated Student Model Improvement (Kenneth R. Koedinger, Elizabeth A. McLaughlin and John C. Stamper); (6) Automatic Discovery of Speech Act Categories in Educational Games (Vasile Rus, Arthur Graesser, Cristian Moldovan and Nobal Niraula); (7) Co-Clustering by Bipartite Spectral Graph Partitioning for Out-of-Tutor Prediction (Shubhendu Trivedi, Zachary Pardos, Gabor Sarkozy and Neil Heffernan); (8) Comparison of methods to trace multiple subskills: Is LR-DBN best? (Yanbo Xu and Jack Mostow); (9) Dynamic Cognitive Tracing: Towards Unified Discovery of Student and Cognitive Models (Jose Gonzalez-Brenes and Jack Mostow); (10) Identifying Learning Behaviors by Contextualizing Differential Sequence Mining with Action Features and Performance Evolution (John S. Kinnebrew and Gautam Biswas); (11) Identifying Students' Characteristic Learning Behaviors in an Intelligent Tutoring System Fostering Self-Regulated Learning (Francois Bouchet, John S. Kinnebrew, Gautam Biswas and Roger Azevedo); (12) Learner Differences in Hint Processing (Ilya Goldin, Kenneth R. Koedinger and Vincent Aleven); (13) Methods to find the number of latent skills (Behzad Beheshti, Michel C. Desmarais and Rhouma Naceur); (14) Mining Student Behavior Patterns in Reading Comprehension Tasks (Terry Peckham and Gordon McCalla); (15) Model-Based Collaborative Filtering Analysis of Student Response Data: Machine-Learning Item Response Theory (Yoav Bergner, Stefan Droschler, Gerd Kortemeyer, Saif Rayyan, Daniel Seaton and David E. Pritchard); (16) Predicting drop-out from social behaviour of students (Tomas Obsivac, Lubomir Popelinsky, Jaroslav Bayer, Jan Geryk and Hana Bydzovska); (17) Searching for Variables and Models to Investigate Mediators of Learning from Multiple Representations (Martina A. Rau and Richard Scheines); (18) The Impact on Individualizing Student Models on Necessary Practice Opportunities (Jung In Lee and Emma Brunskill); (19) Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra (Ryan S.J.D. Baker, Sujith M. Gowda, Michael Wixon, Jessica Kalka, Angela Z. Wagner, Aatish Salvi, Vincent Aleven, Gail W. Kusbit, Jaclyn Ocumpaugh and Lisa Rossi); (20) Using Edit Distance to Analyse Errors in a Natural Language to Logic Translation Corpus (Dave Barker-Plummer, Robert Dale, and Richard Cox); (21) Calculating Probabilistic Distance to Solution in a Complex Problem Solving Domain (Leigh Ann Sudol, Kelly Rivers and Thomas K. Harris); (22) Classification via clustering for predicting final marks based on student participation in forums (M.I. Lopez, J.M. Luna, C. Romero, and S. Ventura); (23) Development of a Workbench to Address the Educational Data Mining Bottleneck (Ma. Mercedes T. Rodrigo, Ryan S. J. D. Baker, Bruce McLaren, Alejandra Jayme and Thomas T. Dy); (24) Early Prediction of Student Self-Regulation Strategies by Combining Multiple Models (Jennifer L. Sabourin, Bradford W. Mott and James C. Lester); (25) Identifying Successful Learners from Interaction Behaviour (Judi McCuaig and Julia Baldwin); (26) Interaction Networks: Generating High Level Hints Based on Network Community Clusterings (Michael Eagle, Matthew Johnson and Tiffany Barnes); (27) Interleaved Practice with Multiple Representations: Analyses with Knowledge Tracing Based Techniques (Martina A. Rau and Zachary A. Pardos); (28) Learning Gains for Core Concepts in a Serious Game on Scientific Reasoning (Carol Forsyth, Philip Pavlik Jr, Arthur C. Graesser, Zhiqiang Cai, Mae-Lynn Germany, Keith Millis, Heather Butler, Diane Halpern and Robert P. Dolan); (29) Leveraging First Response Time into the Knowledge Tracing Model (Yutao Wang and Neil T. Heffernan); (30) Meta-learning Approach for Automatic Parameter Tuning: A case of study with educational datasets (M.M. Molina, J.M. Luna, C. Romero, and S. Ventura); (31) Mining Concept Maps to Understand University Students' Learning (Jin Soung Yoo and Moon-Heum Cho); (32) Policy Building--An Extension To User Modeling (Michael V. Yudelson and Emma Brunskill); (33) The real world significance of performance prediction (Zachary A. Pardos, Qing Yang Wang and Shubhendu Trivedi); (34) The Rise of the Super Experiment (John C. Stamper, Derek Lomas, Dixie Ching, Steven Ritter, Kenneth R. Koedinger and Jonathan Steinhart); (35) Using Student Modeling to Estimate Student Knowledge Retention (Yutao Wang and Joseph Beck); (36) A promising classification method for predicting distance students' performance (Diego Garcia-Saiz and Marta Zorrilla); (37) Analyzing paths in a student database (Donatella Merlini, Renza Campagni and Renzo Sprugnoli); (38) Analyzing the behavior of a teacher network in a Web 2.0 environment (Eliana Scheihing, Carolina Aros and Daniel Guerra); (39) Automated Detection of Mentors and Players in an Educational Game (Fazel Keshtkar, Brent Morgan and Arthur Graesser); (40) Categorizing Students' Response Patterns using the Concept of Fractal Dimension (Rasil Warnakulasooriya and William Galen); (41) CurriM: Curriculum Mining (M. Pechenizkiy, N. Trcka, P. De Bra and Pedro A. Toledo); (42) Data mining techniques for design of ITS student models (Ritu Chaturvedi and C. I. Ezeife); (43) Deciding on Feedback Polarity and Timing (Stuart Johnson and Osmar Zaiane); (44) Finding Dependent Test Items: An Information Theory Based Approach (Xiaoxun Sun); (45) Fit-to-Model Statistics for Evaluating Quality of Bayesian Student Ability Estimation (Ling Tan); (46) Inferring learners' knowledge from observed actions (Anna N. Rafferty, Michelle M. Lamar and Thomas L. Griffiths); (47) Learning Paths in a Non-Personalizing e-Learning Environment (Agathe Merceron, Sebastian Schwarzrock, Margarita Elkina, Andreas Pursian, Liane Beuster, Albrecht Fortenbacher, Leonard Kappe, and Boris Wenzlaff); (48) Similarity Functions for Collaborative Master Recommendations (Alexandru Surpatean, Evgueni Smirnov and Nicolai Manie); (49) Social Networks Analysis for Quantifying Students' Performance in Teamwork (Pedro Crespo and Claudia Antunes); (50) Speaking (and touching) to learn: a method for mining the digital footprints of face-to-face collaboration (Roberto Martinez Maldonado, Kalina Yacef and Judy Kay); (51) Stress Analytics in Education (Rafal Kocielnik, Mykola Pechenizkiy and Natalia Sidorova); and (52) Variable Construction and Causal Discovery for Cognitive Tutor Log Data: Initial Results (Stephen E. Fancsali). Individual papers contain figures, tables, references and footnotes. [Support for this publication was provided by Carnegie Learning, Pearson and LearnLab.]

The Second International Conference on Educational Data Mining (EDM2009) was held at the University of Cordoba, Spain, on July 1-3, 2009. EDM brings together researchers from computer science, education, psychology, psychometrics, and statistics to analyze large data sets to answer educational research questions. The increase in instrumented educational software and databases of student test scores, has created large repositories of data reflecting how students learn. The EDM conference focuses on computational approaches for using those data to address important educational questions. The broad collection of research disciplines ensures cross fertilization of ideas, with the central questions of educational research serving as a unifying focus. This publication presents the following papers: (1) A Comparison of Student Skill Knowledge Estimates (Elizabeth Ayers, Rebecca Nugent, Nema Dean); (2) Differences Between Intelligent Tutor Lessons, and the Choice to Go Off-Task (Ryan S.J.d. Baker); (3) A User-Driven and Data-Driven Approach for Supporting Teachers in Reflection and Adaptation of Adaptive Tutorials (Dror Ben-Naim, Michael Bain, and Nadine Marcus); (4) Detecting Symptoms of Low Performance Using Production Rules (Javier Bravo and Alvaro Ortigosa); (5) Predicting Students Drop Out: A Case Study (Gerben W. Dekker, Mykola Pechenizkiy and Jan M. Vleeshouwers); (6) Using Learning Decomposition and Bootstrapping with Randomization to Compare the Impact of Different Educational Interventions on Learning (Mingyu Feng, Joseph E. Beck and Neil T. Heffernan); (7) Does Self-Discipline impact students' knowledge and learning? (Yue Gong, Dovan Rai, Joseph E. Beck, and Neil T. Heffernan); (8) Consistency of Students' Pace in Online Learning (Arnon Hershkovitz and Rafi Nachmias); (9) Student Consistency and Implications for Feedback in Online Assessment Systems (Tara M. Madhyastha and Steven Tanimoto); (10) Edu-mining for Book Recommendation for Pupils (Ryo Nagata, Keigo Takeda, Koji Suda, Junichi Kakegawa, and Koichiro Morihiro); (11) Conditional Subspace Clustering of Skill Mastery: Identifying Skills that Separate Students (Rebecca Nugent, Elizabeth Ayers, and Nema Dean); (12) Determining the Significance of Item Order In Randomized Problem Sets (Zachary A. Pardos and Neil T. Heffernan); (13) Learning Factors Transfer Analysis: Using Learning Curve Analysis to Automatically Generate Domain Models (Philip I. Pavlik Jr., Hao Cen, Kenneth R. Koedinger); (14) Detecting and Understanding the Impact of Cognitive and Interpersonal Conflict in Computer Supported Collaborative Learning Environments (David Nadler Prata, Ryan S.J.d. Baker, Evandro d.B. Costa, Carolyn P. Rose, Yue Cui, Adriana M.J.B. de Carvalho); (15) Using Dirichlet priors to improve model parameter plausibility (Dovan Rai, Yue Gong, Joseph E. Beck); (16) Reducing the Knowledge Tracing Space (Steven Ritter, Thomas K. Harris, Tristan Nixon, Daniel Dickison, R. Charles Murray, and Brendon Towle); (17) Automatic Detection of Student Mental Models During Prior Knowledge Activation in MetaTutor (Vasile Rus, Mihai Lintean, and Roger Azevedo); (18) Automatic Concept Relationships Discovery for an Adaptive E-course (Marian Simko, Maria Bielikova); (19) Unsupervised MDP Value Selection for Automating ITS Capabilities (John Stamper and Tiffany Barnes); (20) Recommendation in Higher Education Using Data Mining Techniques (Cesar Vialardi, Javier Bravo Agapito, Leila Shafti, Alvaro and Ortigosa); (21) Developing an Argument Learning Environment Using Agent-Based ITS (ALES) (Safia Abbas and Hajime Sawamura); (22) A Data Mining Approach to Reveal Representative Collaboration Indicators in Open Collaboration Frameworks (Antonio R. Anaya and Jesus G. Boticario); (23) Dimensions of Difficulty in Translating Natural Language into First-Order Logic (Dave Barker-Plummer, Richard Cox, and Robert Dale); (24) Predicting Correctness of Problem Solving from Low-level Log Data in Intelligent Tutoring Systems (Suleyman Cetintas, Luo Si, Yan Ping Xin, and Casey Hord); (25) Back to the future: a non-automated method of constructing transfer models (Ming Feng and Joseph Beck); (26) How do Students Organize Personal Information Spaces? (Sharon Hardof-Jaffe, Arnon Hershkovitz, Hama Abu-Kishk, Ofer Bergman, and Rafi Nachmias); (27) Improving Student Question Classification (Cecily Heiner and Joseph L. Zachary); (28) Why, What, and How to Log? Lessons from LISTEN (Jack Mostow and Joseph E. Beck); (29) Process Mining Online Assessment Data (Mykola Pechenizkiy, Nikola Trcka, Ekaterina Vasilyeva, Wil van der Aalst, and Paul De Bra); (30) Obtaining Rubric Weights For Assessments By More Than One Lecturer Using A Pairwise Learning Model (J. R. Quevedo and E. Montanes); (31) Collaborative Data Mining Tool for Education (Enrique Garcia, Cristobal Romero, Sebastian Ventura, Miguel Gea, and Carlos de Castro); (32) Predicting Student Grades in Learning Management Systems with Multiple Instance Genetic Programming (Amelia Zafra and Sebastian Ventura); and (33) Visualization of Differences in Data Measuring Mathematical Skills (Lukas Zoubek and Michal Burda). Individual papers contain tables, figures, footnotes, references and appendices.

Interactive problem solving environments, such as intelligent tutoring systems and educational video games, produce large amounts of transactional data which make it a challenge for both researchers and educators to understand how students work within the environment. Researchers have modeled the student-tutor interactions using complex network representations to automatically derive next-step hints, derive high-level approaches, and create visualizations of student behavior. However, students do not explore the complete problem space. The nonuniform exploration of the problem results in smaller networks and less next-step hints. In this work we explore the possibility of using frequency estimation to uncover locations in the network with differing amounts of student-saturation. Identification of these regions can be used to locate specific problem approaches and strategies that would be most improved by additional student-data. [For complete proceedings, see ED560503.]