Search

This position paper discusses the role of open access research data within mathematics education, a relatively new initiative across the wider research community. International and national policy documents are explored and examples from both the scientific and social science paradigms of mathematical sciences and mathematics education respectively are provided. Within these examples, some of the more well-known concerns associated with making data open and accessible are acknowledged and debated.

We propose an innovative, effective, and data-agnostic method to train a deep-neural network model with an extremely small training dataset, called VELR (Voting-based Ensemble Learning with Rejection). In educational research and practice, providing valid labels for a sufficient amount of data to be used for supervised learning can be very costly and often impractical. The shortage of training data often results in deep neural networks being overfitting. There are many methods to avoid overfitting such as data augmentation and regularization. Though, data augmentation is considerably data dependent and does not usually work well for natural language processing tasks. Moreover, regularization is often quite task specific and costly. To address this issue, we propose an ensemble of overfitting models with uncertainty-based rejection. We hypothesize that misclassification can be identified by estimating the distribution of the class-posterior probability P(y|x) as a random variable. The proposed VELR method is data independent, and it does not require changes to the model structure or the re-training of the model. Empirical studies demonstrated that VELR achieved classification accuracy of 0.7 with only 200 samples per class on the CIFAR-10 dataset, but 75% of input samples were rejected. VELR was also applied to a question generation task using a BERT language model with only 350 training data points, which resulted in generating questions that are indistinguishable from human-generated questions. The paper concludes that VELR has potential applications to a broad range of real-world problems where misclassification is very costly, which is quite common in the educational domain. [For the complete proceedings, see ED630829.]

The intangible concept of data, as part of statistical literacy, can be complex for young children to grasp. Inquiry as a pedagogy has potential for supporting student development of statistical literacy as the investigation process is driven by the inquiry question. The aim of this paper is to gain insight into how a teacher's communication processes with her students supported their emerging understandings about the abstract concept of data. In this exploratory case study, we present data from a Year 4 classroom (age 9) in a guided mathematical inquiry within the STEM context of agricultural science. The inquiry question the students addressed was, "How big is a leaf?" The inquiry focused on linking data to the real-life context the data represented.

The State Higher Education Executive Officers Association's (SHEEO) "Communities of Practice" project builds upon SHEEO's ongoing efforts to measure the capacity and effective use of state postsecondary data systems and provides states with opportunities to develop solutions to common issues with those systems. The sixth Community of Practice convening, "Aligning K-12 and Postsecondary Education," was held December 7-8, 2021, in Denver, Colorado. The two-day meeting included representatives from 13 states: Delaware, Georgia, Hawaii, Idaho, Kentucky, Louisiana, Maryland, Missouri, Pennsylvania, Rhode Island, Tennessee, Utah, and Washington. Teams included representatives from SHEEO agencies, state K-12 agencies, P-20 partnership organizations, state longitudinal data systems, and others. The Community of Practice also addressed the impacts of the COVID-19 pandemic on student transitions and how state data systems can be used to promote equitable outcomes for low-income students and students of color. During the convening, teams explored practical uses of state P-20 data for improving college access, equity, and success and how state postsecondary data can better inform the K-12 to postsecondary pipeline. This white paper highlights key themes and findings of the convening, including challenges and lessons learned from the participating state teams and suggestions of topics for further consideration. Case studies describing ongoing efforts in Georgia and Pennsylvania are included in the appendices. Presentations from the December 2021 convening are available on SHEEO's state postsecondary data website.

The university admission test comes after the high school graduation exam - the baccalaureate. The baccalaureate results of each candidate must be known by the university admissions committee. They provide information on the degree of intelligence, the skills acquired up to this date, but also the presence of inclinations and skills indispensable to the fulfillment of professional aspirations. The university entrance exam should not be focused only on quantity and quality of knowledge. Besides that, one of the objectives of this exam must be to test the interest in completing the studies through the university level for which he opts, but also the candidate's skills that "offer" him the productive and satisfying course of the entire cycle of higher education, even the perspective of future achievements. To realize that three psychological investigation tools of candidates (tests) can be used, necessary to highlight: (1) personality profile of the candidate; (2) interest profile, motivational; and (3) aptitude profile. The paper proposes a new admission methodology: the data collected through the proposed tests and correlated with the high school graduation data can accept the candidates, as admitted to the profile they opted for, or can redirect them to choose the right path. [For the full proceedings, see ED630948.]

Despite calls to increase the focus on explainability and interpretability in EDM and, in particular, student success prediction, so that it becomes useful for personalized intervention systems, only few efforts have been undertaken in that direction so far. In this paper, we argue that this is mainly due to the limitations of current Explainable Artificial Intelligence (XAI) approaches regarding interpretability. We further argue that the issue, thus, calls for a a combination of AI and social science methods utilizing the strengths of both. For this, we introduce a step-wise model of interpretability where the first step constitutes of knowing important features, the second step of understanding counterfactuals regarding a particular person's prediction, and the third step of uncovering causal relations relevant for a set of similar students. We show that LIME, a current XAI method, reaches the first but not subsequent steps. To reach step two, we propose an extension to LIME, Minimal Counterfactual-LIME, finding the smallest number of changes necessary to change a prediction. Reaching step three, however, is more involved and additionally requires theoretical and causal reasoning - to this end, we construct an easily applicable framework. Using artificial data, we showcase that our methods can recover connections among features; additionally, we demonstrate its applicability on real-life data. Limitations of our methods are discussed and collaborations with social scientists encouraged. [For the full proceedings, see ED623995.]

This work explores how population-based engagement prediction can address cold-start at scale in large learning resource collections. The paper introduces: (1) VLE, a novel dataset that consists of content and video based features extracted from publicly available scientific video lectures coupled with implicit and explicit signals related to learner engagement; (2) two standard tasks related to predicting and ranking context-agnostic engagement in video lectures with preliminary baselines; and (3) a set of experiments that validate the usefulness of the proposed dataset. Our experimental results indicate that the newly proposed VLE dataset leads to building context-agnostic engagement prediction models that are significantly performant than ones based on previous datasets, mainly attributing to the increase of training examples. VLE dataset's suitability in building models towards Computer Science/ Artificial Intelligence education focused on e-learning/MOOC use-cases is also evidenced. Further experiments in combining the built model with a personalising algorithm show promising improvements in addressing the cold-start problem encountered in educational recommenders. This is the largest and most diverse publicly available dataset to our knowledge that deals with learner engagement prediction tasks. The dataset, helper tools, descriptive statistics and example code snippets are available publicly. [For the full proceedings, see ED623995.]

The automatic code generation is the process of generating source code snippets from a program, i.e., code for generating code. Its importance lies in facilitating software development, particularly important is helping in the implementation of software designs such as engineering diagrams, in such a case, automatic code generation copes with the problem of how to obtain code from a graphic representation, for instance an UML diagram or a Relational Diagram. Some advantages of automatic code generation are: a) to obtain the source code more quickly and to do it with lower margins of error; b) it is promising to be applied in teaching contexts, whilst provide instructors with a tool to teach, the expected results of assignments can be assessed by comparing the results of students and the automatic generated code. Furthermore, one of the most frequently tasks in classrooms when teaching relational databases is the design of Entity-Relationship Diagrams which eventually become SQL code. The manual transition from an Entity-Relationship Diagram to SQL code is a time-consuming process and requires of a skilled eye to be successfully performed. In this paper, we present "DerSql," an extension of the DIA Diagrammer, a well-known free software engineering tool, to automatically generate SQL code from an Entity-Relationship Diagrams. The results are tested for the case of 1 -- 1 and 1 -- n arities relationships. We consider that "DerSql" represents a remarkable tool for teaching while it is a promising advance in developing DIA as a 4th Generation software engineering application. [For the full proceedings, see ED638044.]

In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of nonignorable missing data in the VLE log le data, and this is expected to negatively impact IRT item parameter estimation accuracy, which then negatively impacts any future ability estimates utilized in the VLE. In the psychometric literature, methods for handling missing data are mostly centered around conditions in which the data and the amount of missing data are not as large as those that come from VLEs. In this paper, we introduce a semi-supervised learning method to deal with a large proportion of missingness contained in VLE data from which one needs to obtain unbiased item parameter estimates. The proposed framework showed its potential for obtaining unbiased item parameter estimates that can then be fixed in the VLE in order to obtain ongoing ability estimates for operational purposes. [This paper was published in: V. Cavalli-Sforza, A. N. Rafferty, C. Romero, & J. Whitehill (Eds.), "Proceedings of The 13th International Conference on Educational Data Mining (EDM 2020)," (pp. 715-719).]

ITSs have the requirement to be adaptive to the student with AI. The classical ITS architecture defines three components to split the data and to keep it flexible and thus adaptive. However, there is a lack of abstract descriptions how to put adaptive behavior into practice. This paper defines how you can structure your data for case based systems in a way that adaptivity is easier to achieve while maintaining the classical splitting of the system and reducing the data footprint. Building a case based system from a collection of exchangeable steps is also possible with this approach. Two variants of adaptivity based on the data structure are explored and both can be used in conjunction. [For the full proceedings, see ED621108.]

Feedback on how students progress through completing subgoals can improve students' learning and motivation in programming. Detecting subgoal completion is a challenging task, and most learning environments do so either with "expert-authored" models or with "data-driven" models. Both models have advantages that are complementary -- expert models encode domain knowledge and achieve reliable detection but require "extensive authoring efforts" and often cannot capture all students' possible solution strategies, while data-driven models can be easily scaled but may be less accurate and interpretable. In this paper, we take a step towards achieving the best of both worlds -- utilizing a data-driven model that can intelligently detect subgoals in students' correct solutions, while benefiting from human expertise in editing these data-driven subgoal rules to provide more accurate feedback to students. We compared our hybrid "humanized" subgoal detectors, built from data-driven subgoals modified with expert input, against an existing data-driven approach and baseline supervised learning models. Our results showed that the hybrid model outperformed all other models in terms of overall accuracy and F1-score. Our work advances the challenging task of automated subgoal detection during programming, while laying the groundwork for future hybrid expert-authored/data-driven systems. [For the full proceedings, see ED615472.]

The application of item response theory (IRT) is almost universal in the development, implementation, and maintenance of large-scale assessments. Therefore, establishing the fit of IRT models to data is essential as the viability of calibration and equating implementations depend on it. In a typical test administration situation, measurement disturbances that influence model data fit are expected. Unfortunately, test administrations nationwide experienced new measurement disturbances because of the COVID-19 pandemic. Given the substantial disruption in education, did the response patterns of test takers change enough that model data fit is threatened and the degree of confidence in applying IRT analyses diminished? Using data from a large-scale state assessment system's 2019 and 2021 administration of the same test forms, model and data fit statistics for items and test takers were evaluated. The summary item fit index Q[subscript 1] (Yen, 1993) and the person fit statistic l[subscript z] (Choi, 2010; Drasgow et. al., 1985) were used for the analyses. Results from the study provide evidence that there wasn't a greater risk to the use of IRT models in 2021 than in previous years, despite the measurement disturbances introduced by the COVID-19 pandemic.

Typical data science instruction uses generic datasets like survival rates on the Titanic, which may not be motivating for students. Will introducing real-life data science problems fill this motivational deficit? To analyze this question, we contrasted learning with generic datasets and artificial problems (Phase 1) with a community-sourced dataset and authentic problems (Phase 2) in the context of an 8-week virtual internship. Retrospective survey questions indicated interns experienced increased motivation in Phase 2. Additionally, analysis of intern discourse using Linguistic Inquiry and Word Count (LIWC) indicated a significant difference in linguistic measures between the two phases. Phase 1 had significantly greater measures of pronouns with a small-medium effect size, 2nd person words with a medium-large effect size, positive emotion with a medium effect size, inter-rogations with a medium-large effect size, question marks with a medium-large effect size, risk with a medium-large effect size, and causal words with a medium effect size. These results in conjunction with a retrospective survey suggest that phase 1 had more questions asked, more causal relationships defined, and included linguistic features of success and failure. Results from Phase 2 indicated that community-sourced data and problems may increase motivation for learning data science. [For the full proceedings, see ED623995.]

Big data (BD) is the buzz phrase these days. Everyone is talking about its potential, its volume, its variety, and its velocity. Knowledge management (KM) has been around since the mid-1990s. The goals of KM have been to collect, store, categorize, mine, and process data into knowledge. The methods of knowledge acquisition varied from organizational culture to the next. Typical processes converted data into information through traditional databases and then applied business intelligence and data mining methodologies to extract knowledge. With the recent arrival of big data as a disruptive technology and the center of big data, this paper attempts to combine KM and BD fields together. Both areas could help each other tremendously. KM historically, when applied correctly, has helped managers to make decisions faster and better, prevented reinventing the wheel, preserved some talented processes through keeping track of best practices, and prompted innovation due to knowledge sharing and dissemination. BD deals with massive amount of data and does not require a traditional database to be effective. BD has its tools and requirement that can be enhanced through KM. The final aim of this paper is to recreate a model where both big data and knowledge management coexist. The author hopes with a better understanding of both fields to develop a new course where the focus is a productive intersection of knowledge management and big data. To keep up with changing times, this paper will bring the needed awareness of these fields for information systems and business students. [For the full proceedings, see ED575713.]

Accountability seems forever engrained into the K-12 environment, as has been the expectation of delivering quality education to school aged children and adolescents. Yet, repeated failure of this expectation has focused the public's and policy maker's attention on the limitations of major accountability systems. This paper explores applications of machine learning, predictive analytics, and data visualization to student information available to educational decision makers. In particular, we demonstrate how to use individual academic performance histories to identify "at-risk" students in real time for advising, academic coaching, and other support services and how to aggregate longitudinal data at the school or district level for system modeling, profiling, comparison, and intervention evaluation.

Supervised machine learning has become one of the most important methods for developing educational and intelligent tutoring software; it is the backbone of many educational data mining methods for estimating knowledge, emotion, and other aspects of learning. Hence, in order to ensure optimal utilization of computing resources and effective analysis of models, it is essential that researchers know which evaluation metrics are best suited to educational data. In this article, we focus on the problem of wrapper feature selection, where predictors are added to models based on how much they improve model accuracy in terms of a given metric. We compared commonly-used machine learning algorithms including naive Bayes, support vector machines, logistic regression, and random forests on 11 diverse learning-related datasets. We optimized feature selection based on nine different metrics, then evaluated each to address research questions about how effective each metric was in terms of the others (e.g., does optimizing for precision also result in good F1?) as well as calibration (i.e., are predictions produced by models accurate probabilities of correctness?). We provide empirical evidence that the Matthews correlation coefficient (MCC) produced the overall best results across the other metrics, but that root mean squared error (RMSE) selected the best-calibrated models. Finally, we also discuss issues related to the number of features selected when optimizing for each metric, as well as the types of datasets for which certain metrics were more effective. [For the full proceedings, see ED607784.]

In higher education, predictive analytics can provide actionable insights to diverse stakeholders such as administrators, instructors, and students. Separate feature sets are typically used for different prediction tasks, e.g., student activity logs for predicting in-course performance and registrar data for predicting long-term college success. However, little is known about the overall utility of different data sources across prediction tasks and the fairness of their predictions with respect to different subpopulations. Using data from over 2,000 college students at a large public university, we examined the utility of institutional data, learning management system (LMS) data, and survey data for accurately and fairly predicting short-term and long-term student success. We found that institutional data and LMS data both have decent predictive power, but survey data shows very little predictive utility. Combining institutional data with LMS data leads to even higher accuracy than using either alone. In terms of fairness, using institutional data consistently underestimates historically disadvantaged student subpopulations more than their peers, whereas LMS data tend to overestimate some of these groups more often. Combining the two data sources does not fully neutralize the biases and still leads to high rates of underestimation among disadvantaged groups. Moreover, algorithmic biases affect not only demographic minorities but also students with acquired disadvantages. These analyses serve to inform more cost-effective and equitable use of student data for predictive analytics applications in higher education. [For the full proceedings, see ED607784.]

Twelve scholarly papers and eighteen abstracts comprise the content of the nineteenth annual Brick & Click Libraries Conference, held annually at Northwest Missouri State University in Maryville, Missouri. The proceedings, authored by academic librarians and presented at the conference, portray the contemporary and future face of librarianship. The 2019 paper and abstract titles include: (1) Universally Accessible: Practical Methods for Digital Accessibility for Ordinary Librarians (Rebeca Peacock and Sean McCann); (2) Developing Student Agency through Personalized Learning (Elizabeth Stephan and Shevell Thibou); (3) Students of the Stacks: The Fellowship of Experience (Christina Prucha and Sabrina Davis); (4) Library Assessment: How High the Mountain! (Cheryl L. Blevens, Shelley Arvin, and Valentine K. Muyumba); (5) The Tool We Never Teach: The Good, the Bad and the End (Point) of Google's Usefulness in First-Year Experience Assignments (Lindsay Brownfield); (6) Mentoring and You: Providing Meaningful Experiences for Student Employees (Ruth Harries); (7) Spending Miss Alice's Money: Practical Applications for Evolving Libraries (Phillip Jones); (8) Using Data Visualization to Analyze Topic Development by Business Communication Students in a One-Shot Setting (Heidi Blackburn and Jason Heppler); (9) Collaborate for Student Success: Long Night against Procrastination (Carolyn Johnson); (10) Using Acquisitions: A Library's Steps for Ordering Books to Tracking Funds (Natasha Young); (11) Smart Spending -- Affordable Programming (Amber Carr and Kayla Reed); (12) Be a Leader, Not a Boss: Creating a Unified and Fulfilled Student Employment Team (Courtney Gard and Adrianna Bennett); (13) Revitalization of the Liaison Program (Amber Carr, Kayla Reed, and Nancy Crabtree); (14) Maintaining Training (Kayla Reed); (15) Understanding the Basics of Serials...and Beyond (Carol Doms and Stephanie Spratt); (16) Off the Web and Into the Fishing Hole (Carmen Orth-Alfie and Natalie Mahan); (17) Open Educational Resources: Build a Lesson Using Free Government Resources (Stephanie Hallam, Pat Willingham, and Kris Baranovic); (18) Old Acquaintance -- New Friend...Library and IT Partnerships (Martha Allen); (19) Display and Exhibit Design: Maximizing Existing Spaces for Outreach and Stakeholder Engagement (Alyssa Denneler); (20) Cross-Training Student Workers in a Learning Commons Environment (Meredith Knoff); (21) Deciding, Documenting, and Disseminating Library Policies and Practices: A Case Study from a Newly-Created Library Unit (Alea Henle and Rob Withers); (22) Reference Remodeling: Adapting and Promoting Reference Services (Cyndi Landis); (23) The Reference Interview Beyond Reference: Putting it to the E-Resource Troubleshooting Test (Angela Rathmel and Greta Valentine); (24) Town-Town-Gown Relations: Managing a County-Wide Collaborative Digital Community Archive (Christopher Jones, Allison Haack, Monique Shore, and Julia Bauder); (25) "The Library is Home for a Lot of Us Engineers:" How Engineering Majors Experience Library Space (Noël Kopriva); (26) Creating Library FAQ Guidelines Using Query Spy (Danielle M. Drummond); (27) A Reprieve for Sisyphus: How a Rotating ILL Task Schedule Provides Full Task Coverage and Variety for Staff (Gail Williams); (28) Hosting Comic Book Club in an Academic Setting (Mara Inge); (29) We Need Diverse Digital Resources (Jo Monahan); and (30) Accessible & Active eLearning Game Plan (Monica Maher). [For the 2018 proceedings, see ED590389.]

The field of education is undergoing fundamental change with the growing use of data. Fine-scale data collection at the item-response level is now possible. Xerox has developed a system that bridges the paper-to-digital divide by providing the well-established and easy-to-use paper interface to students, but digitizes the responses for scoring, validating, reporting, and managing data using a range of digital technologies. The Ignite™ system supports written responses, shading, connecting lines, multiple choice selections, and other question types. For some users, monitoring is at a very fine-grain level in both time and skill, while for others the data is used for more summative evaluations and strategic planning; one users' details may be another user's overview. All the reports presented in this document use the same basic atomic data elements and associated meta-data. The hierarchical nature of the organization of users requires that these atomic elements be combined in different ways for specialized visual representations, dependent upon the needs of the user. [For complete proceedings, see ED560503.]

Existing personalized learning systems (PLSs) have primarily focused on providing learning analytics using data from learners. In this paper, we extend the capability of current PLSs by incorporating data from instructors. We propose a latent factor model that analyzes instructors' preferences in explicitly "excluding" particular questions from learners' assignments in a particular subject domain. We formulate the problem of predicting instructors' question exclusion preferences as a matrix factorization problem, and incorporate expert-labeled Bloom's Taxonomy tags on each question as a factor in our statistical model to improve model interpretability. Experimental results on a real-world educational dataset demonstrate that the proposed model achieves superior prediction performance compared to several other baseline methods commonly used in recommender systems. Additionally, by explicitly incorporating Bloom's Taxonomy, the model provides meaningful interpretations that help understand why instructors exclude certain questions. Since instructor preference data contains their insights after years of teaching experience, our proposed model has the potential to further improve the question recommendations that PLSs make for learners. [For the full proceedings, see ED596512.]

In this paper, the authors attempted to evaluate design of leaning units with use of Learning Analytics technique on page flip information. Traditional formative assessment has been carried out by giving assignments and evaluating their results. However, the information that teacher can get from the evaluation is limited and coarse-grained. The authors set a research question that whether one can evaluate relation between learning objectives of learning units and the learners' actual activities in the units from page flip histories. The experimental result showed that the intensity of relation between the assignments and learning materials were different for each unit. Quantitatively, the correlation coefficients between "p-value of chi-square tests between range of page flip count and grade of assignment" and "proportion of number of questions in an assessment with reference page" was -0.889. With use of this relation, the authors attempted to evaluate design of units, and found out that the thing that may be critical to get high grade in an assignment of a certain unit was how many times they refer to their text book, though this assignment was designed for assessment of learners' skills to use or apply the learned knowledge. If teacher can get such suggestion as feedback, it is possible to utilize for formative assessment. [For full proceedings, see ED571332.]