Search

The growth of Artificial Intelligence (AI) chatbots has created a great deal of discussion in the education community. While many have gravitated towards the ability of these bots to make learning more interactive, others have grave concerns that student created essays, long used as a means of assessing the subject comprehension of students, may be at risk. The bot's ability to quickly create high quality papers, sometimes complete with reference material, has led to concern that these programs will make students too reliant on their ability and not develop the critical thinking skills necessary to succeed. The rise in these applications has led to the need for the development of detection programs that are able to read the students submitted work and return an accurate estimation of if the paper is human or computer created. These detection programs use natural language processing's (NLP) ideas of perplexity, or randomness of the text, and burstiness, or the tendency for certain words and phrases to appear together, plus sophisticated algorithms to compare the essays to preexisting literature to generate an accurate estimation on the likely author of the paper. The use of these systems has been found to be highly effective in reducing plagiarism among students, however concerns have been raised about the limitations of these systems. False positives, false negatives, and cross language identification are three areas of concern amongst faculty and have led to reduced usage of the detection engines. Despite the limitations however, these systems are a valuable tool for educational institutions to maintain academic integrity and ensure that students are submitting original work. [For the full proceedings, see ED656038.]

Predictive analytics is widely used in various domains, including education, to inform decision-making and improve outcomes. However, many predictive models are proprietary and inaccessible for evaluation or modification by researchers and practitioners, limiting their accountability and ethical design. Moreover, predictive models are often opaque and incomprehensible to the officials who use them, reducing their trust and utility. Furthermore, predictive models may introduce or exacerbate bias and inequity, as they have done in many sectors of society. Therefore, there is a need for transparent, interpretable, and fair predictive models that can be easily adopted and adapted by different stakeholders. In this paper, we propose a fair predictive model based on multivariate adaptive regression splines (MARS) that incorporates fairness measures in the learning process. MARS is a non-parametric regression model that performs feature selection, handles non-linear relationships, generates interpretable decision rules, and derives optimal splitting criteria on the variables. Specifically, we integrate fairness into the knot optimization algorithm and provide theoretical and empirical evidence of how it results in a fair knot placement. We apply our "fair"MARS model to real-world data and demonstrate its effectiveness in terms of accuracy and equity. Our paper contributes to the advancement of responsible and ethical predictive analytics for social good. [This paper was presented at an Association for the Advancement of Artificial Intelligence conference.]

Adaptive learning is an area of educational technology that consists in delivering personalized learning experiences to address the unique needs of each learner. An important subfield of adaptive learning is learning path personalization: it aims at designing systems that recommend sequences of educational activities to maximize students' learning outcomes. Many machine learning approaches have already demonstrated significant results in a variety of contexts related to learning path personalization. However, most of them were designed for very specific settings and are not very reusable. This is accentuated by the fact that they often rely on non-scalable models, which are unable to integrate new elements after being trained on a specific set of educational resources. In this paper, we introduce a flexible and scalable approach towards the problem of learning path personalization, which we formalize as a reinforcement learning problem. Our model is a sequential recommender system based on a graph neural network, which we evaluate on a population of simulated learners. Our results demonstrate that it can learn to make good recommendations in the small-data regime. [For the complete proceedings, see ED630829.]

Transactive discussion during collaborative learning is crucial for building on each other's reasoning and developing problem solving strategies. In a tabletop collaborative learning activity, student actions on the interface can drive their thinking and be used to ground discussions, thus affecting their problem-solving performance and learning. However, it is not clear how the interplay of actions and discussions, for instance, how students performing actions or pausing actions while discussing, is related to their learning. In this paper, we seek to understand how the transactivity of actions and discussions is associated with learning. Specifically, we ask what is the relationship between discussion and actions, and how it is different between those who learn (gainers) and those who do not (non-gainers). We present a combined differential sequence mining and content analysis approach to examine this relationship, which we applied on the data from 32 teams collaborating on a problem designed to help them learn concepts of minimum spanning trees. We found that discussion and action occur concurrently more frequently among gainers than non-gainers. Further we find that gainers tend to do more reflective actions along with discussion, such as looking at their previous solutions, than non-gainers. Finally, gainers discussion consists more of goal clarification, reflection on past solutions and agreement on future actions than non-gainers, who do not share their ideas and cannot agree on next steps. Thus this approach helps us identify how the interplay of actions and discussion could lead to learning, and the findings offer guidelines to teachers and instructional designers regarding indicators of productive collaborative learning, and when and how, they should intervene to improve learning. Concretely, the results suggest that teachers should support elaborative, reflective and planning discussions along with reflective actions. [For the complete proceedings, see ED630829.]

This paper presents a mobile application aimed at enhancing the financial literacy of college students by monitoring their spending patterns and promoting better decision-making. The application is developed using the agile methodology with Android Studio and Flutter as development tools and Firebase as a database. The app is divided into sub-applications, with the home page serving as the program's integration point, displaying a summary of the user's financial progress. The app generates valuable insights into the user's current and future financial success, utilizing data analytics and machine learning to provide detailed and summary insights into the user's financial progress. The machine-learning algorithm used in this app is linear regression, which predicts the user's income and expenses for the upcoming month based on their historical spending data. In addition, the app highlights deals and student discounts in the user's vicinity and links to financial articles that promote better financial planning and decision-making. By promoting responsible spending habits and providing valuable financial insights, this mobile application aims to help students become financially literate and make informed financial decisions for future. [For the full proceedings, see ED654100.]

This paper explores the correlation between mental calculation performance and the frequency of using written algorithms in mental calculation tasks. Mental calculation is a mathematical tool used in everyday life situations during and after our formal education. After presenting an overview of the professional literature on this topic, the paper will present calculation methods and show how represented they are in the Curriculum. For the empirical part of the research, a total of 233 Croatian students aged 10 to 22 years were tested and interviewed. The previously mentioned correlation was then analyzed. An overview of the interview results will be presented as well. It was found that school mathematics does not always contribute to the development and flexibility in using mental calculation strategies because of the student preference for acquired written algorithms. Definitely, recommendation is shifting the focus from written calculation and procedures to the mental, discussing the associated strategies and different concepts of number. In this way, formal education could contribute to what students really need later on, in both private and professional situations in which they may find themselves on a daily basis. [For the full proceedings, see ED630948.]

The use of technology as a facilitator in learning environments has become increasingly prevalent with the global pandemic caused by COVID-19. As such, computer-supported collaborative learning (CSCL) gains a wider adoption in contrast to traditional learning methods. At the same time, the need for automated tools capable of assessing and stimulating collaboration between participants has become more stringent, as human monitoring of the increasing volume of conversations becomes overwhelming. This paper introduces a method grounded in dialogism for evaluating students' involvement in chat conversations based on semantic chains computed using language models. These semantic chains reflect emergent voices from dialogism that span and interact throughout the conversation. Our integrated method uses contextual information captured by BERT transformer models to identify links in a chain that connects semantically related concepts from a voice uttered by one or more participants. Two types of visualizations were generated to depict the longitudinal propagation and the transversal inter-animation of voices within the conversation. In addition, a list of handcrafted features derived from the constructed chains and computed for each participant is introduced. Several machine learning algorithms were tested using these features to evaluate the extent to which semantic chains are predictive of student involvement in chat conversations. [This paper was published in: "Ludic, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education, Proceedings of the 6th International Conference on Smart Learning Ecosystems and Regional Development," edited by Ó. Mealha et al., Springer Nature Singapore Pte Ltd., 2022, pp. 67-78.]

The 31 papers in this collection represent approximately 35 percent of the manuscripts which were submitted for consideration to the Research and Theory Division of the Association for Educational Communications and Technology (AECT) for presentation at the 1980 AECT convention. All papers were subjected to a blind reviewing process and the ones finally selected represent some of the most current thinking in educational communications and technology. A listing of selected titles indicates the scope of the research paper presentations: "The Cognitive Effect in Bilingual Learners Given Different Pictorial Elaboration and Memory Tasks,""The Relationship of Communication Apprehension Level and Media Competency,""Implications of a Gestalt Approach to Research in Visual Communications,""Research on Pictures and Instructional Texts: Difficulties and Directions,""Imagery--A Return to Empirical Investigation,""A Meta-Analytic Study of Pictorial Stimulus Complexity,""Learner Interest and Instructional Design: A Conceptual Model,""The Organizing Function of Behavioral Objectives," and "Algorithmic Training for a Complex Perceptual-Motor Task." (LLS)

In this paper, we report the results of a 3-year, quasi-experimental study comparing students' engagement and deep learning of course materials between students who took an undergraduate engineering course that used a video game approach to a control group. The video game, EduTorcs, provided challenges in which students devised control algorithms that race virtual cars through a simulated race track. Theoretically, the study is rooted in Mayer and colleague's cognitive theory of multimedia learning. Engagement was measured with the Experience Sampling Method. Students taking the game-based course reported greater intrinsic motivation and engagement than students taking the course in the traditional way; and they performed significantly better on tests of complex course concepts designed to measure deep learning.

This paper demonstrates how to apply Machine Learning (ML) techniques to analyze student interaction data collected in an online mathematics game. We examined: (1) how different ML algorithms influenced the precision of middle-school students' (N = 359) performance prediction; and (2) what types of in-game features were associated with student math knowledge scores. The results indicated that the Random Forest algorithm showed the best performance in predicting posttest math knowledge scores among the seven algorithms employed. Out of 37 features included in the model, the validity of the students' first mathematical transformation was the most predictive of their math knowledge scores. Implications for game learning analytics and supporting students' algebraic learning are discussed based on the findings.

Numerous approaches have been introduced to automate the process of text summarization, but only few can be easily adapted to multiple languages. This paper introduces a multilingual text processing pipeline integrated in the open-source "ReaderBench" framework, which can be retrofit to cover more than 50 languages. While considering the extensibility of the approach and the problem of missing labeled data for training in various languages besides English, an unsupervised algorithm was preferred to perform extractive summarization (i.e., select the most representative sentences from the original document). Specifically, two different approaches relying on text cohesion were implemented:(1) a graph-based text representation derived from Cohesion Network Analysis that extends TextRank; and (2) a class of submodular set functions. Evaluations were performed on the DUC dataset and use as baseline the implementation of TextRank from Gensim. Our results using the submodular set functions outperform the baseline. In addition, two use cases on English and Romanian languages are presented, with corresponding graphical representations for the two methods. [This paper was published in: 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) Proceedings, 2020, pp. 161-168 (ISBN 978-1-7281-7628-4).]

Systematic classroom observation is often used in evaluating and enhancing the quality of classroom instruction. However, classroom observation can potentially suffer from human bias. In addition, the traditional classroom observation is too expensive for resource-constrained environments (e.g., Sub-Saharan Africa, South and Central Asia). A cost-effective automation of classroom observation could potentially enhance both quality and resolution of feedback to the teacher, and hence potentially result in enhancing quality of instruction. Audio-based automatic classroom observation using supervised deep learning techniques has yielded good results in limited contexts. However, one challenge when using supervised techniques is the high cost of collecting and labelling the classroom audio data. One solution for such data-starved scenarios is to use semi-supervised learning (SSL) which requires significantly lesser data and labels. This paper explores an audio-adaptation of the state-of-the-art SSL FixMatch algorithm to automate classroom observation. An adaptation of the FixMatch algorithm was proposed to automate the coding for the Stallings class observation system. The proposed system was trained on classroom audio data collected in the wild. The supervised approach had an F1-score of 0.83 on 100% labeled data. The proposed FixMatch adaptation achieved an impressive F1-score of 0.81 on 20% labeled data, 0.79 on 15% labeled data, 0.76 on 10% labeled data, and 0.72 using only 5% of labeled data. This suggests that algorithms like FixMatch that use consistency regularization and pseudo-labeling have a great potential for being used to automate classroom observation using a small labelled set of audio snippets.

Multiple choice questions are traditionally expensive to produce. Recent advances in large language models (LLMs) have led to fine-tuned LLMs that generate questions competitive with human-authored questions. However, the relative capabilities of ChatGPT-family models have not yet been established for this task. We present a carefully-controlled human evaluation of three conditions: a fine-tuned, augmented version of Macaw, instruction-tuned Bing Chat with zero-shot prompting, and human-authored questions from a college science textbook. Our results indicate that on six of seven measures tested, both LLM's performance was not significantly different from human performance. Analysis of LLM errors further suggests that Macaw and Bing Chat have different failure modes for this task: Macaw tends to repeat answer options whereas Bing Chat tends to not include the specified answer in the answer options. For Macaw, removing error items from analysis results in performance on par with humans for all metrics; for Bing Chat, removing error items improves performance but does not reach human-level performance. [This paper was published in the "CEUR Workshop Proceedings," 2023.]

Developing models and using mathematics are two key practices in internationally recognized science education standards, such as the Next Generation Science Standards (NGSS). However, students often struggle with these two intersecting practices, particularly when developing mathematical models about scientific phenomena. Formative performance-based assessments designed to elicit fine-grained data about students' competencies on these practices can be leveraged to develop embedded AI scaffolds to support students' learning. In this paper, we present the design and initial classroom test of virtual labs that automatically assess fine-grained sub-components of students' mathematical modeling competencies based on their actions within the learning environment. We describe how we leveraged underlying machine-learned and knowledge-engineered algorithms to trigger scaffolds, delivered proactively by a pedagogical agent, that address students' individual difficulties as they work. Results show that the students who received automated scaffolds for a given practice on their first virtual lab improved on that practice for the next virtual lab on the same science topic in a different scenario (a near-transfer task). These findings suggest that real-time automated scaffolds based on fine-grained assessment can foster students' mathematical modeling competencies related to the NGSS.

Spatial analytics receive increased attention in educational data mining. A critical issue in stop detection (i.e., the automatic extraction of timestamped and located stops in the movement of individuals) is a lack of validation of stop accuracy to represent phenomena of interest. Next to a radius that an actor does not exceed for a certain duration to establish a stop, this study presents a reproducible procedure to optimize a range parameter for K-12 classrooms where students sitting within a certain vicinity of an inferred stop are tagged as being visited. This extension is motivated by adapting parameters to infer teacher visits (i.e., on-task and off-task conversations between the teacher and one or more students) in an intelligent tutoring system classroom with a dense layout. We evaluate the accuracy of our algorithm and highlight a tradeoff between precision and recall in teacher visit detection, which favors recall. We recommend that future research adjust their parameter search based on stop detection precision thresholds. This adjustment led to better cross-validation accuracy than maximizing parameters for an average of precision and recall (F1 = 0.18 compared to 0.09). As stop sample size shrinks with higher precision cutoffs, thresholds can be informed by ensuring sufficient statistical power in offline analyses. We share avenues for future research to refine our procedure further. Detecting teacher visits may benefit from additional spatial features (e.g., teacher movement trajectory) and can facilitate studying the interplay of teacher behavior and student learning. [For the complete proceedings, see ED630829.]

Dynamic Item Response Models extend the standard Item Response Theory (IRT) to capture temporal dynamics in learner ability. While these models have the potential to allow instructional systems to actively monitor the evolution of learner proficiency in real time, existing dynamic item response models rely on expensive inference algorithms that scale poorly to massive datasets. In this work, we propose Variational Temporal IRT (VTIRT) for fast and accurate inference of dynamic learner proficiency. VTIRT offers orders of magnitude speedup in inference runtime while still providing accurate inference. Moreover, the proposed algorithm is intrinsically interpretable by virtue of its modular design. When applied to 9 real student datasets, VTIRT consistently yields improvements in predicting future learner performance over other learner proficiency models. [For the complete proceedings, see ED630829.]

Within the last decade, different educational data mining techniques, particularly quantitative methods such as clustering, and regression analysis are widely used to analyze the data from educational games. In this research, we implemented a quantitative data mining technique (clustering) to further investigate students' feedback. Students played educational games within a week on the educational games platform, Legends of Learning and after a week, we asked them to fulfill the feedback survey about their feelings on the use of this platform. To analyze the collected data from students, firstly, we prepared clusters and selected one prototype student closest to the centroid of each cluster to interview. Interviews were held to explain the clusters more and due to time and resource limitations, we were unable to interview all (N=60) students, thus only the most representative students were interviewed. In addition to the students, we conducted an interview with the teacher as well to get her detailed feedback and observations on the usage of educational games. We also asked students to take an exam before and after the research to see the impact of games on their grades. Our results depict that though educational games can increase students' motivation, they may negatively impact some students' grades. And even though playing games made students feel interested and fun, they would not like to play them on a daily basis. Hence, using educational games for a certain duration such as subject revision weeks may positively influence students' grades and motivation. [For the complete proceedings, see ED630829.]

Predictive student models are increasingly used in learning environments due to their ability to enhance educational outcomes and support stakeholders in making informed decisions. However, predictive models can be biased and produce unfair outcomes, leading to potential discrimination against some students and possible harmful long-term implications. This has prompted research on fairness metrics meant to capture and quantify such biases. Nonetheless, so far, existing fairness metrics used in education are predictive performance-oriented, focusing on assessing biased outcomes across groups of students, without considering the behaviors of the models nor the severity of the biases in the outcomes. Therefore, we propose a novel metric, the Model Absolute Density Distance (MADD), to analyze models' discriminatory behaviors independently from their predictive performance. We also provide a complementary visualization-based analysis to enable fine-grained human assessment of how the models discriminate between groups of students. We evaluate our approach on the common task of predicting student success in online courses, using several common predictive classification models on an open educational dataset. We also compare our metric to the only predictive performance-oriented fairness metric developed in education, ABROCA. Results on this dataset show that: (1) fair predictive performance does not guarantee fair models' behaviors and thus fair outcomes; (2) there is no direct relationship between data bias and predictive performance bias nor discriminatory behaviors bias; and (3) trained on the same data, models exhibit different discriminatory behaviors, according to different sensitive features too. We thus recommend using the MADD on models that show satisfying predictive performance, to gain a finer-grained understanding on how they behave and regarding who and to refine models selection and their usage. Altogether, this work contributes to advancing the research on fair student models in education. Source code and data are in open access at https://github.com/melinaverger/MADD. [For the complete proceedings, see ED630829.]

The Indian Institute of Science is proud to host the fully in-person sixteenth iteration of the International Conference on Educational Data Mining (EDM) during July 11-14, 2023. EDM is the annual flagship conference of the International Educational Data Mining Society. The theme of this year's conference is "Educational data mining for amplifying human potential." Not all students or seekers of knowledge receive the education necessary to help them realize their full potential, be it due to a lack of resources or lack of access to high quality teaching. The dearth in high-quality educational content, teaching aids, and methodologies, and non-availability of objective feedback on how they could become better teachers, deprive our teachers from achieving their full potential. The administrators and policy makers lack tools for making optimal decisions such as optimal class sizes, class composition, and course sequencing. All these handicap the nations, particularly the economically emergent ones, who recognize the centrality of education for their growth. EDM-2023 has striven to focus on concepts, principles, and techniques mined from educational data for amplifying the potential of all the stakeholders in the education system. The spotlights of EDM-2023 include: (1) Five keynote talks by outstanding researchers of eminence; (2) A plenary Test of Time award talk and a Banquet talk; (3) Five tutorials (foundational as well as advanced); (4) Four thought provoking panels on contemporary themes; (5) Peer reviewed technical paper and poster presentations; (6) Doctoral students consortium; and (7) An enchanting cultural programme. [Individual papers are indexed in ERIC.]

The Jeong (2020) study found that greater use of backward and depth-first processing was associated with higher scores on students' argument maps and that analysis of only the first five nodes students placed in their maps predicted map scores. This study utilized the jMAP tool and algorithms developed in the Jeong (2020) study to determine if the same processes produce higher-quality causal maps. This study analyzed the first five nodes that students (n = 37) placed in their causal maps to reveal that: 1) use of backward, forward, breadth-first, and depth-first processing produced maps of similar quality; and 2) backward processing had three times more impact on maps scores than depth-first processing to suggest that linking events into chains using backward chaining is one approach to constructing higher quality causal maps. These findings are compared with prior research findings and discussed in terms of noted differences in the task demands of constructing argument versus causal maps to gain insights into why, how, and when specific processes/strategies can be applied to create higher-quality causal maps and argument maps. These insights provide guidance on ways to develop diagramming and analytic tools that automate, analyze, and provide real-time support to improve the quality of students' maps, learning, understanding, and problem-solving skills. [For the full proceedings, see ED636095.]

Educational Robotics is increasingly used in elementary-school classrooms to develop students' algorithmic thinking and programming skills. However, most research appears descriptive and lacks experimental evidence on the effects of teaching interventions using robotics to develop algorithmic thinking. Using the robots Dash and Dot, this study examined algorithmic thinking development in groups of children aged 6, 9, and 12. The results showed a statistically significant main effect between the age of students and algorithmic thinking skills and a statistically significant main effect between intervention and algorithmic thinking. In conclusion, the findings underscore the necessity of providing learners with structured, scaffolded activities tailored to their age to effectively nurture algorithmic thinking skills when engaging in Dash and Dot activities. [For the full proceedings, see ED636095.]

For efficiency, Computerized Adaptive Test (CAT) algorithm selects items with the maximum information, typically with a 50% probability of being answered correctly. However, examinees may not be satisfied if they only correctly answer 50% of the items. Researchers discovered that changing the item selection algorithms to choose easier items (i.e., success probability > 50%), albeit not optimum from a measurement efficiency standpoint, would provide a better experience. The current study aims to investigate the impact of changing the success probability on measurement efficiency. A Monte-Carlo simulation was performed on the Open Matrices Item Bank and simulated item bank. A total of 1500 examinees were generated. We modified the item selection algorithm with the expected success probability of 60%, 70%, and 80%. Each examinee was assigned to five item selection methods: maximum-information, random, p=0.6, p=0.7, and p=0.8. The results indicated that traditional CAT was 60-70% shorter than random item selection. Altering the success probability did not affect the estimation of the examinee's ability. Increasing the probability of success in CAT increased the number of items required to achieve specified levels of precision. Practical considerations on how to maximize the trade-off between examinees' experiences and measurement efficiency are mentioned in the discussion. [For the full proceedings, see ED654100.]

This study offers an analysis and comparison of search results from Google concerning the topic of Artificial Intelligence (AI) in two geographically and politically different contexts: the United States and Italy. As new AI systems, tools, and solutions are developed and implemented in each sector of human life on a global scale, certain imaginaries of AI are emerging. These imaginaries constitute the ground for the public understanding, support, and disapproval of certain AI technologies and regulations. As citizens turn into users, Google remains the dominant gatekeeper of information, thus becoming an influential actor in sharping AI imaginaries. The following analysis is a response to the criticism of Google's search results, considering Google as an essential producer and certifier of AI imaginaries for general public. The comparison of search queries conducted in this analysis shows that the sources which Google presents in its search results add to different types of AI imaginaries, consequently influencing public opinion in different, often asymmetrical, ways. [For the full proceedings, see ED654100.]

This work proposes Dynamic Linear Epsilon-Greedy, a novel contextual multi-armed bandit algorithm that can adaptively assign personalized content to users while enabling unbiased statistical analysis. Traditional A/B testing and reinforcement learning approaches have trade-offs between empirical investigation and maximal impact on users. Our algorithm seeks to balance these objectives, allowing platforms to personalize content effectively while still gathering valuable data. Dynamic Linear Epsilon-Greedy was evaluated via simulation and an empirical study in the ASSISTments online learning platform. In simulation, Dynamic Linear Epsilon-Greedy performed comparably to existing algorithms and in ASSISTments, slightly increased students' learning compared to A/B testing. Data collected from its recommendations allowed for the identification of qualitative interactions, which showed high and low knowledge students benefited from different content. Dynamic Linear Epsilon-Greedy holds promise as a method to balance personalization with unbiased statistical analysis. All the data collected during the simulation and empirical study are publicly available at https://osf.io/zuwf7/. [This paper was published in: "UMAP '23: Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization," June 26-29, 2023.]

Computational thinking (CT) is a fundamental ability for learners in today's society. Although CT assessments and interventions have been studied widely, little is known about CT predictions. This study predicted students' CT achievement in the ICILS 2018 using five machine learning models. These models were trained on the data from five European countries and then tested on the Korean and the Danish sample, respectively. Results indicate that the models trained on the individualistic-European data were generalizable to the individualistic European country, Denmark, but not to the collectivistic Asian country, Korea. This study fills a void in the CT literature and highlights the importance of considering the contextual relevance of data sources when making algorithmic predictions.

Recent research suggests that neural networks, algorithms designed to reflect the human brain's behavior to recognize patterns, can be used to develop data dashboards that provide teachers with more specific and frequent feedback to improve their instruction (Jacobs et al., 2022). This qualitative case study examines six teachers' perceptions of the usability, utility, and accuracy of an interactive automated dashboard developed to provide teachers with detailed personalized feedback on their use of activity type (whole group, small group, independent work, and transition time), presented in a quantifiable and non-evaluative manner. Overall, our findings highlight the feasibility of automated tools to support meaningful feedback opportunities for teachers. Specifically, teachers found feedback on transition time during lessons to be particularly helpful.

Text summarization is an effective reading comprehension strategy. However, summary evaluation is complex and must account for various factors including the summary and the reference text. This study examines a corpus of approximately 3,000 summaries based on 87 reference texts, with each summary being manually scored on a 4-point Likert scale. Machine learning models leveraging Natural Language Processing (NLP) techniques were trained to predict the extent to which summaries capture the main idea of the target text. The NLP models combined both domain and language independent textual complexity indices from the ReaderBench framework, as well as state-of-the-art language models and deep learning architectures to provide semantic contextualization. The models achieve low errors -- normalized MAE ranging from 0.13-0.17 with corresponding R2 values of up to 0.46. Our approach consistently outperforms baselines that use TF-IDF vectors and linear models, as well as Transfomer-based regression using BERT. These results indicate that NLP algorithms that combine linguistic and semantic indices are accurate and robust, while ensuring generalizability to a wide array of topics. [This paper was published in: A. I. Cristea and C. Troussas (Eds.), "ITS 2021: Intelligent Tutoring Systems proceedings," pp. 321-332, 2021. Springer, Cham Switzerland.]

The six papers that comprise this document deal with topics related to the National Institute for Occupational Safety and Health (NIOSH) and/or its online document retrieval system, NIOSHTIC. Specific subjects include: (1) a method to catalog multiple sources of NIOSH publications for dissemination; (2) an input rejection algorithm used to provide quality control for NIOSHTIC input; (3) document selection criteria for NIOSHTIC; (4) the development and implementation of a publications dissemination program for NIOSH; (5) a conceptual model for the screening and indexing of documents for a multidisciplinary information system; and (6) an information user needs survey--a decision guide for resource allocation. Among the materials provided with the papers are a NIOSHTIC selection matrix, a relevancy matrix for NIOSH current research system, a quality control defect weighting scheme, and tables presenting the information services of the Clearinghouse for Occupational Safety and Health Information and a summary of areas to be included in an information user needs questionnaire. (JD)

Research into "gaming the system" behavior in intelligent tutoring systems (ITS) has been around for almost two decades, and detection has been developed for many ITSs. Machine learning models can detect this behavior in both real-time and in historical data. However, intelligent tutoring system designs often change over time, in terms of the design of the student interface, assessment models, and data collection log schemas. Can gaming detectors still be trusted, a decade or more after they are developed? In this research, we evaluate the robustness/degradation of gaming detectors when trained on older data logs and evaluated on current data logs. We demonstrate that some machine learning models developed using past data are still able to predict gaming behavior from student data collected 16 years later, but that there is considerable variance in how well different algorithms perform over time. We demonstrate that a classic decision tree algorithm maintained its performance while more contemporary algorithms struggled to transfer to new data, even though they exhibited better performance on unseen students in both New and Old data sets by themselves. Examining the feature importance values provides some explanation for the differences in performance between models, and offers some insight into how we might safeguard against detector rot over time. [For the full proceedings, see ED623995.]

Online educational technologies facilitate pedagogical experimentation, but typical experimental designs assign a fixed proportion of students to each condition, even if early results suggest some are ineffective. Experimental designs using multi-armed bandit (MAB) algorithms vary the probability of condition assignment for a new student based on prior results, placing more students in more effective conditions. While stochastic MAB algorithms have been used for educational experiments, they collect data that decreases power and increases false positive rates [22]. Instead, we propose using adversarial MAB algorithms, which are less exploitative and thus may exhibit more robustness. Through simulations involving data from 20+ educational experiments [29], we show data collected using adversarial MAB algorithms does not have the statistical downsides of that from stochastic MAB algorithms. Further, we explore how differences in condition variability (e.g., performance gaps between students being narrowed by an intervention) impact MAB versus uniform experimental design. Data from stochastic MAB algorithms systematically reduce power when the better arm is less variable, while increasing it when the better arm is more variable; data from the adversarial MAB algorithms results in the same statistical power as uniform assignment. Overall, these results demonstrate that adversarial MAB algorithms are a viable "off-the-shelf" solution for researchers who want to preserve the statistical power of standard experimental designs while also benefiting student participants. [For the full proceedings, see ED623995.]

This study investigates how utilizing student-invented strategies in the classroom can inform teachers' pedagogical content knowledge. Two elementary school teachers participated in professional development discussing the benefits of invented strategies. Data was then gathered as the participants implemented this practice in their classrooms. Data was analyzed qualitatively to show the ways in which invented strategies can be useful in a teacher's development of their pedagogical content knowledge, including their Knowledge of Content and Students, Knowledge of Content and Teaching, as well as Knowledge of Content and Curriculum. [For the complete proceedings, see ED630210.]

The COVID-19 epidemic had caused one of the most significant disruptions to the global education system. Many educational institutions faced sudden pressure to switch from face-to-face to online delivery of courses. The conventional classes are no longer the primary means of delivery; instead, online education and resources have become the prominent approach. With the increasing demand for supplementary course materials to fulfill the needs of each area of study, students began to use search engines and online resources that contain discussions, practical demonstrations, and tutorial videos to aid students in their studies and course work. This study addresses the underlying challenges of retrieving relevant online educational materials by introducing an intelligent agent for semantic data mining. It works as middleware infrastructure that allow context-aware data processing and mining. YouTube was used to assess the consistency of the proposed model since it returns a large number of results in its search pool. The results showed that using the extraction of topics method, the similarities scores with the proposed model provided favorable results. Furthermore, an improvement in video ranking and sorting was realized. According to the findings, using this method provided users with a more productive and reliable study experience. [For the full proceedings, see ED639633.]

Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students' practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self-explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa = 0.43). [This paper was published in: "Artificial Intelligence in Education: 19th International Conference, AIED 2018, London, UK, June 27-30, 2018, Proceedings, Part I," (Lecture Notes in Computer Science; Vol. 10947), (Lecture Notes in Artificial Intelligence), pp. 409-419.]

Detecting wheel-spinning behaviors of students who interact with an Intelligent Tutoring System (ITS) is important for generating pertinent and effective feedback and developing more enriching learning experiences. This analysis compares decision tree and bagged tree models of student productive persistence (i.e., mastering a skill) using the ASSISTment 2009-2010 dataset for n = 4,217 middle-school students in the United States to predict whether a student is wheel-spinning. Although both models yielded high predictive accuracy, bagged trees significantly outperformed decision trees. Results show that (1) a tree-based model is effective at accurately predicting wheel-spinning and (2) students are taking more than the average amount of attempts to master a skill.