Search

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit with panel data. Two challenges arise with higher frequency data (e.g., monthly versus yearly): (1) achieving excellent pre-treatment fit is typically more challenging; and (2) overfitting to noise is more likely. Aggregating data over time can mitigate these problems but can also destroy important signal. In this paper, we bound the bias for SCM with disaggregated and aggregated outcomes and give conditions under which aggregating tightens the bounds. We then propose finding weights that balance both disaggregated and aggregated series. [This paper was published in: "AEA Papers and Proceedings," Volume 114, 2024.]

Despite increased efforts to assess the adoption rates of open science and robustness of reproducibility in sub-disciplines of education technology, there is a lack of understanding of why some research is not reproducible. Prior work has taken the first step toward assessing reproducibility of research, but has assumed certain constraints which hinder its discovery. Thus, the purpose of this study was to replicate previous work on papers within the proceedings of the "International Conference on Educational Data Mining" to accurately report on which papers are reproducible and why. Specifically, we examined 208 papers, attempted to reproduce them, documented reasons for reproducibility failures, and asked authors to provide additional information needed to reproduce their study. Our results showed that out of 12 papers that were potentially reproducible, only one successfully reproduced all analyses, and another two reproduced most of the analyses. The most common failure for reproducibility was failure to mention libraries needed, followed by non-seeded randomness. [For the complete proceedings, see ED630829. Additional funding for this paper was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN).]

The profound advancements in technology have rendered novel forms of data and data visualisation increasingly accessible to individuals within society, thereby influencing daily decision-making processes. To address this change, this study sets out to review recent research on data-driven inquiries at the K-12 level from two perspectives: innovative data visualisation and non-traditional data sources. Our findings indicate that transnumeration of multiple data representations, along with data moves throughout the process of data visualisation, can potentially enhance the development of visual reasoning and data modelling skills.

The General Data Protection Regulation (GDPR) in the European Union contains directions on how user data may be collected, stored, and when it must be deleted. As similar legislation is developed around the globe, there is the potential for repercussions across multiple fields of research, including educational data mining (EDM). Over the past two decades, the EDM community has taken consistent steps to protect learner privacy within our research, whilst pursuing goals that will benefit their learning. However, recent privacy legislation may cause our practices to need to change. The right to be forgotten states that users have the right to request that all their data (including deidentified data generated by them) be removed. In this paper, we discuss the potential challenges of this legislation for EDM research, including impacts on Open Science practices, data modeling, and data sharing. We also consider changes to EDM best practices that may aid compliance with this new legislation. [For the complete proceedings, see ED630829.]

Students who take an online course, such as a MOOC, use the course's discussion forum to ask questions or reach out to instructors when encountering an issue. However, reading and responding to students' questions is difficult to scale because of the time needed to consider each message. As a result, critical issues may be left unresolved, and students may lose the motivation to continue in the course. To help address this problem, we build predictive models that automatically determine the urgency of each forum post, so that these posts can be brought to instructors' attention. This paper goes beyond previous work by predicting not just a binary decision cut-off but a post's level of urgency on a 7-point scale. First, we train and cross-validate several models on an original data set of 3,503 posts from MOOCs at University of Pennsylvania. Second, to determine the generalizability of our models, we test their performance on a separate, previously published data set of 29,604 posts from MOOCs at Stanford University. While the previous work on post urgency used only one data set, we evaluated the prediction across different data sets and courses. The best-performing model was a support vector regressor trained on the Universal Sentence Encoder embeddings of the posts, achieving an RMSE of 1.1 on the training set and 1.4 on the test set. Understanding the urgency of forum posts enables instructors to focus their time more effectively and, as a result, better support student learning. [For the complete proceedings, see ED630829.]

This paper presents a tool for creating student models in logistic regression. Creating student models has typically been done by expert selection of the appropriate terms, beginning with models as simple as IRT or AFM but more recently with highly complex models like BestLR. While alternative methods exist to select the appropriate predictors for the regression-based models (e.g., stepwise selection or LASSO), we are unaware of their application to student modeling. Such automatic methods of model creation offer the possibility of better student models with either reduced complexity or better fit, in addition to relieving experts from the burden of searching for better models by hand with possible human error. Our new functions are now part of the preexisting R package LKT. We explain our search methods with two datasets demonstrating the advantages of using the tool with stepwise regression and regularization (LASSO) methods to aid in feature selection. For the stepwise method using BIC, the models are simpler (due to the BIC penalty for parameters) than alternatives like BestLR with little lack of fit. For the LASSO method, the models can be made simpler due to the fitting procedure involving a regularization parameter that penalizes large absolute coefficient values. However, LASSO also offers the possibility of highly complex models with exceptional fit. [For the complete proceedings, see ED630829.]

Knowledge components (KCs) have many applications. In computing education, knowing the demonstration of specific KCs has been challenging. This paper introduces an entirely data-driven approach for: (1) discovering KCs; and (2) demonstrating KCs, using students' actual code submissions. Our system is based on two expected properties of KCs: (1) generate learning curves following the power law of practice; and (2) are predictive of response correctness. We train a neural architecture (named KC-Finder) that classifies the correctness of student code submissions and captures problem-KC relationships. Our evaluation on data from 351 students in an introductory Java course shows that the learned KCs can generate reasonable learning curves and predict code submission correctness. At the same time, some KCs can be interpreted to identify programming skills. We compare the learning curves described by our model to four baselines, showing that: (1) identifying KCs with naive methods is a difficult task; and (2) our learning curves exhibit a substantially better curve fit. Our work represents a first step in solving the data-driven KC discovery problem in computing education. [For the complete proceedings, see ED630829.]

This paper reports on students' experiences of describing and representing variation in hypothetical data. Fifty-six students (8-9 years-old) experienced collecting and working with quantitative data for two years as part of a STEM education project. The task described here was an end-of-year survey question, with three parts about a hypothetical context for surveying students in two different Australian cities: recording the data, describing the potential variation in the data, and creating a representation of what the data might look like when only a descriptive account of the context and variables were provided. The data analysis framework utilised provides a means of determining students' readiness for further development of statistical ideas.

Data literacy is the ability to collect, manage, evaluate, and apply data, in a critical manner. It is a relatively new field of study, dating only from the 2010s. It includes the skills necessary to discover and access data, manipulate data, evaluate data quality, conduct analysis using data, interpret results of analyses, and understand the ethics of using data. This paper considers data literacy education across three frameworks: the competency model defining data literacy, the assessment of data literacy competencies, and methods for the development of data literacy in an organization. These principles are applied to a discussion of the development of an open online course supporting the development of data literacy in the form of a corresponding data structure encompassing the three frameworks identified in the literature. [For the full proceedings, see ED636095.]

This paper presents a mobile application aimed at enhancing the financial literacy of college students by monitoring their spending patterns and promoting better decision-making. The application is developed using the agile methodology with Android Studio and Flutter as development tools and Firebase as a database. The app is divided into sub-applications, with the home page serving as the program's integration point, displaying a summary of the user's financial progress. The app generates valuable insights into the user's current and future financial success, utilizing data analytics and machine learning to provide detailed and summary insights into the user's financial progress. The machine-learning algorithm used in this app is linear regression, which predicts the user's income and expenses for the upcoming month based on their historical spending data. In addition, the app highlights deals and student discounts in the user's vicinity and links to financial articles that promote better financial planning and decision-making. By promoting responsible spending habits and providing valuable financial insights, this mobile application aims to help students become financially literate and make informed financial decisions for future. [For the full proceedings, see ED654100.]

In this study, we held a 12-hour workshop focused on training the Data-Driven Learning (DDL) approach for in-service Chinese teachers aiming to implement this pedagogy in Mandarin Chinese classrooms in the future. We analyzed data from a postworkshop questionnaire to understand how the individual-level traits of Chinese teachers (such as their experience, ability, and motivations) relate to their integration of corpus-based methods. The questionnaire results indicate that teacher confidence in using the corpus is critical for the future application of DDL in teaching Chinese as a second language. Teachers who use corpus-based pedagogies are more positive at designing and guiding DDL activities and are more willing to practice DDL in language classes. [For the complete volume, "Intelligent CALL, Granular Systems and Learner Data: Short Papers from EUROCALL 2022 (30th, Reykjavik, Iceland, August 17-19, 2022)," see ED624779.]

This paper explores the effect of using multitask learning for abstractive summarization in the context of small training corpora. In particular, we incorporate four different tasks (extractive summarization, language modeling, concept detection, and paraphrase detection) both individually and in combination, with the goal of enhancing the target task of abstraction summarization via multitask learning. We show that for many task combinations, a model trained in a multitask setting outperforms a model trained only for abstractive summarization, with no additional summarization data introduced. Additionally, we do a comprehensive search and find that certain tasks (e.g. paraphrase detection) consistently benefit abstractive summarization, not only when combined with other tasks but also when using different architectures and training corpora. [This paper was published in: "Findings of the Association for Computational Linguistics: EMNLP 2021," Association for Computational Linguistics, 2021, pp. 1652-1661.]

This paper explores three simple data manipulation techniques (synthesis, augmentation, curriculum) for improving abstractive summarization models without the need for any additional data. We introduce a method of data synthesis with paraphrasing, a data augmentation technique with sample mixing, and curriculum learning with two new difficulty metrics based on specificity and abstractiveness. We conduct experiments to show that these three techniques can help improve abstractive summarization across two summarization models and two different small datasets. Furthermore, we show that these techniques can improve performance when applied in isolation and when combined. [This paper was published in: "Findings of the Association for Computational Linguistics: EMNLP 2021," Association for Computational Linguistics, 2021, pp. 2043-2052.]

Recommender systems in educational contexts have proven effective to identify learning resources that fit the interests and needs of learners. Their usage has been of special interest in online self-learning scenarios to increase student retention and improve the learning experience. In current recommendation techniques, and in particular, in collaborative filtering recommender systems, the quality of the recommendation is largely based on the explicit or implicit information obtained about the learners. On free massive online learning platforms, however, the information available about learners may be limited and based mostly on logs from website analytics tools such as Google Analytics. In this paper, we address the challenge of recommending meaningful content with limited information from users by using rating estimation strategies from a log system. Our approach posits strategies to mine logs and generates effective ratings through the counting and temporal analysis of sessions. We evaluate different rating penalty strategies and compare the use of per-user and global metrics for rating estimation. The results show that using the average number of lessons viewed per-user is better than using global metrics with a p-value under 0.01 for 4 of our 5 hypotheses, showing statistical significance. Additionally, the results show that functions that penalize the rating to a lesser degree behave better and lead to a better recommendation. [For the full proceedings, see ED623995.]

This work explores how population-based engagement prediction can address cold-start at scale in large learning resource collections. The paper introduces: (1) VLE, a novel dataset that consists of content and video based features extracted from publicly available scientific video lectures coupled with implicit and explicit signals related to learner engagement; (2) two standard tasks related to predicting and ranking context-agnostic engagement in video lectures with preliminary baselines; and (3) a set of experiments that validate the usefulness of the proposed dataset. Our experimental results indicate that the newly proposed VLE dataset leads to building context-agnostic engagement prediction models that are significantly performant than ones based on previous datasets, mainly attributing to the increase of training examples. VLE dataset's suitability in building models towards Computer Science/ Artificial Intelligence education focused on e-learning/MOOC use-cases is also evidenced. Further experiments in combining the built model with a personalising algorithm show promising improvements in addressing the cold-start problem encountered in educational recommenders. This is the largest and most diverse publicly available dataset to our knowledge that deals with learner engagement prediction tasks. The dataset, helper tools, descriptive statistics and example code snippets are available publicly. [For the full proceedings, see ED623995.]

A comparative longitudinal data analysis between two online non-thesis master's programs--natural resource management and environmental science--in a college of natural resources to determine the relationship between student characteristics and disenrollment risks. Risks varied between the two programs, with significance found to increase the risk of disenrollment due to cumulative GPA, gender, time between degrees, and the number of terms not enrolled. [For the full proceedings, see ED648717.]

Computer-mediated social learning contexts have become increasingly popular over the last few years; yet existing models of students' cognitive-affective states have been slower to adopt dyadic interaction data for predictions. Here, we explore the possibility of capitalizing on the inherently social component of collaborative learning by using keystroke log data to make predictions across conversational partners (i.e., using person A's data to make prediction about if person B is mind wandering). Log files from 33 dyads (total N = 66) were used to examine: (a) how mind wandering (defined here as task-unrelated thought) during computer-mediated conversations is related to critical outcomes of the conversation (trust, likability, agreement); (b) if task-unrelated thought can be predicted by the keystrokes of one's partner; and (c) how much data is needed to make predictions by testing various window-sizes of data preceding task-unrelated thought reports. Results indicated a negative relationship between task-unrelated thought and perceptions of the conversation, suggesting that attention is an important factor during computer mediated chat conversations. Finally, in line with our hypothesis, results from mixed effects models showed that one's level of task-unrelated thought was predicted by the keystroke patterns of their conversational partner, but only using small window sizes (5s worth of data). [For the complete proceedings, see ED630829.]

Problem decomposition into sub-problems or subgoals and recomposition of the solutions to the subgoals into one complete solution is a common strategy to reduce difficulties in structured problem solving. In this study, we use a datadriven graph-mining-based method to decompose historical student solutions of logic-proof problems into Chunks. We design a new problem type where we present these chunks in a Parsons Problem fashion and asked students to reconstruct the complete solution from the chunks. We incorporated these problems within an intelligent logic tutor and called them Chunky Parsons Problems (CPP). These problems demonstrate the process of problem decomposition to students and require them to pay attention to the decomposed solution while they reconstruct the complete solution. The aim of introducing CPP was to improve students' problem-solving skills and performance by improving their decomposition-recomposition skills without significantly increasing training difficulty. Our analysis showed that CPPs could be as easy as Worked Examples (WE). And, students who received CPP with simple explanations attached to the chunks had marginally higher scores than those who received CPPs without explanation or did not receive them. Also, the normalized learning gain of these students shifted more towards the positive side than other students. Finally, as we looked into their proof-construction traces in posttest problems, we observed them to form identifiable chunks aligned with those found in historical solutions with higher efficiency. [For the complete proceedings, see ED630829.]

Due to the precautionary measures during the COVID-19 pandemic many universities offered unproctored take-home exams. We propose methods to detect potential collusion between students and apply our approach on event log data from take-home exams during the pandemic. We find groups of students with suspiciously similar exams. In addition, we compare our findings to a proctored comparison group. By this, we establish a rule of thumb for evaluating which cases are "outstandingly similar", i.e., suspicious cases. [For the complete proceedings, see ED630829.]

Within the last decade, different educational data mining techniques, particularly quantitative methods such as clustering, and regression analysis are widely used to analyze the data from educational games. In this research, we implemented a quantitative data mining technique (clustering) to further investigate students' feedback. Students played educational games within a week on the educational games platform, Legends of Learning and after a week, we asked them to fulfill the feedback survey about their feelings on the use of this platform. To analyze the collected data from students, firstly, we prepared clusters and selected one prototype student closest to the centroid of each cluster to interview. Interviews were held to explain the clusters more and due to time and resource limitations, we were unable to interview all (N=60) students, thus only the most representative students were interviewed. In addition to the students, we conducted an interview with the teacher as well to get her detailed feedback and observations on the usage of educational games. We also asked students to take an exam before and after the research to see the impact of games on their grades. Our results depict that though educational games can increase students' motivation, they may negatively impact some students' grades. And even though playing games made students feel interested and fun, they would not like to play them on a daily basis. Hence, using educational games for a certain duration such as subject revision weeks may positively influence students' grades and motivation. [For the complete proceedings, see ED630829.]

The Indian Institute of Science is proud to host the fully in-person sixteenth iteration of the International Conference on Educational Data Mining (EDM) during July 11-14, 2023. EDM is the annual flagship conference of the International Educational Data Mining Society. The theme of this year's conference is "Educational data mining for amplifying human potential." Not all students or seekers of knowledge receive the education necessary to help them realize their full potential, be it due to a lack of resources or lack of access to high quality teaching. The dearth in high-quality educational content, teaching aids, and methodologies, and non-availability of objective feedback on how they could become better teachers, deprive our teachers from achieving their full potential. The administrators and policy makers lack tools for making optimal decisions such as optimal class sizes, class composition, and course sequencing. All these handicap the nations, particularly the economically emergent ones, who recognize the centrality of education for their growth. EDM-2023 has striven to focus on concepts, principles, and techniques mined from educational data for amplifying the potential of all the stakeholders in the education system. The spotlights of EDM-2023 include: (1) Five keynote talks by outstanding researchers of eminence; (2) A plenary Test of Time award talk and a Banquet talk; (3) Five tutorials (foundational as well as advanced); (4) Four thought provoking panels on contemporary themes; (5) Peer reviewed technical paper and poster presentations; (6) Doctoral students consortium; and (7) An enchanting cultural programme. [Individual papers are indexed in ERIC.]

Assessing students' performance in online learning could be executed not only by the traditional forms of summative assessments such as using essays, assignments, and a final exam, etc. but also by more formative assessment approaches such as interaction activities, forum posts, etc. However, it is difficult for teachers to monitor and assess students' learning activities using the log data. To provide teachers with a more comprehensive view of students' distinct learning behaviour patterns, and to supply personalized interventions and support to meet the specific needs of each learning group, this study focuses on how to automatically acquire learning logs from Manaba, a Japanese commercial LMS, and how to cluster students' learning activities using the k-means algorithm. Firstly, we developed a program using Python to scrape students' learning activity log information from the Manaba web pages. We collected 56446 lines of clickstreams log data from 121 students in two computer literacy hybrid classes in the fall semester of 2022 (2022/9-2023/1). Secondly, we convert the raw logs into a structured dataset with 33 features which represent each student's learning activities. Then we extract and select 15 features representing three perspectives: raw activity, time on task, and learning frequency. Thirdly, we grouped students' learning activity patterns with the three perspectives into 5 clusters by the k-means clustering algorithm. As a result, this study identified five distinct learning activity patterns depending on how much, how long and how often the students learned online. For example, cluster 1 seldom learned but spent time on learning whom we considered the disengaged or struggling students, and cluster 5 had more learning activities with little time on each activity whom we considered the well-self-regulated students. The results of this study contribute to how to monitor students' learning activity in online learning and how to assess and support student's learning by their learning activity patterns. [For the full proceedings, see ED636095.]

Geography is one of the defining factors in scientific collaboration. Despite the voluminous evidence for how geographical proximity shapes the formation of collaboration in research articles, it has been rarely examined in the emerging genre of data papers, one that describes research data and has enjoyed growing attention in the data‐driven paradigm of research. This poster presents preliminary findings from our project that aims to evaluate the geographical dynamics behind the production of data papers. We analyze how researchers from different countries collaborate with one another using 6,821 data papers published in Scientific Data and Data in Brief between 2014 and 2020. We found that data papers rely heavily upon domestic collaboration and the collaboration pattern largely mirrors that of research articles, although some distinctiveness was also observed. We discuss future work in conclusion, with the ultimate goal of opening a more meaningful conversation about the relationship between the data‐driven paradigm and knowledge production. [ABSTRACT FROM AUTHOR]

In this era of technology, online data is increasing enormously in various forms be it images, videos, texts, sound, etc. Now with increasing data, there arises the issue of storage and that's the time when Bigdata and its tools come into the picture. To analyze the big data and process it with the available traditional methods, becomes an extremely strenuous task. And to overcome this drawback, there are various big data tools and techniques available. Some of the techniques that are used to analyze big data are the techniques of data mining such as clustering, classification, division, and prediction. The tools that are used in analyzing the big data are MongoDB, NoSQL, HPCC, Apache Storm, Spark, Apache Hadoop, also these tools are used in handling big data. In this paper, analysis of big data tools and techniques has been done with different examples and instances. To understand the tools better, their summaries along with examples are presented. [ABSTRACT FROM AUTHOR]

The value of an instructor is that she exactly recognizes what the learner is struggling with and provides constructive feedback straight to the point. This work aims at a step towards this type of feedback in the context of an introductory programming course, where students perform program execution tracing to align their understanding of Java instructions with reality. The students' submissions are analyzed for repeating mistakes across different exercises by representing the context surrounding the error by a graph and applying graph mining techniques to discover their common grounds. The patterns need to be annotated only once and help to address misconceptions of individual students. They may also be used to select follow-up exercises automatically, that contain the same intricacy. [For the full proceedings, see ED623995.]

An online peer-assessment system typically allows students to give textual feedback to their peers, with the goal of helping the peers improve their work. The amount of help that students receive is highly dependent on the quality of the reviews. Previous studies have investigated using machine learning to detect characteristics of reviews (e.g., Do they mention a problem, make a suggestion, or tell the student where to make a change?). Machine-learning approaches to peer-assessment evaluation are heavily reliant on labeled data to learn how to identify review characteristics. However, attaining reliable labels for those characteristics is always time-consuming and labor-intensive. In this study, we propose to apply pseudo-labeling, a semi-supervised learning-based strategy, to improve the recognition of reviews that detect problems in the reviewed work. This is done by utilizing a small, reliably labeled dataset along with a large unlabeled dataset to train a text classifier. The ultimate goal of this research is to show that for peer assessment evaluation, we can utilize both unlabeled and labeled datasets to obtain a robust auto-labeling system and thereby save much effort in labeling the data. [For the full proceedings, see ED623995.]

Ten years after the announcement of the "rise of the super experiment" at Educational Data Mining 2012, challenges to implementing "internet scale" educational experiments often persist for educational technology providers, especially when they seek to test substantive instructional interventions. Studies that deploy and test interventions, when informed by data-driven modeling, are often described as "close the loop" studies. Studies that close the loop attempt to link improvements in statistical and machine learning models of learning to real-world learning outcomes. After first considering challenges to internet scale experiments, we review several educational data science/mining studies that close the loop between data-driven modeling and learning outcomes. Next, we describe UpGrade, an open source architecture that, when integrated with educational technologies, helps overcome challenges to large-scale field trials (or internet scale experiments) that close the loop between data-driven work and real-world learning outcomes. In addition to describing preliminary randomized experiments that have been conducted and will soon be conducted using the architecture in two educational technology platforms, we end with a "call for contributors and integrators." UpGrade contributors and integrators will be researchers and developers who seek to drive continuous, data-driven improvements in real-world settings where learning with technology occurs. [For the full proceedings, see ED623995.]

Although badges are among the most-used game elements in gamified education, studies about their optimal features to motivate learning are scarce. How should a badge be designed to represent an incentive for a specific goal like optimal exam preparation? This study examines usage data of a higher education learning app to determine whether the used badges have the intended motivational effect. The preliminary results suggest that the badges that were initially implemented in the app have the intended effect in most cases, but the stages of the multi-level badges could be optimized. The methodological framework used in this study can be transferred to usage data of other similar learning tools. With the help of easy-to-interpret outputs of decision trees, researchers and practitioners alike can work towards an optimal badge design. [For the full proceedings, see ED623995.]

The aim of this study is the postgraduate thesis studies in the field of social studies education in Turkey between 2015 and 2021; to examine according to descriptive, methodological and subject distributions. Document analysis technique, which is one of the qualitative research designs, was used in the research. The main data source of the research is postgraduate theses. These theses were obtained by scanning in YÖK Thesis Center, and a total of 785 postgraduate theses, of which 659 are master's and 126 are doctoral theses, were examined. In the research, "Thesis Classification Table" and "Methodical Classification Table" were used as data collection tools. Content analysis technique, which is one of the analysis techniques used for qualitative research, was used in the research. According to the descriptive distribution results of the research; When the thesis type distributions were examined, it was observed that the master's theses were much more than the doctoral theses and the universities where the most studies were conducted were Gazi and Marmara universities. When the research type distributions of the theses were examined, it was observed that the most qualitative research type was used, and the scanning design was mostly used as a design. When the sample distributions of the theses were examined, it was observed that they mostly worked with secondary school students. According to the subject distribution results of the research; It was observed that subjects belonging to the categories of attitude/opinion/concern/belief and teaching method/technique/strategy/approach were studied. Another result is; It is the result of the theses made in the field of social studies education that there are some methodological deficiencies. It has been observed that some of the theses examined do not have a method section, and some theses have a method section, but are very incomplete and faulty in content. [For the full proceedings, see ED630948.]

With the rapid development of science and technology in recent years, the application areas of fuzzy logic have also gained speed. Fuzzy logic is a frequently preferred approach in the educational process, and it can be said that scientific publications on this topic have recently gained momentum in the literature. In this context, the present study aims to shed light on the research forms of fuzzy logic approach in education. The Web of Science (WoS) database was used for data collection. The search reached 374 scientific publications on the topic after various exclusion procedures. The obtained data were visualized using the mapping technique. As a result of the research, it was found that the number of publications on this topic has generally increased from past to present. The analysis revealed that the most influential countries reporting on this topic are the United States, China, and Taiwan. In addition, National Cheng Kung University, Gazi University, Cornell University, and Marmara University have published the most publications. The authors with the most publications on the fuzzy logic approach are Reyna V.F., Castillo, O. and Huang, Y.M. The analysis of the journals showed that Computer Applications in Engineering Education and International Journal of Electrical Engineering Education are the journals that contribute the most to this topic. The concepts that appeared in the clusters of co-occurrences analysis are fuzzy logic, higher education, e-learning, data mining, matlab and education. [For the full proceedings, see ED630948.]

Typical data science instruction uses generic datasets like survival rates on the Titanic, which may not be motivating for students. Will introducing real-life data science problems fill this motivational deficit? To analyze this question, we contrasted learning with generic datasets and artificial problems (Phase 1) with a community-sourced dataset and authentic problems (Phase 2) in the context of an 8-week virtual internship. Retrospective survey questions indicated interns experienced increased motivation in Phase 2. Additionally, analysis of intern discourse using Linguistic Inquiry and Word Count (LIWC) indicated a significant difference in linguistic measures between the two phases. Phase 1 had significantly greater measures of pronouns with a small-medium effect size, 2nd person words with a medium-large effect size, positive emotion with a medium effect size, inter-rogations with a medium-large effect size, question marks with a medium-large effect size, risk with a medium-large effect size, and causal words with a medium effect size. These results in conjunction with a retrospective survey suggest that phase 1 had more questions asked, more causal relationships defined, and included linguistic features of success and failure. Results from Phase 2 indicated that community-sourced data and problems may increase motivation for learning data science. [For the full proceedings, see ED623995.]

The COVID-19 epidemic had caused one of the most significant disruptions to the global education system. Many educational institutions faced sudden pressure to switch from face-to-face to online delivery of courses. The conventional classes are no longer the primary means of delivery; instead, online education and resources have become the prominent approach. With the increasing demand for supplementary course materials to fulfill the needs of each area of study, students began to use search engines and online resources that contain discussions, practical demonstrations, and tutorial videos to aid students in their studies and course work. This study addresses the underlying challenges of retrieving relevant online educational materials by introducing an intelligent agent for semantic data mining. It works as middleware infrastructure that allow context-aware data processing and mining. YouTube was used to assess the consistency of the proposed model since it returns a large number of results in its search pool. The results showed that using the extraction of topics method, the similarities scores with the proposed model provided favorable results. Furthermore, an improvement in video ranking and sorting was realized. According to the findings, using this method provided users with a more productive and reliable study experience. [For the full proceedings, see ED639633.]

Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and unbiased without additional assumptions, regardless of the deep-learning model's quality. In this dataset, incorporating auxiliary data improves precision consistently and, in some cases, substantially.

We interrogate the opportunities and challenges of using mixed methods (MM) within a developmental evaluation (DE) context by drawing on two illustrative cases that investigated educational change in Canada. Methods: Multi-case design and cross-case analysis, with a focus on examining common patterns across the two cases, enabling new ways of portraying and thinking about the data. Findings: We present a four-part framework for charting MM integration (why, what, when, and how) within DE contexts. Then, from the juxtaposition of these cases using this framework, we present three considerations for evaluation practice and future research: (1) balancing the need for data collection versus moving initiatives forward, (2) not assuming stakeholders want co-production, and (3) developing integration literacy.

To innovate firms, need to source knowledge and technology from different actors, including universities. The present paper is a systematic literature review of the benefits obtained by the firms from the collaboration with universities. Out of an initial sample of 4100 articles, 88 empirical studies which measured benefits obtained by the firms from University-industry (U-I) collaboration were selected. The analysis showed that by engaging in U-I collaboration firms may access six types of benefits and, in turn, the size of these benefits is influenced by five groups of factors. It was identified also the high diversity of measurement instruments which makes it harder to obtain statistical comparability between different studies. Most studies were conducted in high-tech sector showing the lack of studies in less technological-intensive sectors. The paper also suggests a list of existing research streams and avenues for future research. [ABSTRACT FROM AUTHOR]

As societies rely increasingly on computers for critical functions, the importance of cybersecurity becomes ever more paramount. Even in recent months there have been attacks that halted oil production, disrupted online learning at the height of COVID, and put medical records at risk at prominent hospitals. This constant threat of privacy leaks and infrastructure disruption has led to an increase in the adoption of artificial intelligence (AI) techniques, mainly machine learning (ML), in state-of-the-art cybersecurity approaches. Oftentimes, these techniques are borrowed from other disciplines without context and devoid of the depth of understanding as to why such techniques are best suited to solve the problem at hand. This is largely due to the fact that in many ways cybersecurity curricula have failed to keep up with advances in cybersecurity research and integrating AI and ML into cybersecurity curricula is extremely difficult. To address this gap, we propose a new methodology to integrate AI and ML techniques into cybersecurity education curricula. Our methodology consists of four components: i) Analysis of Literature which aims to understand the prevalence of AI and ML in cybersecurity research, ii) Analysis of Cybersecurity Curriculum that intends to determine the materials already present in the curriculum and the possible intersection points in the curricula for the new AI material, iii) Design of Adaptable Modules that aims to design highly adaptable modules that can be directly used by cybersecurity educators where new AI material can naturally supplement/substitute for concepts or material already present in the cybersecurity curriculum, and iv) Curriculum Level Evaluation that aims to evaluate the effectiveness of the proposed methodology from both student and instructor perspectives. In this paper, we focus on the first component of our methodology - Analysis of Literature and systematically analyze over 5000 papers that were published in the top cybersecurity conferences during the last five years. Our results clearly indicate that more than 78% of the cybersecurity papers mention AI terminology. To determine the prevalence of the use of AI, we randomly selected 300 papers and performed a thorough analysis. Our results show that more than 19% of the papers implement ML techniques. These findings suggest that AI and ML techniques should be considered for future integration into cybersecurity curriculum to better align with advancements in the field. [ABSTRACT FROM AUTHOR]

when conducting business and management research, the most common strategy employed for collecting primary data is either interviews or questionnaires (or both). When questionnaires are used to collect primary data a very common approach used is to construct questions which can be called Likert-type questions or Likert scales. Numbers are allocated to the question responses of each question. Often, this is followed by performing arithmetic or statistical operations on the allocated numbers. In many cases encountered by the author, the analysis has even included techniques such as parametric statistics and factor analysis. This paper explains why such simplistic approaches are completely invalid and should never be used. It goes on to explain how analysis can be done whilst avoiding typical hazards. Often, the writers of such papers do not understand or explain levels of measurement. Of course, Likert scales are at an ordinal level of measurement which would normally preclude the use of arithmetic, statistical, factor analysis techniques there is an additional problem of reliability because different people will interpret terms such as strongly disagree and disagree, differently. Likert was aware of these problems when writing the original paper in 1932 although at that time the term "levels of measurement" was not in use. This paper provides approaches and suggestions for avoiding the problems of data analysis when using Likert-type questions. This paper should be of assistance to those who intend to use Likert-type questions in the questionnaire. [ABSTRACT FROM AUTHOR]

This paper examines the challenge industrial companies face in transforming pilot AI projects into successful, scalable new businesses models. An evaluation of international industrial firms has revealed common difficulties in maintaining the objectives of AI business models and achieving alignment between AI initiatives and overarching strategic goals. These misalignments result in significant challenges, including scalability issues, unclear return on investment (ROI), and a general misalignment with business objectives. To address these challenges, a comprehensive methodical framework is introduced that guides companies through the complexity of scalable AI implementation in industrial domain. This framework not only provides use-case-specific guidelines, but also defines AI-specific strategies and addresses critical cultural and organizational considerations necessary for successful AI integration. [ABSTRACT FROM AUTHOR]

Foresight and anticipation of technology convergence has been of interest to innovation management scholars and practitioners for almost three decades. Since then, the prevailing data source for analysing convergence movements and for delimiting them from fusion dynamics are patents. However, increasing criticism arose about the limited forecasting power of established analysis methods, insinuating they rather monitor technology convergence than anticipating it. Our ongoing research argues that the discipline is in need of a new data source for analysing technology convergence and thus proposes online job postings as more forward-looking and future-oriented data. This research-inprogress paper provides a brief literature review on the theoretical background of technology convergence and outlines the preliminary design of our ongoing research on anticipating fusion and/or convergence movements in the AI technology field. Next to describing our keyword-based job postings analysis we also introduce a novel interpretation approach for differentiating between technology fusion and convergence. [ABSTRACT FROM AUTHOR]

In the current era, one of the key challenges in utilizing the Holt-Winters Method of forecasting is the accurate selection of smoothing coefficients. To address this issue, researchers have explored an optimization approach that aims to minimize forecasting errors, such as Mean Squared Errors (MSE) or Mean Absolute Deviation (MAD). This paper presents a novel methodology that employs a Genetic Algorithm (GA) to optimize the forecasting error by determining the optimal smoothing coefficients for the Holt-Winters Method. The objective value of the optimization problem is the Mean Square Error (MSE), which serves as a measure of the accuracy of the forecast. To evaluate the effectiveness of the proposed approach, actual test cases based on rice stock commercial commodity in the Philippines during the COVID-19 pandemic were utilized. The paper examines different variants of the Holt-Winters Method and assesses their suitability for capturing the characteristics of the rice stock data. The findings indicate that an additive seasonal effect is more appropriate for modeling the seasonal patterns observed in the rice stock data. Furthermore, the performance of the proposed GA-based approach is compared to other forecasting models to ascertain its efficacy. The results demonstrate promising outcomes, suggesting that the GA-based optimization approach for determining the smoothing coefficients in the Holt-Winters Method improves the accuracy of rice stock forecasting during the COVID-19 pandemic. [ABSTRACT FROM AUTHOR]

The 10 billion J/ψ decays collected with the BESIII experiment offer a unique opportunity to investigate the decays of η and η′ mesons produced in the radiative J/ψ → γη(′) transitions. Using this clean production mechanism, the BESIII experiment is making important contributions to precision studies of the strong and electromagnetic interactions in η(′) decays. Three papers are presented in which the decays η → π+π−π0, η → π0π0π0, η′ → ηπ0π0π0, and η′ → π+π−e+e− are studied to test C and CP symmetry, as well as the predictions of different effective field theories. [ABSTRACT FROM AUTHOR]

The meaning of sustainability is increasing among manufacturing companies, as pressure is coming from both within and outside of the company. Data and sustainability are strongly linked as data, e.g., enables highlighting sustainability actions and measuring implications. This paper studies, how manufacturing companies are seeing the role of sustainability data when innovating their practical actions. We explore the theme by presenting findings from 12 recent interviews with manufacturing companies. The results are demonstrated in a framework for describing companies' data utilisation and sustainability-related actions, divided into four different types: 1) Beginners in data utilisation having no sustainability focus, 2) Beginners in data utilisation with sustainability focus, 3) Advanced data utilisers with rising sustainability focus and 4) Advanced data utilisers with sustainability focus. In addition, the paper presents practical viewpoints of sustainability data in manufacturing companies. Managerial and theoretical implications consist of clarifying scattered viewpoints for the role of sustainability data in manufacturing companies. [ABSTRACT FROM AUTHOR]

Engineering Management (EM) has a long history in Australia as far back as 1968. However, a review of scholarly papers has revealed a lack of attention from researchers for this discipline in the past 10 years, especially topics about industrial demand and competency. This paper is to provide a review of current EM practices in Australia, classified into Occupation and Market demand, Professional Society, and Required Qualifications and Competency. A review of EM practices in Australia has been conducted through 24 scholarly papers from 1991 to 2022 along with a statistical analysis of data from government and professional society. The review has revealed the increasing market demand for engineering managers (EMers). Temporary and permanent visa has been offered since 2005 and 2012 respectively to fill the gap in the domestic market. The EM career path with required years of experience and the trend of qualifications are also synthesized from different research and data. Different from the US, Australia is lacking a professional society that is dedicated to the management and promotion of this discipline in the country. EM competency is currently defined by Engineers Australia (EA) only for migration purposes. The paper raises the awareness of management skills for all engineers on their engineering career path. This paper also provides implications for strategic plans, particularly for educational institutions in developing engineering management programs. Further, recommendations for future research directions are outlined. This is the first literature review paper on EM practices in Australia. [ABSTRACT FROM AUTHOR]

Existing evidence indicates that Free/Libre Open-Source Software (FLOSS) ecosystems offer extensive learning opportunities. Community members actively participate in various activities, both during their interactions with peers and while utilizing these environments. Given that FLOSS repositories contain valuable data on participant interactions and activities, our study focuses on analyzing knowledge exchange and interactions within emails to track learning activities across different phases of the learning process, with a focus on the first phase (Initiation). In this paper, we leverage Natural Language Processing (NLP) and Machine Learning (ML) techniques within a process mining framework. Specifically, we employ NLP techniques to analyze the contents of emails and messages exchanged in these FLOSS repositories to generate event logs for the purpose of modeling learning patterns. Subsequently, we construct corresponding event logs, which serve as input to Disco, the process mining tool, for learning process discovery in these environments. The output comprises visual workflow nets that we interpret as representations of learning activity traces within FLOSS, capturing their sequential occurrences. To enhance the understanding of these models, we incorporate additional statistical details for contextualization and description. This approach enables a nuanced exploration of learning dynamics within FLOSS environments, emphasizing the role of NLP and ML in uncovering valuable insights on how FLOSS participants acquire and exchange knowledge. [ABSTRACT FROM AUTHOR]

Vocational Education and Training (VET) suffers from an increasing rate of student dropout in the school sector, while it is largely neglected in the learning analytics (LA) research. At the same time school dropout rates are consistently rising globally and researchers are trying to better understand this rising phenomenon. Some concentrate on its relation to chronic absenteeism from school and to the student socioeconomic background. In addition, other contextual factors can come into play to explain this complex and challenging social phenome-non. Research also shows that the use of LA could contribute to alleviating school dropout. The main purpose of this paper is to present a LA system that visualizes student dropout in the VET school context providing the rea-sons behind it. The paper describes both the backend and the frontend of the system. Regarding the former, the paper focuses on the LA system design and architecture. Regarding the latter, it presents a set of interactive LA visualisations provided by our system based on existing student data on school dropout. In particular, the paper demonstrates its LA visualisations via a case study in a large VET school in Greece based on student data gathered from four consecutive school years. The LA visualisations show students' dropouts across years in the school unit, trajectories of students' dropouts in a specific school year, and the degree of the association between some geographical area (as the area of student residence) with chronic absenteeism. [ABSTRACT FROM AUTHOR]

The need for data is growing at an extremely steep rate in the ever-digital realm, where terms like "big data" are becoming a thing of the past. All this development requires the use of modern and advanced data handling techniques, where users and researchers can analyze and predict vast amounts of data efficiently. Data warehouses are centralized repositories of data used for business intelligence activities such as analysis and reporting. In this paper, a comparative emphasis has been laid down on two different types of data warehouses, on-premises, and cloud data warehouses. The on-premises are known to be physically housed inside an organization's infrastructure. Cloud data warehouses are online-accessible repositories for data that is stored on cloud platforms. This paper provides a comparative analysis of both types in the context of deployment, scalability, flexibility, query management, cost analysis, access and integration, data security, data storage, data recovery, self-service capabilities and nonetheless, speed and performance. This article further highlights the evolution of data warehouses onto cloud and accentuates the growing demand for an efficient data warehouse, due to the amplification of volume, velocity, variety, value, and veracity of the incoming data in all realms. Furthermore, it provides an in-depth analysis of the advantages of the most suitable data warehouse and discusses the limitations of both. [ABSTRACT FROM AUTHOR]

The fourth industrial revolution was a radically new round in the evolution of many processes in society, including management processes. The digital revolution has provoked the development of new management models and concepts. Agile manufacturing is one of such managerial production concepts that meets modern challenges and requirements of the business environment. Literature review as a research method plays an important role in the studies. This paper presents a systematic literature review of the papers published in SCOPUS and Web of Science database about agile manufacturing since 2000. Specifically, we aim at analysing the research methods on this very popular topic on the economy and management. We believe that by defining the methods used in the agile field we may understand the nature of the research. It is well known that agile manufacturing methods are meant to be upfront in terms of efficiency, but in this paper we want to make a review on research methods to check how research has been made. Therefore, we believe this research is useful for scientists and practitioners. [ABSTRACT FROM AUTHOR]

The continuous growth of data volume to be stored and processed in the shortest possible time brings new requirements in terms of data storage, processing, analysis and visualization. Relational databases are not capable to store and process big data efficiently and NoSQL databases have emerged as an appropriate alternative. The range of available NoSQL systems is wide and the suitability of a given NoSQL system depends on the problem domain. In this paper, a systematic literature review of various NoSQL systems by application domains is performed in order to ease the selection of appropriate NoSQL system for a specific purpose. [ABSTRACT FROM AUTHOR]

Many operational research optimization models, such as linear programming, assume that the data used to build a model is accurate, but in the real world most of these assumptions are only approximately true. In linear programming models, uncertainty is associated with the model coefficients during the formulation stage. A small number of papers discuss these models of linear programming with interval coefficients (LPIC). Using the method presented in paper [1], capable of solving linear programming problems with numerical intervals as intervals, we give in this paper a practical example, to demonstrate the use and the practicality of interval coefficients in linear programming models. [ABSTRACT FROM AUTHOR]