Search

This paper proposes a new methodology to improve territorial cohesion in clustering processes where many variables from different topics are considered. Clustering techniques provide added value to identify typologies, but there are still unsolved challenges when data contain an unbalanced number of variables from different topics. The territorial feature selection method (TFSM) is presented as a method to select the representative variable of each topic such that the interpretability of resulting clusters is preserved and the geographical cohesion is improved with respect to classical approaches. This paper also introduces the thermometer as a new knowledge acquisition tool that allows experts to transfer semantics to the data mining process. TFSM proposes the index of potential explainability ( E k ) as the criteria to select the most promising variables for clustering. E k is based on the combination of inferential testing and metrics such as support. The proposal is applied with the INSESS-COVID19 database, where territorial groups of vulnerable populations were found. A set of 195 variables with 21 unbalanced thematic blocks is used to compare the results with a traditional multiview clustering analysis with promising results from both the geographical and the thematic point of view and the capacity to support further decision making. [ABSTRACT FROM AUTHOR]

This paper shows the added value of using the existing specific domain knowledge to generate new derivated variables to complement a target dataset and the benefits of including these new variables into further data analysis methods. The main contribution of the paper is to propose a methodology to generate these new variables as a part of preprocessing, under a double approach: creating 2nd generation knowledge-driven variables, catching the experts criteria used for reasoning on the field or 3rd generation data-driven indicators, these created by clustering original variables. And Data Mining and Artificial Intelligence techniques like Clustering or Traffic light Panels help to obtain successful results. Some results of the project INSESS-COVID19 are presented, basic descriptive analysis gives simple results that even though they are useful to support basic policy-making, especially in health, a much richer global perspective is acquired after including derivated variables. When 2nd generation variables are available and can be introduced in the method for creating 3rd generation data, added value is obtained from both basic analysis and building new data-driven indicators. [ABSTRACT FROM AUTHOR]

This paper describes the performances of a marine SO 2 absorption scrubber installed onboard a large Ro-Ro cargo ship. The study is based on the reconstruction of an extensive dataset from one-year continuous monitoring of the scrubber's performances and operating conditions. The dataset has been interpreted with a conventional analytical, physical-mathematical, model for absorbers' rating and its combination with an Artificial Intelligence (AI) one. First, the analytical model has been used to provide a deterministic mathematical framework for the interpretation and the prediction of the scrubber's performances in terms of absorbed SO 2 molar flow and SO 2 concentration at the scrubber exit. Then, data mining and AI techniques have been applied to develop an Artificial Neural Network able to predict the error between the actual SO 2 concentration at the scrubber exit and the corresponding analytical model predictions. The final result is a combined model providing superior robustness and accuracy in the prediction of the scrubber performance while preserving a rationale for process design and operation. This interesting outcome suggests that the development of combined, or hybrid, Analytical/AI models can be a reliable and cost-effective way to improve chemical engineers' ability to design and control marine scrubbers, as well as other chemical equipment. • The performances and operation of a marine scrubber has been collected in a dataset. • An Analytical (A) model is used to describe the scrubber performances. • An Artificial Intelligence (AI) model is used to interpret the error of the A model. • The combined A/AI model greatly improve the prediction of scrubber performances. [ABSTRACT FROM AUTHOR]

The vast amount of astronomical information that has become available over this decade has far exceeded that of the last century. The heterogeneity of the data and its overwhelming magnitude have made it impossible to perform manual analysis. As a consequence, new techniques have been developed and different strategies have been amalgamated, such as data science and data mining, in order to carry out more in-depth and exhaustive analyses in search of the extraction of the knowledge contained in data. This paper introduces a data science methodology that consists of successive stages, with the core of this proposal being the step of data preprocessing, with the aim of reducing the complexity of the analysis and enabling hidden knowledge in the data to be uncovered. The proposed methodology was tested on a set of data consisting of artificial light curves that try to mimic the behaviour of the strong gravitational lens phenomenon, as supplied by the Time Delay Challenge 1 (TDC1). Under the data science methodology, diverse statistical methods were implemented for data analysis, and cross-correlation and dispersion methods were applied for the time-delay estimation of strong lensing systems. With this methodology, we obtained time-delay estimations from the TDC1 data set and compared them with earlier results reported by the COSmological MOnitoring of GRAvItational Lenses project (COSMOGRAIL). The empirical evidence leads us to conclude that, with the proposed methodology, we achieve a greater accuracy in estimating time delays compared with estimations made with raw data. [ABSTRACT FROM AUTHOR]

Many disciplines, including the broad Field of Information (iField), offer Data Science (DS) programs. There have been significant efforts exploring an individual discipline's identity and unique contributions to the broader DS education landscape. To advance DS education in the iField, the iSchool Data Science Curriculum Committee (iDSCC) was formed and charged with building and recommending a DS education framework for iSchools. This paper reports on the research process and findings of a series of studies to address important questions: What is the iField identity in the multidisciplinary DS education landscape? What is the status of DS education in iField schools? What knowledge and skills should be included in the core curriculum for iField DS education? What are the jobs available for DS graduates from the iField? What are the differences between graduate‐level and undergraduate‐level DS education? Answers to these questions will not only distinguish an iField approach to DS education but also define critical components of DS curriculum. The results will inform individual DS programs in the iField to develop curriculum to support undergraduate and graduate DS education in their local context. [ABSTRACT FROM AUTHOR]

The data mining is data examining strategies that used to investigate crime data recently put away from different sources to find examples and patterns in crimes. In extra, it tends to be applied to build productivity in settling the crimes quicker and furthermore can be applied to naturally inform the crimes. In any case, there are numerous data mining strategies. To expand productivity of crime location, it is important to choose the data mining strategies appropriately. This paper audits the writings on different data mining applications, particularly applications that applied to tackle the crimes. Survey likewise illuminates research holes and difficulties of crime data mining. In extra to that, this paper gives understanding with regards to the data mining for finding the examples and patterns in crime to be utilized suitably and to be an assistance for amateurs in the examination of crime data mining. [ABSTRACT FROM AUTHOR]

With the emergence of social media and the Internet, bulks of data are generated every day in different fields. However, filtering out useful information, which can be turned into knowledge, requires significant effort. This can be achieved using various Big Data Technologies, statistics, data mining and a variety of machine learning algorithms that corporates and businesses are using to improve their performance every day for a long time now. In this paper, we analyze YouTube trending video data that analyses the target that grabs the user's attention over a relatively short time. We present our analysis by measuring, mining, analyzing and comparing key aspects of time-series YouTube data with respect to its view and audience response statistics from 40 000 trending YouTube videos collected over 205 days. We have performed an exploratory data analysis (EDA) on all of it's aspect to get data insights and used statistics to find similarities between them to understand viewing pattern of different video categories. We also compare and observe the variation of activity over time with the nature of the event that affects the quality of our analysis. We make the viewership pattern clear belonging to each category separately by direct analysis. The results and findings can help us predict the category of the video that could probably be trending next in the upcoming days and help individuals get boosted views and performance when they upload video as per the predictions and analysis. [ABSTRACT FROM AUTHOR]

Currently "Big Data" is an emerging field that presents several Information Technology challenges regarding the capture, storage search, structure, and visualization of this data. The real challenge for organizations is to find ways to extract value from it and provide better services to their clients. The data generated in academic and other institutions is vast and complex. Libraries face new challenges as they seek to determine their role in the handling of Big Data within their organization and use it to develop services. Thus, in most organizations, libraries will not have the knowledge to build new services unaided. Furthermore, libraries have always been information handlers and technology adopters; therefore, Big Data technologies will certainly affect their context. The purpose of this paper is to explore all these issues through a systematic literature review, unveiling the theories that underpin the paper's argument. It attempts to answer several research questions, such as how librarians are involved in the Big Data era? And what are the future research developments of Big Data within the library context? The study considered only papers published between 2012 and 2018 in English and presents the collected literature by grouping them according to the type of library each paper refers to. Thus, it identifies new and evolving roles in the context of all types of libraries. In addition, the study presents several interesting tables, which aim to help librarians locate relevant articles that will inform their practice and guide service development for users of large and complex datasets. [ABSTRACT FROM AUTHOR]

Over the last decade, Educational Data Mining (EDM) and Learning Analytics (LA) have evolved enormously as interrelated research areas and disciplines. Many researchers interested in these areas may wonder why there are two different communities, whether they are the same concept or not, and the differences between them, which is key information for designing their research and publication strategies. To address this, we conducted a systematic review of academic papers about the differences between LA and EDM following the Preferred Reporting Method for Systematic Reviews (PRISMA) guidelines. We selected 10 research works and identified 11 differences. Our conclusions are that, although both use the same data and share similar goals and interests, EDM and LA are different research communities with different origins and focuses, with their respective conferences and journals. However, there is active collaboration between the two communities and their members often tend to publish in both fields' conferences and journals. Additionally, none of the differences are apparently large enough to conclude that LA and EDM follow different paths for improving the teaching-learning process, but rather the opposite. Following a common future line, it seems that the two "sister" communities are working together with the same perspective, along with some "cousin" communities such as AIED (Artificial Intelligence in Education), L@S (Learning at Scale), Learning Science (LS), etc. in the same area that could be called Educational Data Science (EDS). We propose using the term EDS to integrate both LA and EDM with all these related communities. [Display omitted] • Educational Data Mining (EDM) and Learning Analytics (LA) are interrelated and sometimes interchanged terms. • Five original differences between EDM and LA have been identified. • Other six new differences have emerged since these initial five differences. • The term Educational Data Science (EDS) is proposed to integrate EDM and LA. [ABSTRACT FROM AUTHOR]

In the field of data science and data mining, the problem associated with clustering features and determining its optimum number is still under research consideration. This paper presents a new 2D clustering algorithm based on a mathematical topological theory that uses a pseudometric space and takes into account the local and global topological properties of the data to be clustered. Taking into account cluster symmetry property, from a metric and mathematical-topological point of view, the analysis was carried out only in the positive region, reducing the number of calculations in the clustering process. The new clustering theory is inspired by the thermodynamics principle of energy. Thus, both topologies are recursively taken into account. The proposed model is based on the interaction of particles defined through measuring homogeneous-energy criterion. Based on the energy concept, both general and local topologies are taken into account for clustering. The effect of the integration of a new element into the cluster on homogeneous-energy criterion is analyzed. If the new element does not alter the homogeneous-energy of a group, then it is added; otherwise, a new cluster is created. The mathematical-topological theory and the results of its application on public benchmark datasets are presented. [ABSTRACT FROM AUTHOR]

• Anomaly detection is one of the most common problems in data science. • Unsupervised approaches are useful since anomalies are not easily reproducible. • Triangle-based outlier detection is based on geometric reasoning. • Results achieved show a higher robustness than state of the art alternatives. For the last decades, anomaly detection has been one of the most common problems in data mining and computer science projects. The scientific community has made a great effort to develop methods and techniques for the detection of elements that deviate from the norm. Many of these techniques follow an unsupervised approach since anomalies are scarce and not easily reproducible. In this paper, a novel unsupervised anomaly detection method based on geometric reasoning is proposed. Triangle-based Outlier Detection (TOD) falls within the group of distance-based anomaly detection techniques and has been compared to state of the art methods for unsupervised anomaly detection. Results achieved prove that TOD offers greater robustness than alternative methods for the selection of the decision thresholds that condition the detection of outliers. [ABSTRACT FROM AUTHOR]

Design-by-analogy (DbA) is a design methodology wherein new solutions, opportunities, or designs are generated in a target domain based on inspiration drawn from a source domain; it can benefit designers in mitigating design fixation and improving design ideation outcomes. Recently, the increasingly available design databases and rapidly advancing data science and artificial intelligence (AI) technologies have presented new opportunities for developing data-driven methods and tools for DbA support. In this study, we survey existing data-driven DbA studies and categorize individual studies according to the data, methods, and applications into four categories, namely, analogy encoding, retrieval, mapping, and evaluation. Based on both nuanced organic review and structured analysis, this paper elucidates the state-of-the-art of data-driven DbA research to date and benchmarks it with the frontier of data science and AI research to identify promising research opportunities and directions for the field. Finally, we propose a future conceptual data-driven DbA system that integrates all propositions. [ABSTRACT FROM AUTHOR]

Every conference imposing a limit on the length of submissions must deal with the problem of page limit cheating: authors tweaking the parameters of the game such that they can squeeze more content into their paper. We claim that this problem is endemic, although we lack the data to formally prove this. Instead, this paper provides a far from exhaustive summary of ways to cheat the page limit, a case study involving the papers accepted for the Research and Applied Data Science tracks at the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD) 2019, and a discussion of ways for program chairs to tackle this problem. Of the 130 accepted papers in these two ECMLPKDD 2019 tracks, 68 satisfied the page limit; 62 (47.7%) turned out to spill over the page limit, by up to as much as 50%. To misappropriate a phrase from Darrell Huff's "How to Lie with Statistics," we intend for this paper not to be a manual for swindlers; instead, nefarious paper authors already know these tricks, and honest program chairs must learn them in self‐defense. This article is categorized under:Commercial, Legal, and Ethical Issues > Fairness in Data Mining [ABSTRACT FROM AUTHOR]

There is an ongoing methodological debate in philosophy of science concerning the use of case studies as evidence for and/or against theories about science. In this paper, I aim to make a contribution to this debate by taking an empirical approach. I present the results of a systematic survey of the PhilSci-Archive, which suggest that a sizeable proportion of papers in philosophy of science contain appeals to case studies, as indicated by the occurrence of the indicator words "case study" and/or "case studies." These results are confirmed by data mined from the JSTOR database on research articles published in leading journals in the field: Philosophy of Science, the British Journal for the Philosophy of Science (BJPS), and the Journal for General Philosophy of Science (JGPS), as well as the Proceedings of the Biennial Meeting of the Philosophy of Science Association (PSA). The data also show upward trends in appeals to case studies in articles published in Philosophy of Science, the BJPS, and the JGPS. The empirical work I have done for this paper provides philosophers of science who are wary of the use of case studies as evidence for and/or against theories about science with a way to do philosophy of science that is informed by data rather than case studies. [ABSTRACT FROM AUTHOR]

National statistical institutes are using frameworks to organise and set up their official statistical production, e.g. GSBPM. As a sequential approach of statistical production, GSBPM has become a well-established standard using deductive reasoning as analytics' paradigm. For example, the first GSBPM steps are entirely focused on deductive reasoning based on primary data collection and are not suited for inductive reasoning applied to (already existing) secondary data (e.g. big data resulting, for example, from smart ecosystems). Taken into account the apparent potential of big data in the official statistical production, the GSBPM process needs to adapted to incorporate both complementary approaches of analytics (i.e. inductive and deductive reasoning) and, for example, through the usage of, for example, data-informed continuous evaluation at any GSBPM step. This paper discusses the limitations of GSBPM with respect to the usage of big data (using inductive reasoning as analytics' paradigm), and also with respect to trusted smart statistics. The authors give insights on how to augment and empower current statistical production processes by analytics, and also by (trusted) smart statistics. In addition, the paper also highlights challenges and opportunities that should be addressed to embrace this major paradigm shift. [ABSTRACT FROM AUTHOR]

The generation of data from every corner of the world forced the data scientists to think over it, how to handle such voluminous data while processing and storing. Therefore, to tackle this gigantic data called Big Data, its analytics has become very important. The development of innovative tools and algorithms is the need of the hour for the academic world, research study, and IT industry. The uncontrolled and continuous expansion of data sources generates heterogeneous data at a speed of light over the internet including Tweets, Facebook posts/likes, Blogs, news, articles, YouTube videos, website clicks, etc. Big data becomes a new challenge for research communities to extract meaningful information for commercial as well as personal use. There are numerous open source programming platform available free of cost for processing big data such as Hadoop, MapReduce, Hive, Flink, Spark, etc. Hadoop is an open source, distributed computing machine used for big data analytics. Map reduce is one of the most important and useful processing tools written in java language. This tool processes the large-scale data through distributed mode. It converts the large data inputs in small chunks and distributes them on different machines that are interconnected with each other in the form of a cluster. On the other hand, 'R' is another freely available statistical tool that offers a set of different types of libraries for statistical data mining. In this paper, we have presented architecture that allows coordination among MapReduce and R Libraries. This architecture will promote building predictive analytics combined with performance and flexibility for data science as it helps in exporting R libraries and process through MapReduce. The main objective of this paper is to provide in-depth analysis and relative evaluation of most up-to-date tools and models used for big data analytics. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new method for disjoint principal component analysis based on an intelligent search. The method consists of a principal component analysis with constraints, allowing us to determine components that are linear combinations of disjoint subsets of the original variables. The effectiveness of the proposed method contributes to solve one of the crucial problems of multivariate analysis, that is, the interpretation of the vectorial subspaces in the reduction of the dimensionality. The method selects the variables that contribute the most to each of the principal components in a clear and direct way. Numerical results are provided to confirm the quality of the solutions attained by the proposed method. This method avoids a local optimum and obtains a high success rate when reaching the best solution, which occurs in all the cases of our simulation study. An illustration with environmental real data shows the good performance of the method and its potential applications. [ABSTRACT FROM AUTHOR]

Purpose: In a socio-educational context impregnated by social networks, feminist organizations and individuals have turned to social media to spread their knowledge. This paper aims to approach how feminist tweets are produced to ignite meaningful informal learning (IL) processes. Design/methodology/approach: This study uses an interdisciplinary mixed methodology. By using Twitter tracking tools, a database has been enabled to catalogue feminist hashtags into topics and categories for further analysis. These data have been contrasted with surveys to the managers of the most followed feminist accounts in Spain and Catalonia. Findings: From an educational perspective, the analysed feminist hashtags have been organized in 13 different topic categories. The different propagation processes on Twitter – tweeting and retweeting – imply diverse learning processes. Moreover, tweets with complementary information such as images or links generate the most interaction, being the preferred format for IL. Research limitations/implications: Researching with Big Data in educational sciences is a field in development, and Twitter data collection tools are mostly addressed to marketing and economic sectors; thus, free tools with limited services were used, offering the analysis of a brief and concrete situation of a platform in constant change. Although this ephemeral data and its relevance does not prevail over time, it has an impact on citizens' learning. Originality/value: It is the first study in Spain that illustrates the informal education that feminism offers to the community, facing the complexity of measuring Twitter with an educational perspective through the use of marketing tools. [ABSTRACT FROM AUTHOR]

In this paper we develop a method to quantify the accuracy of different pattern extraction techniques for the additive space–time modes often assumed to be present in climate data. It has previously been shown that the standard technique of principal component analysis (PCA; also known as empirical orthogonal functions) may extract patterns that are not physically meaningful. Here we analyze two modern pattern extraction methods, namely dynamical mode decomposition (DMD) and slow feature analysis (SFA), in comparison with PCA. We develop a Monte Carlo method to generate synthetic additive modes that mimic the properties of climate modes described in the literature. The datasets composed of these generated modes do not satisfy the assumptions of any pattern extraction method presented. We find that both alternative methods significantly outperform PCA in extracting local and global modes in the synthetic data. These techniques had a higher mean accuracy across modes in 60 out of 60 mixed synthetic climates, with SFA slightly outperforming DMD. We show that in the majority of simple cases PCA extracts modes that are not significantly better than a random guess. Finally, when applied to real climate data these alternative techniques extract a more coherent and less noisy global warming signal, as well as an El Niño signal with a clearer spectral peak in the time series, and more a physically plausible spatial pattern. [ABSTRACT FROM AUTHOR]

One of the most significant indicators for assessing the quality of university careers is the dropout rate between the first and second year. Both literature on the subjects and the results that emerged from numerous specific investigations into the dropouts of the university system, showed the crucial importance of this junction between the first and the second year. Reasons for dropping out can be quite varied, ranging from incorrect and/or insufficient prospective student orientation, the willingness or need to find a job as quickly as possible, to a lack of awareness of not being able to cope with a particular course of study rather than another. In this paper we focus specifically on the problem of dropouts in Italy, addressing it from a dual point of view. At an aggregate level, the analysis deals with dropout rates in Italy between the first and second year, in order to identify the main trends and dynamics at the national level. Subsequently, we analyze individual-level data from the University of Bari Aldo Moro, aiming to identify the most important contributing factors. This individual-level approach has emerged over recent years, and is generally known as 'Educational Data Mining', focused on the development of ad hoc methods that can be used to discover regularities and new information within databases from contexts related to education. Using supervised classification methods, we are able to identify retrospectively the profile of students who are most likely to dropout. [ABSTRACT FROM AUTHOR]

With the exponential growth of the Internet of Things and Cloud Computing, especially in recent years, the potentiality of Machine Learning (ML) has been demonstrated and amplified, together with data mining developments and the availability of large amounts of data. In order to design a ML system capable of producing effective and accurate predictions and results it is necessary to make sure that it is actually working on large data sets; clean, high quality and complete data really representing the information you are trying to analyze. More data are available, the more accurate are evidently predictions. Forecasting represents an important use of extracted knowledge from data. It is the process of predict future demand for an offered product or service, and it allows for optimizing company decisions, reducing risks, managing stocks, planning sales and making many other internal or in-market assessments. In this paper we present and discuss a novel ensemble technique in forecasting workload of local health department. The proposed approach relies on a real dataset composed by over than 20 M of administrative e-health records. Obtained results demonstrate that our ensemble approach outperforms the state-of-the-art. [ABSTRACT FROM AUTHOR]

Similarity, correlation and association measures play an important role in statistics, information retrieval, data mining and data science, classification and machine learning, recommender systems and decision-making. They have numerous applications in ecology, social and behavioral sciences, biology and bioinformatics, social network and time series analysis, image and natural language processing. Often the measures with the same name introduced on different domains have different properties, and the measures with the same properties have different names. To unify analysis of measures defined on different domains, this paper considers these measures as functions defined on universal domain and satisfying some sets of properties. The general properties of similarity functions (SF) and dissimilarity functions (DF) under the joint name of resemblance functions (RF) studied on universal domain and illustrated by examples on specific domains. The known and the new methods of construction of similarity measures are considered. This paper discusses the following aspects of RF: relationship with fuzzy (valued) relations, T-transitivity and triangle inequality, Minkowski distance and data transformation, cosine SF, RF on domains with involution (negation), aggregation and transformations of RF, visualization of RF. The paper considers also the lattice of RF, composition and min-transitive transformations of SF (fuzzy proximity relations), applications to hierarchical clustering and non-probabilistic entropy of RF. In addition, the paper proposes the method of construction of correlation functions (association measures) using SF. Pearson correlation and Yule's Q association coefficients obtained as particular cases of the general method. One can use the paper as a survey of works on similarity and dissimilarity measures on specific domains, as a guide for constructing new similarity and correlation measures, as a base for the study of mathematical properties of resemblance functions on universal and specific domains, and also as a part of the course on Data Science. [ABSTRACT FROM AUTHOR]

Abstract Data Mining (DM) is a fundamental component of the Data Science process. Over recent years a huge library of DM algorithms has been developed to tackle a variety of problems in fields such as medical imaging and traffic analysis. Many DM techniques are far more flexible than more classical numerial simulation or statistical modelling approaches. These could be usefully applied to data-rich environmental problems. Certain techniques such as artificial neural networks, clustering, case-based reasoning or Bayesian networks have been applied in environmental modelling, while other methods, like support vector machines among others, have yet to be taken up on a wide scale. There is greater scope for many lesser known techniques to be applied in environmental research, with the potential to contribute to addressing some of the current open environmental challenges. However, selecting the best DM technique for a given environmental problem is not a simple decision, and there is a lack of guidelines and criteria that helps the data scientist and environmental scientists to ensure effective knowledge extraction from data. This paper provides a broad introduction to the use of DM in Data Science processes for environmental researchers. Data Science contains three main steps (pre-processing, data mining and post-processing). This paper provides a conceptualization of Environmental Systems and a conceptualization of DM methods, which are in the core step of the Data Science process. These two elements define a conceptual framework that is on the basis of a new methodology proposed for relating the characteristics of a given environmental problem with a family of Data Mining methods. The paper provides a general overview and guidelines of DM techniques to a non-expert user, who can decide with this support which is the more suitable technique to solve their problem at hand. The decision is related to the bidimensional relationship between the type of environmental system and the type of DM method. An illustrative two way table containing references for each pair Environmental System-Data Mining method is presented and discussed. Some examples of how the proposed methodology is used to support DM method selection are also presented, and challenges and future trends are identified. Highlights • A methodology to support Data Mining (DM) method choice in environmental sciences. • DMMCM is a conceptual map providing an overview of common Data Mining methods. • DMMT provides templates of main DM families, to simplify use by practicioners. • Extensive real environmental applications and 5 detailed case studies are shown. • Existing and future challenges in DM for Data Science are postulated. [ABSTRACT FROM AUTHOR]

Query expansion is a method for improving retrieval performance by supplementing an original query with additional terms. This process improves the quality of search engine results and helps users to find the required information. In the recent years, different methods have been proposed in this area. In addition to such a variety of different approaches in this area and necessity of the study of their characteristics, the lack of a comprehensive classification based on candidate expansion terms extraction methods and also suitable and complete criteria to evaluate them, make the precise study, comparison and evaluation of methods for query expansion and choosing appropriate method based on need difficult for researchers. Therefore, in this paper a new useful framework is presented. In the proposed framework, in addition to the identification of three basic approaches based on the candidate expansion terms extraction methods for query expansion and expressing their properties, appropriate criteria for qualitative evaluation of these methods will be described. Next, the proposed approaches will be evaluated qualitatively based on these criteria. Using the systematic and structured framework proposed in this paper leads a useful platform for researchers to be provided for the comparative study of existing methods in the field, investigating their features specially their drawbacks to improve them and choosing appropriate method based on their needs. [ABSTRACT FROM AUTHOR]

Deep Learning and Machine Learning models have recently been shown to be effective in many real world applications. While these models achieve increasingly better predictive performance, their structures have also become much more complex. A common and difficult problem for complex models is overfitting. Regularization is used to penalize the complexity of the model in order to avoid overfitting. However, in most learning frameworks, regularization function is usually set with some hyper-parameters where the best setting is difficult to find. In this paper, we propose an adaptive regularization method, as part of a large end-to-end healthcare data analytics software stack, which effectively addresses the above difficulty. First, we propose a general adaptive regularization method based on Gaussian Mixture (GM) to learn the best regularization function according to the observed parameters. Second, we develop an effective update algorithm which integrates Expectation Maximization (EM) with Stochastic Gradient Descent (SGD). Third, we design a lazy update and sparse update algorithm to reduce the computational cost by 4x and 20x, respectively. The overall regularization framework is fast, adaptive, and easy-to-use. We validate the effectiveness of our regularization method through an extensive experimental study over 14 standard benchmark datasets and three kinds of deep learning/machine learning models. The results illustrate that our proposed adaptive regularization method achieves significant improvement over state-of-the-art regularization methods. [ABSTRACT FROM AUTHOR]

There have been many statistical tools in the market that are used in data analysis. Each tool is popular in its own feature like cost, visualization, packages, statistics, etc. In this paper, some of the top tools have been taken that are used in data analysis then comparison is done based on some important factors to know the best tool in data science field. The factors that taken into consideration are cost, data handling capabilities, graphical capabilities, big data, etc. Comparison are made on the basis of rating of 1 to 5 and by our own experiences on various data analysis tools. This paper consider the facts that researchers has said in their published papers. [ABSTRACT FROM AUTHOR]

Efficient High Performance Computing for Machine Learning has become a necessity in the past few years. Data is growing exponentially in domains like healthcare, government, economics and with the development of IoT, smartphones and gadgets. This big volume of data, needs a storage space which no traditional computing system can offer, and needs to be fed to Machine Learning algorithms so useful information can be extracted out of it. The larger the dataset that is fed to a Machine Learning algorithm the more precise the results will be, but also the time to compute those results will increase. Thus, the need for Efficient High Performance computing in the aid of faster and better Machine Learning algorithms. This paper aims to unveil how one benefits from another, what research has achieved so far and where is it heading. [ABSTRACT FROM AUTHOR]

This survey is an updated and improved version of the previous one published in 2013 in this journal with the title "data mining in education". It reviews in a comprehensible and very general way how Educational Data Mining and Learning Analytics have been applied over educational data. In the last decade, this research area has evolved enormously and a wide range of related terms are now used in the bibliography such as Academic Analytics, Institutional Analytics, Teaching Analytics, Data‐Driven Education, Data‐Driven Decision‐Making in Education, Big Data in Education, and Educational Data Science. This paper provides the current state of the art by reviewing the main publications, the key milestones, the knowledge discovery cycle, the main educational environments, the specific tools, the free available datasets, the most used methods, the main objectives, and the future trends in this research area. This article is categorized under: Application Areas > Education and Learning [ABSTRACT FROM AUTHOR]

This paper draws on regulatory governance scholarship to argue that the analytic phenomenon currently known as ‘Big Data’ can be understood as a mode of ‘design-based’ regulation. Although Big Data decision-making technologies can take the form of automated decision-making systems, this paper focuses on algorithmic decision-guidance techniques. By highlighting correlations between data items that would not otherwise be observable, these techniques are being used to shape the informational choice context in which individual decision-making occurs, with the aim of channelling attention and decision-making in directions preferred by the ‘choice architect’. By relying upon the use of ‘nudge’ – a particular form of choice architecture that alters people’s behaviour in a predictable way without forbidding any options or significantly changing their economic incentives, these techniques constitute a ‘soft’ form of design-based control. But, unlike the static Nudges popularised by Thaler and Sunstein [(2008).Nudge. London: Penguin Books] such as placing the salad in front of the lasagne to encourage healthy eating, Big Data analytic nudges are extremely powerful and potent due to their networked, continuously updated, dynamic and pervasive nature (hence ‘hypernudge’). I adopt a liberal, rights-based critique of these techniques, contrasting liberal theoretical accounts with selective insights from science and technology studies (STS) and surveillance studies on the other. I argue that concerns about the legitimacy of these techniques are not satisfactorily resolved through reliance on individual notice and consent, touching upon the troubling implications for democracy and human flourishing if Big Data analytic techniques driven by commercial self-interest continue their onward march unchecked by effective and legitimate constraints. [ABSTRACT FROM PUBLISHER]

Big data stream is a new hype but a practical computational challenge founded on data streams that are prevalent in applications nowadays. It is quite well known that data streams that are originated and collected from monitoring sensors accumulate continuously to a very huge amount making traditional batch-based model induction algorithms infeasible for real-time data mining or just-in-time data analytics. In this position paper, following a new data stream mining methodology, namely stream-based holistic analytics and reasoning in parallel (SHARP), a list of data analytic challenges as well as improvised methods are looked into. In particular, two types of decision tree algorithms, batch-mode and incremental-mode, are put under test at sensor data that represents a typical big data stream. We investigate whether and to what extent of two improvised methods-outlier removal and balancing imbalanced class distributions-affect the prediction performance in big data stream mining. SHARP is founded on incremental learning which does not require all the training to be loaded into the memory. This important fundamental concept needs to be supported not only by the decision tree algorithms, but by the other improvised methods usually at the preprocessing stage as well. This paper sheds some light into this area which is often overlooked by data analysts when it comes to big data stream mining. [ABSTRACT FROM AUTHOR]

Big data is big news, and large companies in all sectors are making significant advances in their customer relations, product selection and development and consequent profitability through using this valuable commodity. Small and medium enterprises (SMEs) have proved themselves to be slow adopters of the new technology of big data analytics and are in danger of being left behind. In Europe, SMEs are a vital part of the economy, and the challenges they encounter need to be addressed as a matter of urgency. This paper identifies barriers to SME uptake of big data analytics and recognises their complex challenge to all stakeholders, including national and international policy makers, IT, business management and data science communities. The paper proposes a big data maturity model for SMEs as a first step towards an SME roadmap to data analytics. It considers the 'state-of-the-art' of IT with respect to usability and usefulness for SMEs and discusses how SMEs can overcome the barriers preventing them from adopting existing solutions. The paper then considers management perspectives and the role of maturity models in enhancing and structuring the adoption of data analytics in an organisation. The history of total quality management is reviewed to inform the core aspects of implanting a new paradigm. The paper concludes with recommendations to help SMEs develop their big data capability and enable them to continue as the engines of European industrial and business success. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In recent years technological advances have made large-scale data mining, analytics and forecasting solutions ('Big Data') less costly and more readily available to smaller institutions such as airports. This paper considers how Big Data can be utilised to enhance service and operations within an airport environment and whether using Big Data does in fact add to the bottom line, referencing case studies from Gatwick Airport. The paper suggests that, with the right combination of data analysis/data science capabilities in the organisation and the necessary alignment between operational, commercial and technology functions, Big Data does add to the bottom line, directly and indirectly, through automating decisions, embedding analysis in all functions for decision-making support and enabling wider information flow and collaboration in many areas, one of which is the machine learning required to derive bottom-line value from the Internet of Things. [ABSTRACT FROM AUTHOR]

In recent years, with the rapid development of economy, more and more urban residents, while owning their own motor vehicles, are also troubled by the traffic congestion caused by the backward traffic facilities or traffic management methods. The loss of productivity, car accidents, high emissions, and environmental pollution caused by traffic congestion has become a huge and increasingly heavy burden on all countries in the world. Therefore, the prediction of urban road network traffic flow and the rapid and accurate evaluation of traffic congestion are of great significance to the study of urban traffic solutions. This paper focuses on how to apply data science technologies on vehicular networks data to present a prediction method for traffic congestion based on both real-time and predicted traffic data. Two evaluation frameworks are established, and existing methods are used to compare and evaluate the accuracy and efficiency of the presented method. [ABSTRACT FROM AUTHOR]

Text mining have gained great momentum in recent years, with user-generated content becoming widely available. One key use is comment mining, with much attention being given to sentiment analysis and opinion mining. An essential step in the process of comment mining is text pre-processing; a step in which each linguistic term is assigned with a weight that commonly increases with its appearance in the studied text, yet is offset by the frequency of the term in the domain of interest. A common practice is to use the well-known tf-idf formula to compute these weights. This paper reveals the bias introduced by between-participants’ discourse to the study of comments in social media, and proposes an adjustment. We find that content extracted from discourse is often highly correlated, resulting in dependency structures between observations in the study, thus introducing a statistical bias. Ignoring this bias can manifest in a non-robust analysis at best and can lead to an entirely wrong conclusion at worst. We propose an adjustment to tf-idf that accounts for this bias. We illustrate the effects of both the bias and correction with with seven Facebook fan pages data, covering different domains, including news, finance, politics, sport, shopping, and entertainment. [ABSTRACT FROM AUTHOR]

Abstract Time series classification is one of the most important issues in time series data mining. This problem has attracted more and more attention of researchers in recent years. Among proposed methods in literature, 1-Nearest Neighbor (1-NN), its variants and improvements have been widely considered as hard to be beaten on classification of time series. In this paper, we propose a novel non-parametric method to classify time series. The proposed method, namely Weighted Local Dynamic Time Warping Barycenter Averaging k -Nearest Neighbors (WLDBA k -NN), is an improvement of Local Mean-based k -Nearest Neighbors (LM k -NN) algorithm. It improves LM k -NN in that it replaces the local mean vectors by local Dynamic Time Warping Barycenter (DBA) vectors calculated using our method, namely Weighted DBA (WDBA). By experiments, we show that (i) WLDBA k -NN outperforms the Weighted Local Mean-based k -Nearest Neighbors (WLM k -NN) algorithm, and (ii) both WLM k -NN and WLDBA k -NN outperform 1-NN, LM k -NN, k -Nearest Centroid Neighbors (k -NCN), and LM k -NCN in 85 time series datasets of UCR Time Series Classification Archive. The experimental results also show that new local mean vectors used in WLM k -NN and WLDBA k -NN significantly contribute to the improvement of the performance of time series classification. [ABSTRACT FROM AUTHOR]

This paper describes a collection of data-driven aesthetic explorations that investigate the concept of 'cycling as art practice' made possible through the use of self-tracking tools while journeying by bicycle into the landscape. The explorations draw upon the philosophy of the 'Walking Artists' and the concept of 'the dematerialization of the art object' as counterpoints to the Quantified Self Community, which aims to visualize previously invisible aspects of our daily lives for the purposes of positive self-monitoring. In doing so, it begins to draw attention aesthetically, and philosophically, to the way in which representations of our experiences, particularly of landscape, can be formed through the use of such technologies for generative art purposes. [ABSTRACT FROM AUTHOR]

Nowadays, cloud manufacturing (CMfg) as a new service-oriented manufacturing mode has been paid wide attention around the world. However, one of the key technologies for better implementing CMfg services is how to address the problems faced by distributed massive and polymorphic data analysis and processing of manufacturing resource services in CMfg environment. In order to handle the problems, and promote development integration and business application of data mining, the research on technologies and application of data mining for cloud manufacturing resource (CMR) services has been carried out in this paper. The data mining application model and multi hierarchy architecture of CMR services are designed, and the topology of resource service data integrating process based on multi-Agent is presented. Besides, the preprocessing method of manufacturing resource data is proposed, and a CMR virtual data warehouse is established as well. In addition, an improved genetic algorithm oriented to manufacturing resource services is put forward, so as to achieve efficient searching and mining of massive polymorphic data. Finally, a case study is employed to illustrate the effectiveness and applicability of the proposed method in CMR service platform. [ABSTRACT FROM AUTHOR]

Data collected in psychological studies are mainly characterized by containing a large number of variables (multidimensional data sets). Analyzing multidimensional data can be a difficult task, especially if only classical approaches are used (hypothesis tests, analyses of variance, linear models, etc.). Regarding multidimensional models, visual techniques play an important role because they can show the relationships among variables in a data set. Parallel coordinates and Chernoff faces are good examples of this. This article presents self-organizing maps (SOM), a multivariate visual data mining technique used to provide global visualizations of all the data. This technique is presented as a tutorial with the aim of showing its capabilities, how it works, and how to interpret its results. Specifically, SOM analysis has been applied to analyze the data collected in a study on the efficacy of a cognitive and behavioral treatment (CBT) for childhood obesity. The objective of the CBT was to modify the eating habits and level of physical activity in a sample of children with overweight and obesity. Children were randomized into two treatment conditions: CBT traditional procedure (face-to-face sessions) and CBT supported by a web platform. In order to analyze their progress in the acquisition of healthier habits, self-register techniques were used to record dietary behavior and physical activity. In the traditional CBT condition, children completed the self-register using a paper-and-pencil procedure, while in the web platform condition, participants completed the self-register using an electronic personal digital assistant. Results showed the potential of SOM for analyzing the large amount of data necessary to study the acquisition of new habits in a childhood obesity treatment. Currently, the high prevalence of childhood obesity points to the need to develop strategies to manage a large number of data in order to design procedures adapted to personal characteristics and increase treatment efficacy. [ABSTRACT FROM AUTHOR]

In this paper, we review recent progresses in the area of mining data from multiple data sources. The advancement of information communication technology has generated a large amount of data from different sources, which may be stored in different geological locations. Mining data from multiple data sources to extract useful information is considered to be a very challenging task in the field of data mining, especially in the current big data era. The methods of mining multiple data sources can be divided mainly into four groups: (i) pattern analysis, (ii) multiple data source classification, (iii) multiple data source clustering, and (iv) multiple data source fusion. The main purpose of this review is to systematically explore the ideas behind current multiple data source mining methods and to consolidate recent research results in this field. [ABSTRACT FROM AUTHOR]

Process mining techniques analyze processes based on events stored in event logs. Yet, low-level events recorded by information systems may not directly match high-level activities that make sense to process stakeholders. This results in discovered process models that cannot be easily understood. To prevent such situations from happening, low-level events need to be translated into high-level activities that are recognizable by stakeholders. This paper proposes the Guided Process Discovery method (GPD). Low-level events are grouped based on behavioral activity patterns, which capture domain knowledge on the relation between high-level activities and low-level events. Events in the resulting abstracted event log correspond to instantiations of high-level activities. We validate process models discovered on the abstracted event log by checking conformance between the low-level event log and an expanded model in which the high-level activities are replaced by activity patterns. The method was tested using two real-life event logs. We show that the process models discovered with the GPD method are more comprehensible and can be used to answer process questions, whereas process models discovered using standard process discovery techniques do not provide the insights needed. [ABSTRACT FROM AUTHOR]

Data science or "data-driven research" is a research approach that uses real-life data to gain insight about the behavior of systems. It enables the analysis of small, simple as well as large and more complex systems in order to assess whether they function according to the intended design and as seen in simulation. Data science approaches have been successfully applied to analyze networked interactions in several research areas such as large-scale social networks, advanced business and healthcare processes. Wireless networks can exhibit unpredictable interactions between algorithms from multiple protocol layers, interactions between multiple devices, and hardware specific influences. These interactions can lead to a difference between real-world functioning and design time functioning. Data science methods can help to detect the actual behavior and possibly help to correct it. Data science is increasingly used in wireless research. To support data-driven research in wireless networks, this paper illustrates the step-by-step methodology that has to be applied to extract knowledge from raw data traces. To this end, the paper (i) clarifies when, why and how to use data science in wireless network research; (ii) provides a generic framework for applying data science in wireless networks; (iii) gives an overview of existing research papers that utilized data science approaches in wireless networks; (iv) illustrates the overall knowledge discovery process through an extensive example in which device types are identified based on their traffic patterns; (v) provides the reader the necessary datasets and scripts to go through the tutorial steps themselves. [ABSTRACT FROM AUTHOR]

Data, information, knowledge, and wisdom are four basic steps of human perception process of objects. In order to better understand agricultural drought and make proper decisions, it is necessary to extract drought information out of related data (e.g., remotely sensed data) and discover knowledge from the extracted information. This paper explores advantages of Web services in providing on-demand agricultural drought analysis and facilitating the perception process in agricultural drought management. Four Web services, drawROI, getVCIStats, getDroughtPercentageByStates, and getDroughtTimeSeries, are presented in details in this paper. These Web services demonstrate improved support to drought analysis and decision-making for the general public and illustrate the potential of Web services in automating geospatial knowledge discovery and dissemination in the Big Data era. [ABSTRACT FROM AUTHOR]

Physical unclonable functions (PUFs) are circuits that produce outputs determined by random physical variations from fabrication. The PUF studied in this paper utilizes the variation sensitivity of static random access memory (SRAM) data retention voltage (DRV), the minimum voltage at which each cell can retain state. Prior work shows that DRV can uniquely identify circuit instances with 28% greater success than SRAM power-up states that are used in PUFs [1]. However, DRV is highly sensitive to temperature, and until now this makes it unreliable and unsuitable for use in a PUF. In this paper, we enable DRV PUFs by proposing a DRV-based hash function that is insensitive to temperature. The new hash function, denoted DRV-based hashing (DH), is reliable across temperatures because it utilizes the temperature-insensitive ordering of DRVs across cells, instead of using the DRVs in absolute terms. To evaluate the security and performance of the DRV PUF, we use DRV measurements from commercially available SRAM chips, and use data from a novel DRV prediction algorithm. The prediction algorithm uses machine learning for fast and accurate simulation-free estimation of any cell’s DRV, and the prediction error in comparison to circuit simulation has a standard deviation of 0.35 mV. We demonstrate the DRV PUF using two applications—secret key generation and identification. In secret key generation, we introduce a new circuit-level reliability knob as an alternative to error correcting codes. In the identification application, our approach is compared to prior work and shown to result in a smaller false-positive identification rate for any desired true-positive identification rate. [ABSTRACT FROM PUBLISHER]

The rapid development in information and computer technology has facilitated an extreme increase in the collection and storage of digital data. However, the associated rapid increase in digital data volumes does not automatically correlate with new insights and advances in our understanding of those data. The relatively new technique of data mining offers a promising way to extract knowledge and patterns from large, multidimensional and complex data sets. This paper therefore aims to provide a comprehensive overview of existing data mining techniques and related tools and to illustrate the potential of data mining for different research areas by means of example applications. Despite a number of conventional data mining techniques and methods, these classical approaches are restricted to isolated or “silo” data sets and therefore remain primarily stand alone and specialized in nature. Highly complex and mostly interdisciplinary questions in environmental research cannot be answered sufficiently using isolated or area-based data mining approaches. To this end, the linked open data (LOD) approach will be presented as a new possibility in support of complex and inter-disciplinary data mining analysis. The merit of LOD will be explained using examples from medicine and environmental research. The advantages of LOD data mining will be weighed against classical data mining techniques. LOD offers unique and new possibilities for interdisciplinary data analysis, modeling and projection for multidimensional, complex landscapes and may facilitate new insights and answers to complex environmental questions. Our paper aims to encourage those research scientists which do not have extensive programming and data mining knowledge to take advantage of existing data mining tools, to embrace classical data mining and LOD approaches in support of gaining more insight and recognizing patterns in highly complex data sets. [ABSTRACT FROM AUTHOR]

The aim of this paper is to improve the classification performance based on the multiclass imbalanced datasets. In this paper, we introduce a new resampling approach based on Clustering with sampling for Multiclass Imbalanced classification using Ensemble (C-MIEN). C-MIEN uses the clustering approach to create a new training set for each cluster. The new training sets consist of the new label of instances with similar characteristics. This step is applied to reduce the number of classes then the complexity problem can be easily solved by C-MIEN. After that, we apply two resampling techniques (oversampling and undersampling) to rebalance the class distribution. Finally, the class distribution of each training set is balanced and ensemble approaches are used to combine the models obtained with the proposed method through majority vote. Moreover, we carefully design the experiments and analyze the behavior of C-MIEN with different parameters (imbalance ratio and number of classifiers). The experimental results show that C-MIEN achieved higher performance than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the advent of cost effective storage systems and high speed network connectivity, the amount of data gathered by various transactional systems has increased manifold and processing the same has become a real challenge. A number of data mining systems have been developed for processing such huge data in a meaningful way. Most of these systems are stand-alone in nature except a few of them that facilitate a subset of their overall functionality for web-based usage. This is mainly due to the challenges involved in designing the architectural framework required for developing a web-enabled data mining system. Such a system is aimed at analysing the transactional data pertaining to various input domains employing some advanced graph mining techniques like subgraph matching, frequent subgraph discovery and graph visualization. In this paper, we present an innovative approach employing various Java Enterprise Edition (JEE) based technologies for developing a data mining system that operates in the web environment. This paper basically presents the architectural principles required to design a suitable framework for enabling the development of such a system and the implementation challenges faced in realising the same. The paper also discusses functional details of the system involving various graph mining techniques. A few general guidelines based on our understanding during the system implementation are also included in this paper for completeness. [ABSTRACT FROM AUTHOR]

Image authentication receives growing awareness because digital images can be easily modified and the modification is hard to detect. A reversible image authentication method that can improve the accuracy of tamper detection and the quality of watermarking image is presented in this paper. In the proposed method, reversible data hiding is implemented with two identical host images, where the secret information is embedded in one host image while the distortion information is embedded in the other image. In the authentication process, we compared the information extracted from the image with the original authentication information to judge whether the image was tampered with. Experimental results show that the algorithm can improve the accuracy of tamper detection and guarantee the image quality while a large number of authentication information is embedded. [ABSTRACT FROM AUTHOR]