Search

Network attacks are becoming more complex, making it more difficult to detect intrusions. Various research have been done over the years, employing different categorization techniques of Data Mining (DM) and Machine Learning (ML) inspired hybrid approaches to develop robust IDS. Almost all researchers suggested to improve accuracy in intrusion detection with low computational cost. Authors observed that dissimilar sets of features were picked for different classifiers to get the highest accuracy. This paper is dedicated to a review of relevant research, where an in-depth investigation was carried out with two emphasis points of IDS, which contain distinct pre-processing techniques in the form of feature selection and a diversity of classification algorithms. In addition, this paper presents a comparative algorithmic assessment of the DM and ML techniques applied to create an intelligent IDS. A novel feature selection method based on the CART algorithm has also introduced which provides optimal feature subset of the dataset so perfectly that it has made various existing DM and ML classifying algorithms more performant than earlier, this makes classifiers independent of feature selection. To validate the performance of proposed work, experiments have performed using the 'Python' programming language and 'corrected' & '10_percent' of 'Kddcup99' datasets used as benchmark. As an outcome proposed work, we found that feature reduction and selecting a classifier had a significant influence on the rate of intrusion detection accuracy. The results of simulation and comparison analysis of proposed work with existing DM and ML based classification approaches show that the suggested work is more competent in true prediction and attaining maximum intrusion detection accuracy with minimal computing cost of prediction. [ABSTRACT FROM AUTHOR]

The amount of digital data in the universe is growing at an exponential rate with the rapid development of digital information, and this reveals new machine learning methods. Learning algorithms using hyperboxes are a subsection of machine learning methods. Fuzzy min–max neural network (FMNN) are one of the most common and advanced methods using hyperboxes. FMNN is a special type of NeuroFuzzy system that combines the artificial neural network and fuzzy set into a common framework. This paper conducts an extensive bibliometric and network analysis of FMNN literature. Two hundred and sixty-two publications are analysed from the period of 1992–2022. Several analyses are realized in order to identify trends, challenges and key points in a more scientific and objective way that affect the development of knowledge in the FMNN domain. It can be seen from bibliometric analysis that there is rapid development in the last 10 years. Social network analysis results show that Chee Peng Lim is the most active author in the network. Besides, the modifications of FMNN are generally developed for classification. However, there are still potential future research opportunities for clustering. [ABSTRACT FROM AUTHOR]

Healthcare is evolving from standard to personalized, driven by the patients' needs. Personalized healthcare is a medical model based on genetics, genomics, and other biological information that helps to predict risk for disease. To date, machine learning and data mining are the fastest-growing healthcare field used to classify patient cohorts from a large dataset and its application for diabetes subtyping will be a breakthrough. In this review paper, we have identified, analyzed, and summarized how previous studies distinguished diabetes into subtypes besides implementing the methods for diabetes subtyping using data mining and various clustering algorithms. We have discovered that many studies have suggested diabetes can be differentiated into subtypes clinically based on the risk complications, genetically defined, using clinical features, and for treatment selection. As for clustering algorithms, k-means clustering and hierarchical clustering were shown to be widely used in determining sub-clusters of diabetes. To further investigate diabetes subtyping, understanding the specific objective and method of diabetes subtyping using clustering algorithms from a large dataset will be crucial which could contribute to novel knowledge and improvement for diabetes management. [ABSTRACT FROM AUTHOR]

Density-based clustering has received increasing attention for its ability to handle clusters of arbitrary shapes. However, it still has difficulties in mining clusters of arbitrary densities, especially the clusters of sparse regions in the presence of dense regions. To address this problem, this paper presents a new concept called density decreased chain on the mutual k-NN graph. It starts with the local density center whose density is the highest in the data points connected to this center. Based on the density decreased chain, the concept of the core point is redefined. The density of the core point is close to that of the local density center on the same density decreased chain as the core point. According to its definition, the core point in the data with arbitrary densities can be well identified because the local density centers exist in both sparse and dense regions. Further, intra-cluster density decreased chain is defined to mine subclusters in the core points. After forming the subclusters, the remaining data point is hierarchically assigned to one of these subclusters by the density decreased chains containing this remaining data point. The experiments illustrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

For scholarly organizations, students' academic performance (AP) computes student achievements in different academic subjects. Therefore, a systematic literature review based on machine learning approaches to improve student performance is proposed. This field creates a way to discover hidden examples from instructive information. Machine learning (ML) techniques are used for performance prediction. It is become a challenge due to its imbalanced dataset. This review aims to identify the best proposals that focus on various ML methods used for the performance analysis of students. This review will become helpful to the teachers in identifying the weak students, and it helps to improve their performance through proper guidance. Thus, it reflects in the student's background and boosts their growth. It also brings benefits to students, teachers, and institutions. This study focused on applying machine learning techniques to predict students' performance in recent times. With a systematic approach, the research identified the existing prediction methods and tools used to predict students' performance and observed the researchers' type of variables in this research area. In this study, almost more than 100 papers were analyzed to reveal various modern techniques that are commonly used to predict student performance and the goals they need to achieve in this field. The results from the various research will help improve students' academics and monitor the student's performance, which would also improve their literacy rate. [ABSTRACT FROM AUTHOR]

Rapid pavement deterioration is a major problem in areas with harsh weather exposure and/or high truck traffic loading. While several studies in recent years have focused on pavement deterioration and management systems, there is not, to date, a robust method explaining how large amounts of pavement data can be processed to identify homogeneous groups for the calibration of pavement performance models to support prioritization of maintenance, rehabilitation, or design of new pavements. This study employs machine learning to develop an approach, capable of partitioning pavement data with a close response to causal factors including traffic and weather conditions; and considering pavement performance through factors including international roughness index and deflections. Two classes of clustering, i.e. soft and hard partitioning, through K-means and Self Organizing Maps—SOM respectively, were tested to understand the correlation between daily factors and pavements deterioration. The goodness of clustering was tested using extrinsic and intrinsic evaluation methods. It was concluded from the results that SOM clustering provided reliable results, as it employs a soft clustering method, where data points can belong to more than one cluster at the same time. The results allowed better visualization of data points, identifying hidden relationships among clusters, and exploring correlations between external effects and homogeneous families. Moreover, it became apparent that unlike the common Markovian methods, including previous years' condition in the analysis of clusters has marginal to no effects on the detection of homogeneous grouping. The methodology proposed in this paper will produce more reliable pavement homogenous groups as input for performance models and decision-making for the maintenance and rehabilitation of pavements. Moreover, future researchers can use the results of this study to prepare a standardized design for pavement groups. [ABSTRACT FROM AUTHOR]

Although data mining problems require a flat mining table as input, in many real-world applications analysts are interested in finding patterns in a relational database. To this end, new methods and software have been recently developed that automatically add attributes (or features) to a target table of a relational database which summarize information from all other tables. When attributes are automatically constructed by these methods, selecting the important attributes is particularly difficult, because a large number of the attributes are highly correlated. In this setting, attribute selection techniques such as the Least Absolute Shrinkage and Selection Operator (lasso), elastic net, and other machine learning methods tend to under-perform. In this paper, we introduce a novel attribute selection procedure, where after an initial screening step, we cluster the attributes into different groups and apply the group lasso to select both the true attributes groups and then the true attributes. The procedure is particularly suited to high dimensional data sets where the attributes are highly correlated. We test our procedure on several simulated data sets and a real-world data set from a marketing database. The results show that our proposed procedure obtains a higher predictive performance while selecting a much smaller set of attributes when compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Discovering the similar groups is a popular primary step in analysis of biomedical data, which cannot be identified manually. Many supervised and unsupervised machine learning and statistical approaches have been developed to solve this problem. Clustering is an unsupervised learning approach, which organizes the data into similar groups, and is used to discover the intrinsic hidden structure of data. In this paper, we used clustering by fast search and find of density peaks (CDP) approach for cancer subtyping and identification of normal tissues from tumor tissues. In additional, we also address the preprocessing and underlying distance matrix’s impact on finalized groups. We have performed extensive experiments on real-world and synthetic cancer gene expression microarray data sets and compared obtained results with state-of-the-art clustering approaches. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization and its graph regularized extensions have received significant attention in machine learning and data mining. However, existing approaches are sensitive to outliers and noise due to the utilization of the squared loss function in measuring the quality of graph regularization and data reconstruction. In this paper, we present a novel robust graph regularized NMF model (RGNMF) to approximate the data matrix for clustering. Our assumption is that there may exist some entries of the data corrupted arbitrarily, but the corruption is sparse. To address this problem, an error matrix is introduced to capture the sparse corruption. With this sparse outlier matrix, a robust factorization result could be obtained since a much cleaned data could be reconstructed. Moreover, the ℓ1-norm function is used to alleviate the influence of unreliable regularization which is incurred by unexpected graphs. That is, the sparse error matrix alleviates the impact of noise and outliers, and the ℓ1-norm function leads to a faithful regularization since the influence of the unreliable regularization errors can be reduced. Thus, RGNMF is robust to unreliable graphs and noisy data. In order to solve the optimization problem of our method, an iterative updating algorithm is proposed and its convergence is also guaranteed theoretically. Experimental results show that the proposed method consistently outperforms many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The massive growth of data in recent years has led challenges in data mining and machine learning tasks. One of the major challenges is the selection of relevant features from the original set of available features that maximally improves the learning performance over that of the original feature set. This issue attracts researchers' attention resulting in a variety of successful feature selection approaches in the literature. Although there exist several surveys on unsupervised learning (e.g., clustering), lots of works concerning unsupervised feature selection are missing in these surveys (e.g., evolutionary computation based feature selection for clustering) for identifying the strengths and weakness of those approaches. In this paper, we introduce a comprehensive survey on feature selection approaches for clustering by reflecting the advantages/disadvantages of current approaches from different perspectives and identifying promising trends for future research. [ABSTRACT FROM AUTHOR]

Machine Learning is currently an important research field that attracts interest due to its importance for discovering hidden knowledge or patterns from big datasets. In this paper, we propose a heuristic algorithm which can solve problems related to only classification, only clustering, or classification with clustering by creating models with the ability to evolve to another class/cluster configuration without a retraining process for new incoming data. This algorithm combines supervised and unsupervised learning principles for the incremental construction of both classes and clusters, by using the main guidelines from two classical methods of classification based on distance and clustering based on prototypes, such as KNN and K-means. The algorithm is able to deal with labeled and unlabeled samples as inputs in order to create new groups (classes or clusters), merge or reconfigure existing ones. Basically, the creation of new groups follows three sequential steps: (i) locate the provisional group for an input sample using K-means, (ii) using 1NN, locate the nearest sample to the input sample, only considering the samples in the provisional group, and (iii) merge or reconfigure existing groups following specific guidelines. Several benchmarks, related to classification and clustering problems, were evaluated by our proposal; the results were compared with classical algorithms. On the other hand, artificial datasets with labeled and unlabeled samples have been created to show the ability of our algorithm in the hybrid context to solve classification and clustering combined. As a result, the algorithm is able to create clusters and classes, simultaneously, when required. Finally, a real case study of fault diagnosis in rotating machinery is presented for discovering new groups that might represent patterns from unknown data. [ABSTRACT FROM AUTHOR]

Machine Learning techniques have been useful in almost every field of concern. Data Mining, a branch of Machine Learning is one of the most extensively used techniques. The ever-increasing demands in the field of medicine are being addressed by computational approaches in which Big Data analysis, image processing and data mining are on top priority. These techniques have been exploited in the domain of ophthalmology for better retinal fundus image analysis. Blood vessels, one of the most significant retinal anatomical structures are analysed for diagnosis of many diseases like retinopathy, occlusion and many other vision threatening diseases. Vessel segmentation can also be a pre-processing step for segmentation of other retinal structures like optic disc, fovea, microneurysms, etc. In this paper, blood vessel segmentation is attempted through image processing and data mining techniques. The retinal blood vessels were segmented through color space conversion and color channel extraction, image pre-processing, Gabor filtering, image postprocessing, feature construction through application of principal component analysis, k-means clustering and first level classification using Naïve-Bayes classification algorithm and second level classification using C4.5 enhanced with bagging techniques. Association of every pixel against the feature vector necessitates Big Data analysis. The proposed methodology was evaluated on a publicly available database, STARE. The results reported 95.05% accuracy on entire dataset; however the accuracy was 95.20% on normal images and 94.89% on pathological images. A comparison of these results with the existing methodologies is also reported. This methodology can help ophthalmologists in better and faster analysis and hence early treatment to the patients. [ABSTRACT FROM AUTHOR]

The minimum number attribute reduction problem is an important issue when dealing with huge amounts of data. The problem of minimum attribute reduction is formally known to be as an NP complete nonlinearly constrained optimization problem. Social spider optimization algorithm is a new meta-heuristic algorithm of the swarm intelligence field to global solution. The social spider optimization algorithm is emulates the behavior of cooperation between spiders based on the biological laws of the cooperative colony. Inspired by the social spiders, in this paper, an improved social spider algorithm for the minimal reduction problem was proposed. In the proposed algorithm, the fitness function depends on the rough sets dependency degree and it takes into a consideration the number of selected features. For each spider, the fitness function is computed and compared with the global best fitness value. If the current value is better, then the global best fitness is replaced with it and its position became the reduct set. Then, the position of each spider is updated according to its type. This process is repeated until the stopping criterion is satisfied. To validate the proposed algorithm, several real clinical medical datasets which are available from the UCI Machine Learning Repository were used to compute the performance of the proposed algorithm. The experimental results illustrate that the proposed algorithm is superior to state-of-the-art swarm-based in terms of classification accuracy while limiting number of features. [ABSTRACT FROM AUTHOR]

Clustering, as an unsupervised learning method and an important process in data mining, is an aspect of large and distributed data analysis. In many applications, such as peer-to-peer systems, huge volumes of data are distributed between multiple sources. Analysis of these volumes of data and identifying appropriate clusters is challenging due to transmission, processing and storage costs. In this paper, a gossip-based distributed clustering algorithm for P2P networks called Efficient GBDC-P2P is proposed, based on an improved gossip communicative approach by combining the peer sampeling and CYCLON protocol and the idea of partitioning-based data clustering. This algorithm is appropriate for data clustering in unstructured P2P networks, and it is adapted to the dynamic conditions of these networks. In the Efficient GBDC-P2P algorithm, distributed peers perform clustering operation in a distributed way only through local communications with their neighbors. Our approach does not rely on the central server to carry out data clustering task and without the need to synchronize operations. Evaluation results verify the efficiency of our proposed algorithm for data clustering in unstructured P2P networks. Furthermore, comparative analyses with other well-established distributed clustering approaches demonstrate the superior accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

Clustering is a powerful and versatile tool for knowledge discovery, able to provide a valuable information for data analysis in various domains. To perform this task based on streaming data is quite challenging: outdated knowledge needs to be disposed while the current knowledge is obtained from fresh data; since data are continuously flowing, strict efficiency constraints have to be met. This paper presents WCDS, an approach to this problem based on the WiSARD artificial neural network model. This model already had useful characteristics as inherent incremental learning capability and patent functioning speed. These were combined with novel features as an adaptive countermeasure to cluster imbalance, a mechanism to discard expired data, and offline clustering based on a pairwise similarity measure for WiSARD discriminators. In an insightful experimental evaluation, the proposed system had an excellent performance according to multiple quality standards. This supports its applicability for the analysis of data streams. [ABSTRACT FROM AUTHOR]

Data quality issues have attracted widespread attentions due to the negative impacts of dirty data on data mining and machine learning results. The relationship between data quality and the accuracy of results could be applied on the selection of the appropriate model with the consideration of data quality and the determination of the data share to clean. However, rare research has focused on exploring such relationship. Motivated by this, this paper conducts an experimental comparison for the effects of missing, inconsistent, and conflicting data on classification and clustering models. From the experimental results, we observe that dirty-data impacts are related to the error type, the error rate, and the data size. Based on the findings, we suggest users leverage our proposed metrics, sensibility and data quality inflection point, for model selection and data cleaning. [ABSTRACT FROM AUTHOR]

The application of ACO-based algorithms in data mining has been growing over the last few years, and several supervised and unsupervised learning algorithms have been developed using this bio-inspired approach. Most recent works about unsupervised learning have focused on clustering, showing the potential of ACO-based techniques. However, there are still clustering areas that are almost unexplored using these techniques, such as medoid-based clustering. Medoid-based clustering methods are helpful-compared to classical centroid-based techniques-when centroids cannot be easily defined. This paper proposes two medoid-based ACO clustering algorithms, where the only information needed is the distance between data: one algorithm that uses an ACO procedure to determine an optimal medoid set (METACOC algorithm) and another algorithm that uses an automatic selection of the number of clusters (METACOC-K algorithm). The proposed algorithms are compared against classical clustering approaches using synthetic and real-world datasets. [ABSTRACT FROM AUTHOR]

Nonnegative Matrix Factorization (NMF) plays an important role in many data mining and machine learning tasks. Standard NMF uses the Frobenius norm as the loss function which is well-known to be sensitive to noise. To address this issue, we propose a robust formulation of NMF, i.e., Cauchy-NMF, which is derived based on the assumption that the noise generally follows identical independent distributed (i.i.d.) Cauchy distribution. In particular, we derive the Cauchy Balanced NMF model (Cauchy-B-NMF) using Cauchy distribution, where (a) the numerical value of each element in the coefficient matrix is viewed as the posterior probability, which allows the clustering result to be obtained directly from the coefficient matrix without any additional post-processing; (b) a novel manifold regularization term is incorporated into the loss function, explicitly making the distant data points have dissimilar embeddings, while implicitly making the neighbouring data points have similar embeddings; (c) a balanced clustering term is enforced to achieve the desired equal number of data points across different clusters. We derive an efficient computational algorithm to solve the resultant optimization problem, and also provide a rigorous analysis of the algorithm convergence. Experimental results on several benchmarks demonstrate the effectiveness of our algorithms, which consistently provides better clustering results compared to many other NMF variants. [ABSTRACT FROM AUTHOR]

The data on 178 samples of Italian red wines taken from the public machine learning repository UCI have been studied. Computer analysis of 13 physico-chemical properties of the wine samples on their distribution between three groups has been performed via different data mining methods. Classification models: factor, discriminant, canonical, and neural network (multilayer perceptron MLP, Kohonen's map SOFM) ones, predicting models (support vector machine, Bayesian classifier, and nearest neighbor method), and decision trees have been built. The neural network classifiers SOFM 13-3, MLP 13-5-3, and SOFM 16-3 have been trained. It has been found that predicting power of the models is determined by the following variables: proline, flavonoids, color intensity, proteins, and alcohol. [ABSTRACT FROM AUTHOR]

The k-means algorithm is widely used for clustering, compressing, and summarizing vector data. We present a fast and memory-efficient GPU-based algorithm for exact k-means, Asynchronous Selective Batched K-means (ASB K-means). Unlike most GPU-based k-means algorithms that require loading the whole dataset onto the GPU for clustering, the amount of GPU memory required to run our algorithm can be chosen to be much smaller than the size of the whole dataset. Thus, our algorithm can cluster datasets whose size exceeds the available GPU memory. The algorithm works in a batched fashion and applies the triangle inequality in each k-means iteration to omit a data point if its membership assignment, i.e., the cluster it belongs to, remains unchanged, thus significantly reducing the number of data points that need to be transferred between the CPU's RAM and the GPU's global memory and enabling the algorithm to very efficiently process large datasets. Our algorithm can be substantially faster than a GPU-based implementation of standard k-means even in situations when application of the standard algorithm is feasible because the whole dataset fits into GPU memory. Experiments show that ASB K-means can run up to 15x times faster than a standard GPU-based implementation of k-means, and it also outperforms the GPU-based k-means implementation in NVIDIA's open-source RAPIDS machine learning library on all the datasets used in our experiments. [ABSTRACT FROM AUTHOR]

Technology and data analysis have evolved into a resource-rich tool for collecting, researching and comparing student achievement levels in the classroom. There are sufficient resources to discover student success through data analysis by routinely collecting extensive data on student behaviour and curriculum structure. Educational Data Mining (EDM), a method of data analysis in the learning environment, has emerged as an emerging trend in the development of educational data mining and analysis techniques. EDM aids in the comprehension of student behaviour as well as the factors that influence student behaviour and achievement. Student learning patterns, student culture, and instructional skills are all important factors in a successful study of EDM students. This study will look at how technology and data mining are used in the EDM environment and compare the results. We have used previous research to determine which method is best for observing the learning environment and what factors influence student academic performance. Two state-of-the-art models i.e. decision tree (classifier) and DBSCAN (clustering method) are used to predict the performance of an educational institute with higher accuracy. [ABSTRACT FROM AUTHOR]

Innovation and adoption of new technologies in health care industries produce vast data every day. The diverse data in health care include clinical data, health history, and genetic data. Also, real-time monitoring in health care generates huge data, and efficiently examining these big data is a challenging task. Analysis of health care data becomes more important so that proper medications can be provided and issues can be reduced by taking proper precautions based on the history. Data analysis becomes efficient because of automation, however, due to data integrity, data diversity, and inconsistency, the performance gets lagged. Various machine learning models are introduced to handle big data management; however, researchers are still working to attain a better model with improved accuracy. So with the objective to attain maximum classification accuracy, fuzzy c means clustering and generative adversarial network are employed in this research work for health care data clustering and classification. Benchmark lung cancer dataset and Arrhythmia dataset are used in the experimentation. The proposed model exhibits the maximum accuracy of 97.8% for dataset 1 and 98.6% for dataset 2 compared to existing techniques like support vector machine, decision tree, and random forest algorithms. [ABSTRACT FROM AUTHOR]

Text document classification and clustering is an important learning task which fits to both data mining and machine learning areas. The learning task throws several challenges when it is required to process high dimensional text documents. Word distribution in text documents plays a very key role in learning process. Research related to high dimensional text document classification and clustering is usually limited to application of traditional distance functions and most of the research contributions in the existing literature did not consider the word distribution in documents. In this research, we propose a novel similarity function for feature pattern clustering and high dimensional text classification. The similarity function proposed is used to carry supervised learning based dimensionality reduction. The important feature of this work is that the word distribution before and after dimensionality reduction is the same. Experiment results prove the proposed approach achieves dimensionality reduction, retains the word distribution and obtained better classification accuracies compared to other measures. [ABSTRACT FROM AUTHOR]

The use of distance metrics such as the Euclidean or Manhattan distance for nearest neighbour algorithms allows for interpretation as a geometric model, and it has been widely assumed that the metric axioms are a necessary condition for many data mining tasks. We show that this assumption can in fact be an impediment to producing effective models. We propose to use mass-based dissimilarity, which employs estimates of the probability mass to measure dissimilarity, to replace the distance metric. This substitution effectively converts nearest neighbour (NN) algorithms into lowest probability mass neighbour (LMN) algorithms. Both types of algorithms employ exactly the same algorithmic procedures, except for the substitution of the dissimilarity measure. We show that LMN algorithms overcome key shortcomings of NN algorithms in classification and clustering tasks. Unlike existing generalised data independent metrics (e.g., quasi-metric, meta-metric, semi-metric, peri-metric) and data dependent metrics, the proposed mass-based dissimilarity is unique because its self-dissimilarity is data dependent and non-constant. [ABSTRACT FROM AUTHOR]

We consider the problem of clustering a set of points so as to minimize the maximum intra-cluster dissimilarity, which is strongly NP-hard. Exact algorithms for this problem can handle datasets containing up to a few thousand observations, largely insufficient for the nowadays needs. The most popular heuristic for this problem, the complete-linkage hierarchical algorithm, provides feasible solutions that are usually far from optimal. We introduce a sampling-based exact algorithm aimed at solving large-sized datasets. The algorithm alternates between the solution of an exact procedure on a small sample of points, and a heuristic procedure to prove the optimality of the current solution. Our computational experience shows that our algorithm is capable of solving to optimality problems containing more than 500,000 observations within moderate time limits, this is two orders of magnitude larger than the limits of previous exact methods. [ABSTRACT FROM AUTHOR]