Search

In expert systems, data mining methods are algorithms that simulate humans’ problem-solving capabilities. Clustering methods as unsupervised machine learning methods are crucial approaches to categorize similar samples in the same categories. The use of different clustering algorithms to a given dataset produces clusters with different qualities. Hence, many researchers have applied clustering combination methods to reduce the risk of choosing an inappropriate clustering algorithm. In these methods, the outputs of several clustering algorithms are combined. In these research works, the input hierarchical clusterings are transformed to descriptor matrices and their combination is achieved by aggregating their descriptor matrices. In previous works, only element-wise aggregation operators have been used and the relation between the elements of each descriptor matrix has been ignored. However, the value of each element of the descriptor matrix is meaningful in comparison with its other elements. The current study proposes a novel method of combining hierarchical clustering approaches based on principle component analysis (PCA). PCA as an aggregator allows considering all elements of the descriptor matrices. In the proposed approach, basic clusters are made and transformed to descriptor matrices. Then, a final matrix is extracted from the descriptor matrices using PCA. Next, a final dendrogram is constructed from the matrix that is used to summarize the results of the diverse clustering. The experimental results on popular available datasets show the superiority of the clustering accuracy of the proposed method over basic clustering methods such as single, average and centroid linkage and previously combined hierarchical clustering methods. In addition, statistical tests show that the proposed method significantly outperformed hierarchical clustering combination methods with element-wise averaging operators in almost all tested datasets. Several experiments have also been conducted which confirm the robustness of the proposed method for its parameter setting.

Weighting and normalization are the most important factor that may affect the text representation significantly. This paper presents two novel term weighting schemes to represent text documents, namely, i). Term-weighting scheme for document representation based on Term Frequency - Ranking of Term Frequency (TF-RTF) and ii). Term-weighting scheme for document representation based on Term Frequency - Ranking of fuzzy logic with semantic relationship of terms (TF-RFST). The ranking of each term in a document provides its priority of the document and uses these priorities for document representation in TF-RTF. In TF-RFST, each term is represented based on its frequency and the frequency of semantic related terms for that term. Hence, the ranking of each term is based on the combined frequencies of the term and its semantic related terms with a specific weighting scheme. With appropriate weighting schemes such as TF-RFT and TF-RFST, the proposed methods provide better clustering performance in terms of accuracy, entropy, recall and F-Measure than previously suggested methods, such as word count, Term Frequency-Inverse Document Frequency (TF-IDF), Term Frequency-Inverse Corpus Frequency (TF-ICF), Multi Aspect TF (MATF), BM25 and BM25F. Experiments carried out on the Reuters-8, Reuters-52 and WebKB data sets with K-means and K-means++ clustering algorithms for demonstrate the effectiveness of the proposed term weighting schemes.

Nowadays, data clustering is an important task to the mining research community since the availability of uncertain data is increasing rapidly in many applications such as weather forecasting, business information management systems. In this work, proposed Monte Carlo integration based uncertain objects modeling technique is compared with three state-of-the-art methods namely, kernel density estimation, Dempster–Shafer, and Monte Carlo simulation. Then Kullback–Leibler and Jeffrey divergences are used to measure the similarity between uncertain objects and merge them with modified DBSCAN and k-medoids clustering algorithms. A heuristic algorithm is proposed to find the optimum radius, which is one of the inputs of DBSCAN. All the experiments are performed on one synthesized dataset and three real datasets namely, weather data, Japanese vowels and activity of daily living data. Five performance measures namely, accuracy, precision, recall, F-score, and Jaccard index are considered for comparing proposed method with state-of-the-art methods. Two non-parametric tests namely, Wilcoxon rank sum and sign test are also conducted. These results denote the effectiveness and efficiency of the proposed method over state-of-the-art methods.

Document representation is central to modern natural language processing systems including document clustering. Empirical experiments in recent studies provide strong evidence that unsupervised language models can learn context-aware representations in the given documents and advance several NLP benchmark results. However, existing clustering approaches focus on the dimensionality reduction and do not exploit these informative representations. In this paper, we propose a conceptually simple but experimentally effective clustering framework called Advanced Document Clustering (ADC). In contrast to previous clustering methods, ADC is designed to leverage syntactically and semantically meaningful features through feature-extraction and clustering modules in the framework. We first extract features from pre-trained language models and initialize cluster centroids to spread out uniformly. In the clustering module of ADC, the semantic similarity can be measured using the cosine similarity and centroids update while assigning centroids to a mini-batch input. Also, we utilize cross entropy loss partially, as the self-training scheme can be biased when parameters in the model are inaccurate. As a result, ADC can take advantages of contextualized representations while mitigating the limitations introduced by high-dimensional vectors. In numerous experiments with four datasets, the proposed ADC outperforms other existing approaches. In particular, experiments on categorizing news corpus with fake news demonstrated the effectiveness of our method for contextualized representations.

Many clustering algorithms categorized as K-clustering algorithm require the user to predict the number of clusters (K) to do clustering. Due to lack of domain knowledge an accurate value of K is difficult to predict. The problem becomes critical when the dimensionality of data points is large; clusters differ widely in shape, size, and density; and when clusters are overlapping in nature. Determining the suitable K is an optimization problem. Automatic clustering algorithms can discover the optimal K. This paper presents an automatic clustering algorithm which is superior to K-clustering algorithm as it can discover an optimal value of K. Iterative hill-climbing algorithms like K-Means work on a single solution and converge to a local optimum solution. Here, Genetic Algorithms (GAs) find out near global optimum solutions, i.e. optimal K as well as the optimal cluster centroids. Single-objective clustering algorithms are adequate for efficiently grouping linearly separable clusters. For non-linearly separable clusters they are not so good. So for grouping non-linearly separable clusters, we apply Multi-Objective Genetic Algorithm (MOGA) by minimizing the intra-cluster distance and maximizing inter-cluster distance. Many existing MOGA based clustering algorithms are suitable for either numeric or categorical features. This paper pioneered employing MOGA for automatic clustering with mixed types of features. Statistical testing on experimental results on real-life benchmark data sets from the University of California at Irvine (UCI) machine learning repository proves the superiority of the proposed algorithm.

Noisy image segmentation is a hot topic in natural, medical, and remote sensing image processing. It is among the non-trivial problems of computer vision having to address denoising and segmentation at the same time. Fuzzy C-means (FCM) is a clustering algorithm that has been shown to be effective at dealing with both segmentation-oriented denoising and segmentation at the same time. Moreover, with a high level of noise and other imaging artifacts, FCM loses its ability to perform image segmentation effectively. This paper introduces a Particle Swarm Optimization (PSO)-based feature enhancement approach in the wavelet domain for noisy image segmentation. This approach applies adaptive wavelet shrinkage using FCM clustering performance as an evaluation mechanism and also as the segmentation algorithm. The PSO-based process helps to enhance intensity features for clustering-based denoising, and also provides adaptivity for the system that performs well on a range of real, synthetic, and simulated noisy images with different noise levels and range/spatial properties. Furthermore, the algorithm applies edge enhancement based on Canny edge detector in order to further improve accuracy. Experiments are presented using three different datasets each degraded with different types of common noise. The presented algorithms show effective and consistent performance over a range of severe noise levels without the need for any parameter tuning.

Text summarization has become an important research area, especially in the biomedical domain, where information overload is a major problem. In this paper, we propose a novel biomedical text summarization system that combines two popular data mining techniques: clustering and frequent itemset mining. Biomedical paper is expressed as a set of biomedical concepts using the UMLS metathesaurus. The K-means algorithm is used to cluster similar sentences. Then, the Apriori algorithm is applied to discover the frequent itemsets among the clustered sentences. Finally, the salient sentences from each cluster are selected to build the summary using the discovered frequent itemsets. For the evaluation step, we selected randomly 100 biomedical papers from the BioMed Central database full-text, and we evaluated the performances of our system by comparing the resulting summaries with the abstracts of these papers using the ROUGE metrics in term of recall, precision, and F-measure. We also compared the obtained summaries with those achieved by five well-known summarizers: TextRank, TextTeaser, SweSum, ItemSet Based Summarizer, Microsoft AutoSummarize, and two baselines: summarization using only the frequent itemsets mining (FRQ-CL), and summarization using only the clustering (CL-FRQ). The results demonstrate that this combination can successfully enhance the summarization performances, and the proposed system outperforms other tested summarizers.

Image Matting is an actively researched topic due to its massive impact on the filmmaking industry. It refers to the accurate extraction of the foreground from an image and is a severely under-constrained problem. To constrain this issue, user input in the form of a trimap is required. However, users must generate this trimap manually, which is an exhausting and time-consuming process. To tackle this issue, we develop an expert system which is capable of generating trimaps automatically without any user interaction. We utilize three different techniques which use machine learning at the core. Our expert system receives knowledge from these three techniques which are processed by our system to generate trimaps automatically. We strongly believe that our study will have a strong impact on the filmmaking industry. Our study lays the groundwork for automatic trimap generation. Furthermore, our study can be applied to expert and intelligent systems related to image retrieval and automatic foreground/primary object segmentation in images or videos. We propose a simple yet effective approach to generate optimal trimaps automatically by combining image saliency, graph cut segmentation (lazy snapping), and fuzzy c-means clustering (FCM). Lazy snapping is an interactive segmentation technique that requires foreground and background scribbles as input. Instead of using user-provided foreground scribbles, we utilize a saliency map as foreground scribbles and input it to the lazy snapping. This results in a coarsely segmented foreground object. We use the corners as the background scribbles based on the assumption that most foreground objects are located in the center of the image. To generate an optimal trimap, we locally cluster the boundary region of the foreground segmentation using FCM. We tested our algorithm on alpha matting evaluation and salient object datasets. In addition to generating accurate trimaps automatically, the alpha mattes generated by our optimal trimaps contain fewer artifacts as compared to trimaps generated by previous works. Moreover, the alpha mattes computed using our optimal trimaps were computed faster as compared to computing alpha mattes with trimaps from previous works. We showed in our experiments that optimal trimaps can improve alpha matte quality by reducing artifacts. Finally, our approach does not rely on depth data like the previous methods. Our experiments show the effectiveness of our method.

Document clustering is a gathering of textual content documents into groups or clusters. The main aim is to cluster the documents, which are internally logical but considerably different from each other. It is a crucial process used in information retrieval, information extraction and document organization. In recent years, the spectral clustering is widely applied in the field of machine learning as an innovative clustering technique. This research work proposes a novel Spectral Clustering algorithm with Particle Swarm Optimization (SCPSO) to improve the text document clustering. By considering global and local optimization function, the randomization is carried out with the initial population. This research work aims at combining the spectral clustering with swarm optimization to deal with the huge volume of text documents. The proposed algorithm SCPSO is examined with the benchmark database against the other existing approaches. The proposed algorithm SCPSO is compared with the Spherical K-means, Expectation Maximization Method (EM) and standard PSO Algorithm. The concluding results show that the proposed SCPSO algorithm yields better clustering accuracy than other clustering techniques.

Many approaches have been proposed to recognize clusters in subspaces. However, their performance is highly sensitive to input parameter values. The purpose and expected ranges of these parameters may not available to a non-expert user. The parameter setting producing optimal results can only be known after repeated execution of the clustering process every time with a different set, which is very time consuming. Most of the existing algorithms show high runtimes due to excessive data scans. In this work, we propose a subspace clustering technique that estimates the distance threshold parameter automatically from the data for each attribute and works on the basis of single linkage clustering, in bottom up, greedy fashion. The experimental results show that, the algorithm produces optimal results without accepting any input from the user, achieves up to 10 times better runtime and improved accuracy in a single run without requiring any tuning of parameter values.

Computer vision-guided traffic management is an emerging area of research. Intelligent traffic signal control using computer vision is a less explored area of research. In this paper, we propose a new approach of traffic flow-based intelligent signal timing by temporally clustering optical flow features of moving vehicles using Temporal Unknown Incremental Clustering (TUIC) model. First, we propose a new inference scheme that works approximately 5-times faster as compared to the one originally proposed in TUIC in a dense traffic intersection. The new inference scheme can trace clusters representing moving objects that may be occluded while being tracked. Cluster counts of approach roads have been used for signal timing for traffic intersections. It is done by detecting cluster motion inside the regions-of-interest (ROI) marked at the entry and exit locations of intersection approaches. Departure rates are learned using Gaussian regression to parameterize traffic variations. Using the learned parameters as a function of cluster count, an adaptive signal timing algorithm, namely Throughput and Average Waiting Time Optimization (TAWTO) has been proposed. Experimental results reveal that the proposed method can achieve better average waiting time and throughput as compared to the state-of-the-art signal timing algorithms. We intend to publish two datasets as part of this work for enabling the research community to explore computer vision aided solutions for typical problems such as intelligent traffic controlling, violation detection in chaotic road intersections, etc.

Integer programming models for community detection in relational networks have diverse applications in different fields. From making our lives easier by improving search engine optimization to saving our lives by aiding in threat detection and disaster management, researches in this niche have added value to human experience and knowledge. Besides the community structure, the influential nodes or members in a complex network are highly effective at diffusing information quickly to others in the community. Prior research dealing with the use of optimization models for clustering networks has independently focused on detecting communities. In this research, we propose a new integer linear programming model to detect community structure in real-life networks and also identify the most influential node within each community. We validate the proposed model by testing it on a well-established community network. Further, the performance of the proposed model is evaluated by comparing it with the existing best performing optimization model as well as three heuristic approaches for community detection. The experimental results indicate that in most cases the proposed integer programming model performs better than the existing optimization model with respect to modularity, Silhouette coefficient and computational time. Besides, our model yields superior Silhouette and competitive modularity values compared to the heuristic approaches in many cases.

Maintaining a good balance between convergence and diversity in many-objective optimization is a key challenge for most Pareto dominance-based multi-objective evolutionary algorithms. In most existing multi-objective evolutionary algorithms, a certain fixed metric is used in the selection operation, no matter how far the solutions are from the Pareto front. Such a selection scheme directly affects the performance of the algorithm, such as its convergence, diversity or computational complexity. In this paper, we use a more structured metric, termed augmented penalty boundary intersection, which acts differently on each of the non-dominated fronts in the selection operation, to balance convergence and diversity in many-objective optimization problems. In diversity maintenance, we apply a distance-based selection scheme to each non-dominated front. The performance of our proposed algorithm is evaluated on a variety of benchmark problems with 3 to 15 objectives and compared with five state-of-the-art multi-objective evolutionary algorithms. The empirical results demonstrate that our proposed algorithm has highly competitive performance on almost all test instances considered. Furthermore, the combination of a special mate selection scheme and a clustering-based selection scheme considerably reduces the computational complexity compared to most state-of-the-art multi-objective evolutionary algorithms.

This paper presents a combinatorial problem called a pick-up routing problem with a three-dimensional (3D-PRP) loading constraint, clustered backhauls at the operational level, and train loading at the tactical level for an intermodal transportation network. A two-phase approach, called clustering first, packing-routing second, is proposed for use during the first stage. The clustering of backhauls is carried out using the k-means algorithm. A hybrid approach is provided, which combines the packing of orders by first solving a 3D loading problem for each cluster using machine learning with a best-fit-first strategy, with routing using a genetic algorithm. During the second stage, the train-loading problem is solved using a mixed integer programming approach to minimise the total costs by incorporating various cost types, in which detention and demurrage costs are taken into account. All solution approaches are computationally evaluated on real-world data provided by an international logistics firm and new randomly generated instances. Comparisons are carried out using both exact solution methods and heuristic approaches, and the proposed approach was shown to be more effective for real-world problems. © 2019 Elsevier Ltd

With rapid explosion of information, clustering emerged as an active research area for knowledge discovery. Most of the existing clustering algorithms become ineffective when inappropriate parameters are provided or applied on a dataset which consists of clusters of diverse shapes, sizes, and varying densities. To overcome these issues, many graph based hybrid clustering algorithms have been proposed but these algorithms first generate a complete graph of the dataset which takes O(N2) time where N is the number of data points which limits their application on large datasets. This paper proposes an algorithm namely a fast hybrid clustering technique based on local nearest neighbor using minimum spanning tree to reduce the computational overhead. In the first step, the algorithm partitions the dataset into large number of sub-clusters based on dispersion of data points to capture the geometry of clusters. After partitioning the dataset, a minimum spanning tree based on the centroids of each of the sub-clusters is constructed to identify the adjacent pairs. A novel merge method is proposed to find the genuine clusters by repeatedly merging the adjacent sub-clusters. The cohesion and intra-similarity are introduced to compute the level of dispersion of data points with respect to the centers of an adjacent pair and average edge weight of a sub-clusters respectively. The algorithm takes O(N3/2) time which is a N factor improvement over the popular hybrid clustering algorithms. Experimental analyses on both synthetic as well as gene expression datasets demonstrate that the proposed technique shows significant improvement over competing clustering algorithms in terms of execution time and improved cluster quality. Moreover, the proposed algorithm does not require any user defined parameters and it can estimate the number of clusters more accurately.

This paper presents a preliminary algorithm which is designed to reduce the processing cost of continuing clustering in dynamic networks. This algorithm considers that various types of changes (Inserts and deletes) might affect the clustered data over time. It promises to provide both a reliable and updated answer for clustering problem at all times. By altering the well-known K-means algorithm, this enhanced version has three parts: Initializer and Sorter, the main objective of this part is to initialize the algorithm and to store some data that can be used to reduce the calculations later on, The Dynamic Modifier, this part applies the modifications on clusters and also updates the centroids and the related info to keep the clusters valid, and The Detector, which detects the potent nodes which might need to swap their clusters after applying the recent changes that Dynamic Modifier applied. This algorithm reduces the amount of calculations by using the related data from the last scope of the clustered network to detect the potent nodes, so it can only check them for further modifications. The simulation results indicate that the number of checked nodes and the total consumed time during each iteration is reduced significantly comparing to the traditional K-means algorithm.

Graphical abstract, The absence of dedicated vaccines or drugs makes the COVID-19 a global pandemic, and early diagnosis can be an effective prevention mechanism. RT-PCR test is considered as one of the gold standards worldwide to confirm the presence of COVID-19 infection reliably. Radiological images can also be used for the same purpose to some extent. Easy and no contact acquisition of the radiological images makes it a suitable alternative and this work can help to locate and interpret some prominent features for the screening purpose. One major challenge of this domain is the absence of appropriately annotated ground truth data. Motivated from this, a novel unsupervised machine learning-based method called SUFMACS (SUperpixel based Fuzzy Memetic Advanced Cuckoo Search) is proposed to efficiently interpret and segment the COVID-19 radiological images. This approach adapts the superpixel approach to reduce a large amount of spatial information. The original cuckoo search approach is modified and the Luus-Jaakola heuristic method is incorporated with McCulloch’s approach. This modified cuckoo search approach is used to optimize the fuzzy modified objective function. This objective function exploits the advantages of the superpixel. Both CT scan and X-ray images are investigated in detail. Both qualitative and quantitative outcomes are quite promising and prove the efficiency and the real-life applicability of the proposed approach.

Lane changing behavior generally expresses uncertainty due to the impact of environmental factors, and unreasonable lane changes can cause serious collisions. High precision prediction of lane changing intent is helpful to enhance proactivity in driving safety protection. This study proposed a lane-changing prediction model based on Fuzzy C-means clustering algorithm and adaptive Neural Network (FCMNN), which introduced a new prediction process: (1) Unsupervised learning method: categorize original dataset into different clusters according to their distribution features; (2) Supervised learning method: optimize sub-Neural Network structures and weighting parameters for each cluster or pattern. Through comparing with several traditional methods under different simulation scenarios, the proposed model effectively improve the prediction performance and stability. The results obtained in this study will be helpful to deeply analyze the intent recognition of driving behavior, improve the safety of lane-changing behavior, and provide key technology in driving prediction of Advanced Driver Assistance System (ADAS).

A variety of algorithms have recently emerged in the field of cluster analysis. Consequently, based on the distribution nature of the data, an appropriate algorithm can be chosen for the purpose of clustering. It is difficult for a user to decide a priori which algorithm would be the most appropriate for a given dataset. Algorithms based on graphs provide good results for this task. However, these algorithms are vulnerable to outliers with limited information about edges contained in the tree to split a dataset. Thus, in several fields, the need for better clustering algorithms increases and for this reason utilizing robust and dynamic algorithms to improve and simplify the whole process of data clustering has become an urgent need. In this paper, we propose a novel distance-based clustering algorithm called the critical distance clustering algorithm. This algorithm depends on the Euclidean distance between data points and some basic mathematical statistics operations. The algorithm is simple, robust, and flexible; it works with quantitative data that are real-valued, not qualitative, and categorical with different dimensions. In this work, 26 experiments are conducted using different types of real and synthetic datasets taken from different fields. The results prove that the new algorithm outperforms some popular clustering algorithms such as MST-based clustering, K-means, and Dbscan. Moreover, the algorithm can precisely produce more reasonable clusters even when the dataset contains outliers and without specifying any parameters in advance. It also provides a number of indicators to evaluate the established clusters and prove the validity of the clustering.

Machine learning has become a pillar of today’s expert and intelligent systems where a special attention has been drawn to unsupervised methods. Support vector data description is one of the most interesting methods proposed in this context that addresses two main challenges: the absence of ground truth labels and the presence of outliers in data. It is well known that the ultimate problem has a great impact on a system performance since it typically prevents and limits its generalization and hence its failure. Even though the strengths of the Support vector data description method, its application into multi class and large scale context, as computer vision systems, raises bottleneck issues due to its runtime complexity and mono-class context application. We propose, in this paper, an approach inspired from the movement behavior of particle swarm and relies on the use of core set paradigm to overcome the aforementioned drawbacks of Support vector data description method. Indeed, since the proposed method uses some parameters corresponding to kernel function and swarm’s movement formulation, we present and study in this paper some strategies to ensure a dynamic adaptation of these parameters. The main qualities of the proposed method is that it provides a fast scheme for clustering a set of points without neither a prior knowledge on the number of naturally occurring cluster in the data nor an assumption on clusters shapes. A quantitative and comparative performance assessment is carried out over the Berkeley and BuffaloXiph datasets. The results witness the efficiency of the method in terms of both classification accuracy and computational performance. These quantitative assessments reinforce the significance as well as the importance of embedding the proposed method in other intelligent systems application areas.

Diabetes mellitus is a medical condition examined by data miners for reasons such as significant health complications in affected people, the economic impact on healthcare networks, and so on. In order to find the main causes of this disease, researchers look into the patient's lifestyle, hereditary information, etc. The goal of data mining in this context is to find patterns that make early detection of the disease and proper treatment easier. Due to the high volume of data involved in therapeutic contexts and disease diagnosis, provision of the intended treatment method become almost impossible over a short period of time. This justifies the use of pre-processing techniques and data reduction methods in such contexts. In this regard, clustering and meta-heuristic algorithms maintain important roles. In this paper, a method based on the k-means clustering algorithm is first utilized to detect and delete outliers. Then, in order to select significant and effective features, four bi-objective meta-heuristic algorithms are employed to choose the least number of significant features with the highest classification accuracy using support vector machines (SVM). In addition, the 10-fold cross validation (CV) method is used to validate the constructed model. Using real case data, it is concluded that the multi-objective firefly (MOFA) and multi-objective imperialist competitive algorithm (MOICA) with a 100% classification accuracy outperform the non-dominated sorting genetic algorithm (NSGA-II) and multi-objective particle swarm optimization (MOPSO) with the accuracies of 98.2% and 94.6%, respectively.

The reconstruction of cross-cut shredded text documents (RCCSTD) is an important problem in forensics and is a real, complex and notable issue for information security and judicial investigations. It can be considered a special kind of greedy square jigsaw puzzle and has attracted the attention of many researchers. Clustering fragments into several rows is a crucial and difficult step in RCCSTD. However, existing approaches achieve low clustering accuracy. This paper therefore proposes a new clustering algorithm based on horizontal projection and a constrained seed K-means algorithm to improve the clustering accuracy. The constrained seed K-means algorithm draws upon expert knowledge and has the following characteristics: 1) the first fragment in each row is easy to distinguish and the unidimensional signals that are extracted from the first fragment can be used as the initial clustering center; 2) two or more prior fragments cannot be clustered together. To improve the splicing accuracy in the rows, a penalty coefficient is added to a traditional cost function. Experiments were carried out on 10 text documents. The accuracy of the clustering algorithm was 99.1% and the overall splicing accuracy was 91.0%, according to our measurements. The algorithm was compared with two other approaches and was found to offer significantly improved performance in terms of clustering accuracy. Our approach obtained the best results of RCCSTD problem based on our experiment results. Moreover, a more complex and real problem – reconstruction of cross-cut shredded dual text documents (RCCSDTD) problem – was tried to solve. The satisfactory results for RCCSDTD problems in some cases were obtained, to authors’ best knowledge, our method is the first feasible approach for RCCSDTD problem. On the other hand, the developed system is fundamentally an expert system that is being specifically applied to solve RCCSTD problems.

Clustering algorithms usually assume that the number K of clusters is known, although there is often no prior knowledge about the underlying set. Consequently, the significance of the defined groups needs to be validated. Cluster validity indexes are commonly used to perform the validation of clustering results. However, most of them are considered to be dependent on the number of data objects and often tend to ignore small and low-density groups. Furthermore, suboptimal clustering solutions are frequently selected when the clusters are in a certain degree of overlapping or low separation. Thus, we propose a new non-parametric internal validity index based on within-cluster mutual equidistant-scattering for crisp clustering. Eight different validity indexes were analysed to detect the number of clusters in a data set. Experiments on both synthetic and real-world data show the effectiveness and reliability of our approach to evaluate the hyperparameter K.

For the sake of credit risk assessment, credit scoring has become a critical tool to discriminate “bad” applicants from “good” applicants for financial institutions. Accordingly, a wide range of supervised machine learning algorithms have been successfully applied to credit scoring; however, integration of unsupervised learning with supervised learning in this field has drawn little consideration. In this work, we propose a combination strategy of integrating unsupervised learning with supervised learning for credit risk assessment. The difference between our work and other previous work on unsupervised integration is that we apply unsupervised learning techniques at two different stages: the consensus stage and dataset clustering stage. Comparisons of model performance are performed based on three credit datasets in four groups: individual models, individual models + consensus model, clustering + individual models, clustering + individual models + consensus model. As a result, integration at either the consensus stage or dataset clustering stage is effective on improving the performance of credit scoring models. Moreover, the combination of the two stages achieves the best performance, thereby confirming the superiority of the proposed integration of unsupervised and supervised machine learning algorithms, which boost our confidence that this strategy can be extended to many other credit datasets from financial institutions.

Credit risk assessment is an important task for the implementation of the bank policies and commercial strategies. In this paper, we used a discrete Bayesian network with a latent variable to model the payment default of loans subscribers. The proposed Bayesian network includes a built-in clustering feature. A full procedure for learning its parameters, based on a customized Expectation-Maximization algorithm was provided. This model allows evaluating the payment default probability taking into account several factors and handling a multi-class situation. Relying on a real data set describing loans contracts, we calibrated the model and performed several analyses. The obtained results highlight a regime switching of the default probability distribution: Two classes were determined showing a change in credit risk profiles.

There are two main reasons why traditional clustering schemes are incompetent for high-dimensional categorical data. First, traditional methods usually represent each cluster by all dimensions without difference; and second, traditional clustering methods only rely on an individual dimension of projection as an attribute’s weight ignoring relevance among attributes. We solve these two problems by a MapReduce-based subspace clustering algorithm (called PUMA) using multi-attribute weights. The attribute subspaces are constructed in our PUMA by calculating an attribute-value weight based on the co-occurrence probability of attribute values among different dimensions. PUMA obtains sub-clusters corresponding to respective attribute subspaces from each computing node in parallel. Lastly, PUMA measures various scale clusters by applying the hierarchical clustering method to iteratively merge sub-clusters. We implement PUMA on a 24-node Hadoop cluster. Experimental results reveal that using multi-attribute weights with subspace clustering can achieve better clustering accuracy on both synthetic and real-world high dimensional datasets. Experimental results also show that PUMA achieves high performance in terms of extensibility, scalability and the nearly linear speedup with respect to number of nodes. Additionally, experimental results demonstrate that PUMA is reasonable, effective, and practical to expert systems such as knowledge acquisition, word sense disambiguation, automatic abstracting and recommender systems.

In many practical machine learning problems, the acquisition of labeled data is often expensive and time consuming. To reduce this labeling cost, active learning has been introduced in many scientific fields. This study considers the problem of active learning of a regression model in the context of an optimal experimental design. Classical optimal experimental design approaches are based on the least square errors of labeled samples. Recently, a couple of active learning approaches that take advantage of both labeled and unlabeled data have been developed based on Laplacian regularized regression models with a single criterion. However, these approaches are susceptible to selecting undesirable samples when the number of initially labeled samples is small. To address this susceptibility, this study proposes an active learning method that considers multiple complementary criteria. These criteria include sample representativeness, diversity information, and variance reduction of the Laplacian regularization model. Specifically, we developed novel density and diversity criteria based on a clustering algorithm to identify the samples that are representative of their distributions, while minimizing their redundancy. Experiments were conducted on synthetic and benchmark data to compare the performance of the proposed method with that of existing methods. Experimental results demonstrate that the proposed active learning algorithm outperforms its existing counterparts.

Metric learning, which aims to determine an appropriate distance function to measure the similarity and dissimilarity between data points accurately, is one of the most popular methods to enhance the performance of many machine learning methods such as K-means clustering and K nearest neighbor classifier algorithms. These algorithms may not perform well because of the use of normal Euclidean distance function that ignores any statistical regularities that might be estimated from a large training set of labeled examples. In many real-world applications, the Euclidean distance may not be fit to capture the intrinsic similarity and dissimilarity between the data points. Compared to existing metric learning algorithms, which use large amount of labeled data in the form of must-link (ML) and cannot-link constraints as side information where the granularity of true clustering is unknown, our proposed approach uses few labeled data in the form of relative-distance constraints such as equality constraints, Ceq, and inequality constraints, Cneq. For satisfying such constraints, we need to project the initial Euclidean distance matrix by using Bregman projection on the convex subset of constraints in such a way that all the constraints are satisfied. Since Bregman projection is not orthogonal, means while satisfying the current constraint previously satisfied constraints may get unsatisfied, we need to select a proper subset of constraints for learning better distance function. The multi-objective framework is utilized for selecting a good subset of constraints which can help in getting the proper labeling of the data set. The selected subset of constraints is used for adjusting the initial kernel-matrix. K-means clustering technique is applied to the adjusted kernel matrix to label the data set. In order to evaluate the quality of obtained labeling, different external and internal cluster validity indices are deployed. The values of these indices are simultaneously optimized using the search capability of MOO with the aim of selecting the appropriate subset of constraints. The proposed approach is evaluated on UCI Human Activity Recognition using Smartphone Dataset v1.0 along with nine other popular data sets. Results show that our approach outperforms the state of the art semi-supervised metric learning algorithms with respect to different internal and external cluster validity indices.

In most applications of data clustering the input data includes vectors describing the location of each data point, from which distances between data points can be calculated and a proximity matrix constructed. In some applications, however, the only available input is the proximity matrix, that is, the distances between each pair of data point. Several clustering algorithms can still be applied, but if the proximity matrix has missing values no standard method is directly applicable. Imputation can be done to replace missing values, but most imputation methods do not apply when only the proximity matrix is available. As a partial solution to fill this gap, we propose the Proximity Matrix Completion (PMC) algorithm. This algorithm assumes that data is missing due to one of two reasons: complete dissimilarity or incomplete observations; and imputes values accordingly. To determine which case applies the data is modeled as a graph and a set of maximum cliques in the graph is found. Overlap between cliques then determines the case and hence the method of imputation for each missing data point. This approach is motivated by an application in plant breeding, where what is needed is to cluster new experimental seed varieties into sets of varieties that interact similarly to the environment, and this application is presented as a case study in the paper. The applicability, limitations and performance of the new algorithm versus other methods of imputation are further studied by applying it to datasets derived from three well-known test datasets.

In unsupervised learning, the problem of finding the appropriate number of clusters -usually notated as k- is very challenging. Its importance lies in the fact that k is a vital hyperparameter for the most clustering algorithms. One algorithmic approach for tacking this problem is to apply a certain clustering algorithm with various cluster configurations and decide to use the one that maximizes a certain internal validity measure. This is a promising and computationally efficient approach since the independent runs are parallelizable. In this paper, we attempt to improve over this estimation approach by incorporating a consensus clustering approach into k estimating scheme. The weighted consensus clustering scheme employs four different indices namely Silhouette (SH), Calinski–Harabasz (CH), Davies–Bouldin (DB), and Consensus (CI) indices to estimate the correct number of cluster. Computational experiments in a dataset with clusters ranging from 2 to 7 show the profound advantages of weighted consensus clustering for correctly finding k in comparison to individual clustering method (e.g, k-means) and simple consensus clustering.

Clustering of a group of interval-valued time series of unequal length is often encountered and the key point of this clustering is the distance measure between two interval-valued time series. However, most distance measure methods apply to interval-valued time series of equal length, and another methods applicable to unequal-length ones usually show high computational cost. In order to give a reasonable and efficient distance measure, this paper first proposes a new representation in the form of a sequence of 3-tuples for interval-valued time series. In this representation, fully take into account the time-axis and value-axis information to decrease the loss of information. Meanwhile, this representation is guaranteed to achieve dimensionality reduction. Based on the new representation, dynamic time warping algorithm is then employed and an improved dynamic time warping algorithm is produced. Furthermore, a hierarchical clustering algorithm based on the new proposed distance measure is designed for interval-valued time series of equal or unequal length. Experimental results show the effectiveness of the proposed distance and quantify the performance of the designed clustering method.

Idiopathic Parkinsons disease (PD) is currently the second most important neurodegenerative disease in incidence. Deep brain stimulation (DBS) constitutes a successful and necessary therapy; however, the continuous stimulation it provides can be associated with multiple side effects. DBS uses an implanted pulse generator that delivers, through a set of electrodes, electrical stimulation to the target area, normally the Sub Thalamic Nucleus. Recently, Closed-loop DBS has emerged as a promising new strategy, where the device stimulates only when necessary, thereby reducing any adverse effects. Here, we present a Closed-loop DBS system for PD, which is able to recognize, with 100% accuracy, when the patient is going to enter into the tremor phase, thus allowing the device to stimulate only in such cases. The expert system has been designed and implemented within the data stream mining paradigm, suitable for our scenario since it can cope with continuous data of a theoretical infinite length and with a certain variability, which uses the synchronization among the neural population within the Sub Thalamic Nucleus as the continuous data stream input to the system.

Recently, heterogeneous network representation learning has attracted a lot of attentions due to its potential applications. Our works in this paper are concentrated on how to leverage the output of network representation learning by combining with the topic similarity between nodes in content-based heterogeneous information network (C HIN). These unique challenges come from the shortage of topic similarity evaluation between text-based nodes which limit the accuracy of the similarity search as well other network mining tasks. Moreover, the massive sizes of current real-world network also raises challenges for traditional standalone-based heterogeneous network analysis models. Different from previous network representation learning models, such as: Node2Vec or Metapath2Vec, our proposed W-MethPath2Vec model uses the topic-driven meta-path-based random walk mechanism for generating heterogeneous neighborhood of nodes as the learning features. Then, these learning nodes’ features are used to train the learning model which is used for solving various heterogeneous network mining tasks such as: node similarity search, clustering, classification, link prediction, etc. The W-MethPath2Vec model enables the simultaneous modeling of structural and topic correlations between nodes in heterogeneous networks. Moreover, the W-MethPath2Vec model is implemented in the Apache Spark-based distributed framework which enables the capability of handling large-scaled networks. We tested our W-MethPath2Vec model with the previous state-of-the-art approaches in the real-world datasets to demonstrate the effectiveness of our proposed model.

We propose a new heuristic algorithm for the well-known Orienteering Problem, which aims at maximising the prize collected at vertices of a graph, visiting them through a simple closed tour whose length (also known as travel time) is limited by an upper bound. The algorithm is based on the Adaptive Large Neighbourhood Search metaheuristic paradigm, and uses a clustering of the graph to operate on groups of nearby vertices. Extensive computational results show that the proposed algorithm is able to find very good solutions (it produced 27 new best known solutions on a set of benchmark instances) and that it fares favourably compared with other state-of-the-art heuristics when tested with both long and short computation time budgets.

The emergence of ``curse of dimensionality” issue as a result of high reduces datasets deteriorates the capability of learning algorithms, and also requires high memory and computational costs. Selection of features by discarding redundant and irrelevant features functions as a crucial machine learning technique aimed at reducing the dimensionality of these datasets, which improves the performance of the learning algorithm. Feature selection has been extensively applied in many application areas relevant to expert and intelligent systems, such as data mining and machine learning. Although many algorithms have been developed so far, they are still unsatisfying confronting high-dimensional data. This paper presented a new hybrid filter-based feature selection algorithm based on acombination of clustering and the modified Binary Ant System (BAS), called FSCBAS, to overcome the search space and high-dimensional data processing challenges efficiently. This model provided both global and local search capabilities between and within clusters. In the proposed method, inspired by genetic algorithm and simulated annealing, a damped mutation strategy was introduced that avoided falling into local optima, and a new redundancy reduction policy adopted to estimate the correlation between the selected features further improved the algorithm. The proposed method can be applied in many expert system applications such as microarray data processing, text classification and image processing in high-dimensional data to handle the high dimensionality of the feature space and improve classification performance simultaneously. The performance of the proposed algorithm was compared to that of state-of-the-art feature selection algorithms using different classifiers on real-world datasets. The experimental results confirmed that the proposed method reduced computational complexity significantly, and achieved better performance than the other feature selection methods.