Search

Feature selection is an important topic in data mining, especially for high dimensional dataset. Feature selection is a process commonly used in machine learning, wherein subsets of the features available from the data are selected for application of learning algorithm. The best subset contains the least number of dimensions that most contribute to accuracy. Feature selection methods can be decomposed into three main classes, one is filter method, another one is wrapper method and third one is embedded method. This chapter presents an empirical comparison of feature selection methods and its algorithm. In view of the substantial number of existing feature selection algorithms, the need arises to count on criteria that enable to adequately decide which algorithm to use in certain situation. This chapter reviews several fundamental algorithms found in the literature and assess their performance in a controlled scenario.

Clustering generic data, i.e., data not specific to a particular field, is a challenging problem due to their diverse complex structures in the original feature space. Traditional approaches address this problem by complementing clustering with feature learning methods, which either capture the intrinsic structure of the data or represent the data such that clusters are better revealed. In this paper, we propose an approach referred to as Extreme Learning Machine for Joint Embedding and Clustering (ELM-JEC), which incorporates desirable properties of both types of feature learning methods at the same time, specifically by (1) preserving the manifold structure of the data in the original space; (2) maximizing the class separability of the data in the embedded space. Since either type of method has improved clustering performance in some cases, our motivation is to integrate the two desirable properties to further improve the accuracy and robustness of clustering. Additional notable features of ELM-JEC are that it provides nonlinear feature mappings and achieves feature learning and clustering in the same formulation. The proposed approach can be implemented using alternating optimization, and its clustering performance compares favorably with several state-of-the-art methods on the real-world benchmark datasets. MOE (Min. of Education, S’pore)

This work draws inspiration from three important sources of research on dissimilarity-based clustering and intertwines those three threads into a consistent principled functorial theory of clustering. Those three are the overlapping clustering of Jardine and Sibson, the functorial approach of Carlsson and M\'{e}moli to partition-based clustering, and the Isbell/Dress school's study of injective envelopes. Carlsson and M\'{e}moli introduce the idea of viewing clustering methods as functors from a category of metric spaces to a category of clusters, with functoriality subsuming many desirable properties. Our first series of results extends their theory of functorial clustering schemes to methods that allow overlapping clusters in the spirit of Jardine and Sibson. This obviates some of the unpleasant effects of chaining that occur, for example with single-linkage clustering. We prove an equivalence between these general overlapping clustering functors and projections of weight spaces to what we term clustering domains, by focusing on the order structure determined by the morphisms. As a specific application of this machinery, we are able to prove that there are no functorial projections to cut metrics, or even to tree metrics. Finally, although we focus less on the construction of clustering methods (clustering domains) derived from injective envelopes, we lay out some preliminary results, that hopefully will give a feel for how the third leg of the stool comes into play., Comment: Minor revisions. 24 pages, 1 figure

Clustering, the process of grouping together similar items into distinct partitions, is a common type of unsupervised machine learning that can be useful for summarizing and aggregating complex multi-dimensional data. However, data can be clustered in many ways, and there exist a large body of algorithms designed to reveal different patterns. While having access to a wide variety of algorithms is helpful, in practice, it is quite difficult for data scientists to choose and parameterize algorithms to get the clustering results relevant for their dataset and analytical tasks. To alleviate this problem, we built Clustervision , a visual analytics tool that helps ensure data scientists find the right clustering among the large amount of techniques and parameters available. Our system clusters data using a variety of clustering techniques and parameters and then ranks clustering results utilizing five quality metrics. In addition, users can guide the system to produce more relevant results by providing task-relevant constraints on the data. Our visual user interface allows users to find high quality clustering results, explore the clusters using several coordinated visualization techniques, and select the cluster result that best suits their task. We demonstrate this novel approach using a case study with a team of researchers in the medical domain and showcase that our system empowers users to choose an effective representation of their complex data.

We propose in this paper a novel approach for unsupervised clustering of services’ behaviors. These behaviors are modeled as multivariate time series that capture the evaluation of several service quality attributes for a period of time. The importance weights of quality attributes are derived based on the Shannon’s entropy concept and the service data is flattened in a format that is convenient for clustering. The flattening process spans over a time oriented aggregation transformation, which leverages Haar reduction. The reduction is modeled as a maximization of an objective function. The absence of ground truth is tackled by performing a set of tests to determine the best number of clusters and clustering algorithms. Extensive experiments were conducted to validate the proposed unsupervised clustering approach.

Clustering of data points has been a profound research avenue in the history of machine learning algorithms. Using learning automata which are autonomous decision making entities, in this paper, the learning automata clustering algorithm is proposed. In learning automata clustering, each data point is affiliated with a learning automaton where the learning automaton determines the cluster membership of that data point. The cluster rectification is done through a reinforcement signal for each learning automaton which is fabricated from the Euclidean distance of that data point and the mean value of its designated cluster. Finally, the learning automata clustering is compared with four centroid-based clustering algorithms, K-means, K-means++, K-medians, and K-medoids and results demonstrate the high clustering accuracy and comparable Silhouette coefficient of the proposed method.

In this paper, a manifold-based dictionary learning method for the bag-of-features (BoF) representation optimized toward information clustering is proposed. First, the spectral representation, which unwraps the manifolds of the data and provides better clustering solutions, is formed. Then, a new dictionary is learned in order to make the histogram space, i.e., the space where the BoF historgrams exist, as similar as possible to the spectral space. The ability of the proposed method to improve the clustering solutions is demonstrated using a wide range of datasets: two image datasets, the 15-scene dataset and the Corel image dataset, one video dataset, the KTH dataset, and one text dataset, the RT-2k dataset. The proposed method improves both the internal and the external clustering criteria for two different clustering algorithms: 1) the -means and 2) the spectral clustering. Also, the optimized histogram space can be used to directly assign a new object to its cluster, instead of using the spectral space (which requires reapplying the spectral clustering algorithm or using incremental spectral clustering techniques). Finally, the learned representation is also evaluated using an information retrieval setup and it is demonstrated that improves the retrieval precision over the baseline BoF representation.

In this paper, an ensemble of clustering trees (ECTs) is adopted to improve the performance of the Fuzzy Min–Max (FMM) network with individual clustering trees. The key advantage of combining FMM and ECT together is to formulate an accurate and useful learning model that is able to perform online clustering and to explain its predictions. The online clustering capability is inherited from the FMM hyperboxes, while the explanatory capability arises from the underlying decision trees of ECT. Four different mean measures, namely harmonic, geometric, arithmetic, and root mean square, are incorporated into FMM for computing its hyperbox centroids. A series of benchmark and real-world data sets are used for evaluating the FMM-ECT performance. The results are analyzed and compared with those from other models. The outcomes indicate that FMM-ECT is able to achieve comparable clustering performances, with the advantage of providing explanations of its predictions using a decision tree.

An original computational approach for cluster analysis is proposed.The method consists of two phases, which are based on Self-Organizing Map.Topology-preserving and connectivity functions are used in the clustering process.The method is proved using three benchmark datasets and a real biological dataset.Automation in parameterization results in a user-friendly methodology. Data clustering is aimed at finding groups of data that share common hidden properties. These kinds of techniques are especially critical at early stages of data analysis where no information about the dataset is available. One of the mayor shortcomings of the clustering algorithms is the difficulty for non-experts users to configure them and, in some cases, interpret the results. In this work a computational approach with a two-layer structure based on Self-Organizing Map (SOM) is presented for cluster analysis. In the first level, a quantization of the data samples using topology-preserving metrics to automatically determine the number of units in the SOM is proposed. In the second level the obtained SOM prototypes are clustered by means of a connectivity analysis to explore the quality of the partitioning with different number of clusters. The most important benefit of this two-layer procedure is that computational load decreases considerably in comparison with data based clustering methods, making it possible to cluster large data sets and to consider several different clustering alternatives in a limited time. This methodology produces a two-dimensional map representation of the, usually, high dimensional input space, along with quantitative information on viable clustering alternatives, which facilitates the exploration of the possible partitions in a dataset. The efficiency and interpretation of the methodology is illustrated by its application to artificial, benchmark and real complex biological datasets. The experimental results demonstrate the ability of the method to identify possible segmentations in a dataset, compared to algorithms that only yield a single clustering solution. The proposed algorithm tackles the intrinsic limitations of SOM and the parameter settings associated with the clustering methodology, without requiring the number of clusters or the SOM architecture as a prerequisite, among others. This way, it makes possible its application even by researchers with a limited expertise in machine learning.

Clustering is one of the main data mining tasks with many proven techniques and successful real-world applications. However, in changing environments, the existing systems need to be regularly updated in order to describe in the best possible way an observed phenomenon at each point in time. Since changes lead to uncertainty, the respective systems also require an adequate modeling of the involved kinds of uncertainty. This paper presents a novel method for dynamic clustering called dynamic rough-fuzzy support vector clustering (D-RFSVC). Its main idea is to take advantage of the knowledge acquired in previous cycles to speed up model updating while tracking the structural changes that clusters can experience over time. The core method of the proposed approach is the well-known support vector clustering algorithm, which can be used for large datasets employing powerful optimization techniques. The computational experiments, together with a conceptual and numerical comparative study, highlight the potential D-RFSVC has in dynamic environments.

Active learning is used for classification when labeling data are costly, while the main challenge is to identify the critical instances that should be labeled. Clustering-based approaches take advantage of the structure of the data to select representative instances. In this paper, we developed the active learning through density peak clustering (ALEC) algorithm with three new features. First, a master tree was built to express the relationships among the nodes and assist the growth of the cluster tree. Second, a deterministic instance selection strategy was designed using a new importance measure. Third, tri-partitioning was employed to determine the action to be taken on each instance during iterative clustering, labeling, and classifying. Experiments were performed with 14 datasets to compare against state-of-the-art active learning algorithms. Results demonstrated that the new algorithm had higher classification accuracy using the same number of labeled data.

We present a closed-loop unsupervised clustering method for motion vectors extracted from highly dynamic video scenes. Motion vectors are assigned to nonconvex homogeneous clusters characterizing direction, size and shape of regions with multiple independent activities. The proposed method is based on support vector clustering. Cluster labels are propagated over time via incremental learning. The proposed method uses a kernel function that maps the input motion vectors into a high-dimensional space to produce nonconvex clusters. We improve the mapping effectiveness by quantifying feature similarities via a blend of position and orientation affinities. We use the Quasiconformal Kernel Transformation to boost the discrimination of outliers. The temporal propagation of the clusters’ identities is achieved via incremental learning based on the concept of feature obsolescence to deal with appearing and disappearing features. Moreover, we design an online clustering performance prediction algorithm used as a feedback that refines the cluster model at each frame in an unsupervised manner. We evaluate the proposed method on synthetic data sets and real-world crowded videos and show that our solution outperforms state-of-the-art approaches.

Clustering in energy markets is a top topic with high significance on expert and intelligent systems. The main impact of is paper is the proposal of a new clustering framework for the automatic classification of electricity customers’ loads. An automatic selection of the clustering classification algorithm is also highlighted. Finally, new customers can be assigned to a predefined set of clusters in the classification phase. The computation time of the proposed framework is less than that of previous classification techniques, which enables the processing of a complete electric company sample in a matter of minutes on a personal computer. The high accuracy of the predicted classification results verifies the performance of the clustering technique. This classification phase is of significant assistance in interpreting the results, and the simplicity of the clustering phase is sufficient to demonstrate the quality of the complete mining framework.

Classification rules should be open for public inspection to ensure fairness.These rules can be originally induced from some dataset. If induced classification rules are supported only by a small number of objects in the dataset, publication can lead to identification of objects supporting the rule, given their speciality. Eventually, it is possible to retrieve information about the identified objects. This identifiability is not desirable in terms of data privacy.In this paper, to avoid such privacy breaches, we propose rule clustering for achievingk-anonymity of all induced rules, i.e., the induced rules are supported by at leastkobjects in the dataset. The proposed approach merges similar rules to satisfyk-anonymity while aiming to maintain the classification accuracy. Two numerical experiments were executed to verify both the accuracy of the classifier with the rules obtained by the proposed method and the ratio of decision classes revealed from leaked information about objects. The experimental results show the usefulness of the proposed method.

Clustering has been widely used to identify possible structures in data and help users to understand data in an unsupervised manner. Traditional clustering methods often provide a single partitioning of the data that groups similar data objects in one group while separates dissimilar ones into different groups. However, it has been well recognized that assuming only a single clustering for a data set can be too strict and cannot capture the diversity in applications. Multiple clustering structures can be hidden in the data, and each represents a unique perspective of the data. Different multi-clustering methods, which aim to discover multiple independent structures hidden in data, have been proposed in the recent decade. Although multi-clustering methods may provide more information for users, it is still challenging for users to efficiently and effectively understand each clustering structure. Subspace multi-clustering methods address this challenge by providing each clustering a feature subspace. Moreover, most subspace multi-clustering methods are especially scalable for high-dimensional data, which has become more and more popular in real applications due to the advances of big data technologies. In this paper, we focus on the subject of subspace multi-clustering, which has not been reviewed by any previous survey. We formulate the subspace multi-clustering problem and categorize the methodologies in different perspectives (e.g., de-coupled methods and coupled methods). We compare different methods on a series of specific properties (e.g., input parameters and different kinds of subspaces) and analyze the advantages and disadvantages. We also discuss several interesting and meaningful future directions.

© 1989-2012 IEEE. Spectral clustering has been playing a vital role in various research areas. Most traditional spectral clustering algorithms comprise two independent stages (e.g., first learning continuous labels and then rounding the learned labels into discrete ones), which may cause unpredictable deviation of resultant cluster labels from genuine ones, thereby leading to severe information loss and performance degradation. In this work, we study how to achieve discrete clustering as well as reliably generalize to unseen data. We propose a novel spectral clustering scheme which deeply explores cluster label properties, including discreteness, nonnegativity, and discrimination, as well as learns robust out-of-sample prediction functions. Specifically, we explicitly enforce a discrete transformation on the intermediate continuous labels, which leads to a tractable optimization problem with a discrete solution. Besides, we preserve the natural nonnegative characteristic of the clustering labels to enhance the interpretability of the results. Moreover, to further compensate the unreliability of the learned clustering labels, we integrate an adaptive robust module with ℓ 2,p loss to learn prediction function for grouping unseen data. We also show that the out-of-sample component can inject discriminative knowledge into the learning of cluster labels under certain conditions. Extensive experiments conducted on various data sets have demonstrated the superiority of our proposal as compared to several existing clustering approaches.

We propose a multi-task clustering method by transferring knowledge of instances.The sample distance in different tasks is reweighted by learning a shared subspace.Related samples from other tasks are reused as auxiliary data to aid clustering.Our method maintains the label marginal distribution of each individual task.Better performance is observed compared with other multi-task clustering methods. Clustering is an essential issue in machine learning and data mining. As there are many related tasks in the real world, multi-task clustering, which improves the clustering performance of each task by transferring knowledge across the related tasks, receives increasing attention recently. Generally knowledge transfer can be accomplished in different ways. Nevertheless, besides transferring knowledge of feature representations, other knowledge transfer ways have seldom been adopted for multi-task clustering. In this paper, we propose a general multi-task clustering algorithm by transferring knowledge of instances. Our algorithm reweights the distance between samples in different tasks by learning a shared subspace, then selects the nearest neighbors for each sample from the other tasks in the learned shared subspace as the auxiliary data to aid the clustering process of each individual task. Experiments on real data sets in text mining and image mining demonstrate that our proposed algorithm outperforms the traditional single-task clustering methods and existing cross-domain multi-task clustering methods.

In this article, a new recursive evolving clustering method is proposed based on the well-known Gustafson–Kessel algorithm. The novelty of the proposed method involves the adaptation and integration of the mutual information based formulation to accommodate the Mahalanobis distance, which functions as the similarity measure and the unification of the clustering generation and pruning mechanisms. Example applications of the method are also discussed in the areas of data compression and knowledge extraction.

In this study, a new perspective on the application of the clustering approach is proposed. The perspective aims to identify the values of the parameters of clustering, including the choice of the algorithm itself, which lead to a possibly faithful rendering of a partition of data, which is known a priori. Motivation and possible interpretations are discussed which can be associated with such a reverse identification process. The essential motivation is associated, but not limited, to the primary objective of cluster analysis, i.e. gaining insight into the structure of the given data-set or family of data-sets. We propose to use evolutionary strategies for reverse analysis to be carried out in view of the characteristics of the problem considered. The concept and the feasibility of the proposed computational approach are illustrated by the analysis of an exemplary data-set. The preliminary results obtained are promising in both technical and cognitive terms.

A new initialization technique is proposed to improve the performance of K-means.A data transformation approach is proposed to solve empty cluster problem.An efficient method is proposed to estimate the optimal number of clusters.Proposed clustering method provides more accurate clustering results. Clustering provides a knowledge acquisition method for intelligent systems. This paper proposes a novel data-clustering algorithm, by combining a new initialization technique, K-means algorithm and a new gradual data transformation approach to provide more accurate clustering results than the K-means algorithm and its variants by increasing the clusters coherence. The proposed data transformation approach solves the problem of generating empty clusters, which frequently occurs for other clustering algorithms. An efficient method based on the principal component transformation and a modified silhouette algorithm is also proposed in this paper to determine the number of clusters. Several different data sets are used to evaluate the efficacy of the proposed method to deal with the empty cluster generation problem and its accuracy and computational performance in comparison with other K-means based initialization techniques and clustering methods. The developed estimation method for determining the number of clusters is also evaluated and compared with other estimation algorithms. Significances of the proposed method include addressing the limitations of the K-means based clustering and improving the accuracy of clustering as an important method in the field of data mining and expert systems. Application of the proposed method for the knowledge acquisition in time series data such as wind, solar, electric load and stock market provides a pre-processing tool to select the most appropriate data to feed in neural networks or other estimators in use for forecasting such time series. In addition, utilization of the knowledge discovered by the proposed K-means clustering to develop rule based expert systems is one of the main impacts of the proposed method.

Unsupervised feature selection (UFS) aims to reduce the time complexity and storage burden, improve the generalization ability of learning machines by removing the redundant, irrelevant and noisy features. Due to the lack of training labels, most existing UFS methods generate the pseudo labels by spectral clustering, matrix factorization or dictionary learning, and convert UFS to a supervised problem. The learned clustering labels reflect the data distribution with respect to classes and therefore are vital to the UFS performance. In this paper, we proposed a novel subspace clustering guided unsupervised feature selection (SCUFS) method. The clustering labels of the training samples are learned by representation based subspace clustering, and features that can well preserve the cluster labels are selected. SCUFS can well learn the data distribution in that it uncovers the underlying multi-subspace structure of the data and iteratively learns the similarity matrix and clustering labels. Experimental results on benchmark datasets for unsupervised feature selection show that SCUFS outperforms the state-of-the-art UFS methods. HighlightsA novel subspace clustering guided unsupervised feature selection (SCUFS) model is proposed.SCUFS learns a similarity graph by self-representation of samples and can uncover the underlying multi-subspace structure of data.The iterative updating of similarity graph and pseudo label matrix can learn a more accurate data distribution.

The text clustering technique is an appropriate method used to partition a huge amount of text documents into groups. The documents size affects the text clustering by decreasing its performance. Subsequently, text documents contain sparse and uninformative features, which reduce the performance of the underlying text clustering algorithm and increase the computational time. Feature selection is a fundamental unsupervised learning technique used to select a new subset of informative text features to improve the performance of the text clustering and reduce the computational time. This paper proposes a hybrid of particle swarm optimization algorithm with genetic operators for the feature selection problem. The k-means clustering is used to evaluate the effectiveness of the obtained features subsets. The experiments were conducted using eight common text datasets with variant characteristics. The results show that the proposed algorithm hybrid algorithm (H-FSPSOTC) improved the performance of the clustering algorithm by generating a new subset of more informative features. The proposed algorithm is compared with the other comparative algorithms published in the literature. Finally, the feature selection technique encourages the clustering algorithm to obtain accurate clusters.

Image content clustering is an effective way to organize large databases thereby making the content based image retrieval process much easier. However, clustering of images with varied background and foreground is quite challenging. In this paper, we propose a novel image content clustering paradigm suitable for clustering large and diverse image databases. In our approach images are represented in a continuous domain based on a probabilistic Gaussian Mixture Model (GMM) with the images modeled as mixture of Gaussian distributions in the selected feature space. The distance metric between the Gaussian distributions is defined in the sense of Kullback–Leibler (KL) divergence. The clustering is done using a semi-supervised learning framework where labeled data in the form of cluster templates is used to classify the unlabelled data. The clusters are formed around initially chosen seeds and are updated in the due course based on user inputs. In our clustering approach the user interaction is done in a structured way as to get maximum inputs from the user in a limited time. We propose two methods to carry out the structured user interaction using which the cluster templates are updated to improve the quality of the clusters formed. The proposed method is experimentally evaluated on benchmark datasets that are specifically chosen to include a wide variation of images around a common theme that is typically encountered in applications like photo-summarization and poses a major semantic gap challenge to conventional clustering approaches. The experimental results presented demonstrate the effectiveness of the proposed approach.

Article history: Received December 5, 2015 Received in revised format February 16 2016 Accepted August 15 2016 Available online August 16 2016 Clustering is absolutely useful information to explore data structures and has been employed in many places. It organizes a set of objects into similar groups called clusters, and the objects within one cluster are both highly similar and dissimilar with the objects in other clusters. The K-mean, C-mean, Fuzzy C-mean and Kernel K-mean algorithms are the most popular clustering algorithms for their easy implementation and fast work, but in some cases we cannot use these algorithms. Regarding this, in this paper, a hybrid model for customer clustering is presented that is applicable in five banks of Fars Province, Shiraz, Iran. In this way, the fuzzy relation among customers is defined by using their features described in linguistic and quantitative variables. As follows, the customers of banks are grouped according to K-mean, C-mean, Fuzzy C-mean and Kernel K-mean algorithms and the proposed Fuzzy Relation Clustering (FRC) algorithm. The aim of this paper is to show how to choose the best clustering algorithms based on density-based clustering and present a new clustering algorithm for both crisp and fuzzy variables. Finally, we apply the proposed approach to five datasets of customer's segmentation in banks. The result of the FCR shows the accuracy and high performance of FRC compared other clustering methods. Growing Science Ltd. All rights reserved. 7 © 201

The occurrence of series of events is always associated with the news report, social network, and Internet media. In this paper, a detecting system for public security events is designed, which carries out clustering operation to cluster relevant text data, in order to benefit relevant departments by evaluation and handling. Firstly, texts are mapped into three-dimensional space using the vector space model. Then, to overcome the shortcoming of the traditional clustering algorithm, an improved fuzzy c-means (FCM) algorithm based on adaptive genetic algorithm and semisupervised learning is proposed. In the proposed algorithm, adaptive genetic algorithm is employed to select optimal initial clustering centers. Meanwhile, motivated by semisupervised learning, guiding effect of prior knowledge is used to accelerate iterative process. Finally, simulation experiments are conducted from two aspects of qualitative analysis and quantitative analysis, which demonstrate that the proposed algorithm performs excellently in improving clustering centers, clustering results, and consuming time.

Data mining is a process of extracting patterns from a large datasets. We are trying to uncover the data bonded features which are hard to visualize and if many feature exists for the data then it becomes difficult to analyze the data. The main aim of the scheme of work is to classify the features and over which try to classify the data sets. Machine learning is one of the techniques of Artificial Intelligence which is used for extracting valuable knowledge from large data base. Machine Learning is also used for extracting patterns, models in data. In this paper we are trying to group the data based on multi-dimensional feature classification. Clustering process makes the similar features to form into one group and or else multiple groups, here in we try to group the features which are similar and form multiple groups. The US schooling data is in the form of flat files. Classification process is performed on the raw data. Classification is performed according to hierarchical clustering. Filtration process is used in order to obtain non-zero values. The saturation points are generated by performing clustering. Based on the clusters obtained the patterns can be extracted. Attribute based classification and hierarchical clustering is performed on the data. The attributes obtained are named as income and expenses. By combining the income and expenses attributes patterns can be identified. By using some combination of the two attributes some patterns have been obtain. By performing clustering on all the combinations of each attribute we can identify the patterns. Then according to this process we find the patterns generated by the feature classification or clustering process against the data classification or clustering. This new methodology tries to give the co-relation between the data and properties of a data set and how they behave.

Creating sessions in scientific conferences consists in grouping papers with common topics taking into account the size restrictions imposed by the conference schedule. Therefore, this problem can be considered as semi-supervised clustering of documents based on their content. This paper aims to propose modifications in traditional clustering algorithms to incorporate size constraints in each cluster. Specifically, two new algorithms are proposed to semi-supervised clustering, based on: binary integer linear programming with cannot-link constraints and a variation of the K-Medoids algorithm, respectively. The applicability of the proposed semi-supervised clustering methods is illustrated by addressing the problem of automatic configuration of conference schedules by clustering articles by similarity. We include experiments, applying the new techniques, over real conferences datasets: ICMLA-2014, AAAI-2013 and AAAI-2014. The results of these experiments show that the new methods are able to solve practical and real problems.

Clustering can be improved with the help of side information about the similarity relationships among instances. Such information has been commonly represented by two types of constraints: pairwise constraints and relative constraints, regarding similarities about instance pairs and triplets, respectively. Prior work has mostly considered these two types of constraints separately and developed individual algorithms to learn from each type. In practice, however, it is critical to understand/compare the usefulness of the two types of constraints as well as the cost of acquiring them, which has not been studied before. This paper provides an extensive comparison of clustering with these two types of constraints. Specifically, we compare their impacts both on human users that provide such constraints and on the learning system that incorporates such constraints into clustering. In addition, to ensure that the comparison of clustering is performed on equal ground (without the potential bias introduced by different learning algorithms), we propose a probabilistic semi-supervised clustering framework that can learn from either type of constraints. Our experiments demonstrate that the proposed semi-supervised clustering framework is highly effective at utilizing both types of constraints to aid clustering. Our user study provides valuable insights regarding the impact of the constraints on human users, and our experiments on clustering with the human-labeled constraints reveal that relative constraint is often more efficient at improving clustering.

Fuzzy c-means (FCM) clustering has been successfully applied in various pattern recognition areas. While FCM is gaining attention, an important issue arising from these studies is the need to determine which attributes of the data should be used. Answering this question is difficult, because there is no labeled training data available in clustering to guide the search. We present a feature selection for FCM. The advantage of our method is that it is intuitively appealing, avoiding combinatorial searches, and allowing us to prune the feature set. Our method is also adaptable and can change through complex scenes in an online environment. We do not have to wait until all data have been generated before learning begins. Finally, to estimate the model parameters, the gradient method is adopted to minimize the fuzzy objective function with the Kullback–Leibler divergence information. Numerical experiments are presented to demonstrate the robustness and accuracy of our method.

Clustering learning and classification learning are two major tasks in pattern recognition. The traditional hybrid clustering and classification algorithms handle them in a sequential way rather than a simultaneous way. Fortunately, multiobjective optimization provides a way to solve this problem. In this paper, an algorithm that learns simultaneous clustering and classification adaptively via multiobjective evolutionary algorithm is proposed. The main idea of this paper is to optimize two objective functions which represent fuzzy cluster connectedness and classification error rate to achieve the goal of simultaneous learning. Firstly, we adopt a graph based representation scheme to encode so that it can generate a set of solutions with different number of clusters in a single run. Then the relationship between clustering and classification is built via the Bayesian theory during the optimization process. The quality of clustering and classification is measured by the objective functions and the feedback drawn from both aspects is used to guide the mutation. At last, a set of nondominated solutions are generated, from which the final Pareto optimal solution is selected by using Adjusted Rand Index. The results on synthetic datasets and real-life datasets demonstrate the rationality and effectiveness of the proposed algorithm. Furthermore, we apply the proposed algorithm to image segmentation including texture images and synthetic aperture radar images, the experimental results show the superiority of the proposed algorithm compared with other five algorithms. HighlightsA simultaneous adaptive clustering and classification learning via MOEA is proposed.New clustering objective function is designed, and two objective functions complement with each other.The number of clusters can be determined adaptively during the learning process.The drawback from clustering/classification learning is used to guide the search.We extend it to texture image and SAR image segmentation to show its effectiveness.

Multi-task clustering and multi-view clustering have severally found wide applications and received much attention in recent years. Nevertheless, there are many clustering problems that involve both multi-task clustering and multi-view clustering, i.e., the tasks are closely related and each task can be analyzed from multiple views. In this paper, we introduce a multi-task multi-view clustering framework which integrates within-view-task clustering, multi-view relationship learning, and multi-task relationship learning. Under this framework, we propose two multi-task multi-view clustering algorithms, the bipartite graph based multi-task multi-view clustering algorithm, and the semi-nonnegative matrix tri-factorization based multi-task multi-view clustering algorithm. The former one can deal with the multi-task multi-view clustering of nonnegative data, the latter one is a general multi-task multi-view clustering method, i.e., it can deal with the data with negative feature values. Experimental results on publicly available data sets in web page mining and image mining show the superiority of the proposed multi-task multi-view clustering algorithms over either multi-task clustering algorithms or multi-view clustering algorithms for multi-task clustering of multi-view data.

A hybrid intelligent model comprising a modified fuzzy min-max (FMM) clustering neural network and a modified clustering tree (CT) is developed. A review of clustering models with rule extraction capabilities is presented. The hybrid FMM-CT model is explained. We first use several benchmark problems to illustrate the cluster evolution patterns from the proposed modifications in FMM. Then, we employ a case study with real data related to power quality monitoring to assess the usefulness of FMM-CT. The results are compared with those from other clustering models. More importantly, we extract explanatory rules from FMM-CT to justify its predictions. The empirical findings indicate the usefulness of the proposed model in tackling data clustering and power quality monitoring problems under different environments.

Meta-clustering is a popular approach for finding multiple clusterings in the dataset, taking a large number of base clusterings as input for further user navigation and refinement. However, the effectiveness of meta-clustering is highly dependent on the distribution of the base clusterings and open challenges exist with regard to its stability and noise tolerance. In addition, the clustering views returned may not all be relevant, hence there is open challenge on how to rank those clustering views. In this paper we propose a simple and effective filtering algorithm that can be flexibly used in conjunction with any meta-clustering method. In addition, we propose an unsupervised method to rank the returned clustering views. We evaluate the framework (rFILTA) on both synthetic and real-world datasets, and see how its use can enhance the clustering view discovery for complex scenarios.

Fuzzy clustering is a useful segmentation tool which has been widely used in many applications in real life problems such as in pattern recognition, recommender systems, forecasting, etc. Fuzzy clustering algorithm on picture fuzzy set (FC-PFS) is an advanced fuzzy clustering algorithm constructed on the basis of picture fuzzy set with the appearance of three membership degrees namely the positive, the neutral and the refusal degrees combined within an entropy component in the objective function to handle the problem of incomplete modeling in fuzzy clustering. A disadvantage of FC-PFS is its capability to handle complex data which include mix data type (categorical and numerical data) and distinct structured data. In this paper, we propose a novel picture fuzzy clustering algorithm for complex data called PFCA-CD that deals with both mix data type and distinct data structures. The idea of this method is the modification of FC-PFS, using a new measurement for categorical attributes, multiple centers of one cluster and an evolutionary strategy – particle swarm optimization. Experiments indicate that the proposed algorithm results in better clustering quality than others through clustering validity indices.

Nowadays, data always have multiple representations, and a good feature representation usually leads to a good clustering performance. Existing multi-view clustering works generally integrate multiple complementary information to gain better clustering performance rather than relying on a single view. However, these works usually focus on the combination of information rather than improving the feature representation capability of each view. As a new method, extreme learning machine (ELM) has excellent feature representation capability, easy parameter selection, and promising performance in various clustering tasks. This paper proposes a novel multi-view clustering framework with ELM to further improve clustering performance, and implements three algorithms based on this framework. In this framework, the normalized features of each individual view are mapped onto a higher dimensional feature space by the ELM random mapping. Afterwards, the unsupervised multi-view clustering is performed in this feature space. Thus far, this is the first work on multi-view clustering with ELM. Numerous baseline methods on five real-world datasets are empirically compared to show the effectiveness of the proposed algorithms. As indicated, the proposed algorithms yield superior clustering performance when compared with several state-of-art multi-view clustering methods in recent literatures.

PurposeThe quick growth of web-based and mobile e-learning applications such as massive open online courses have created a large volume of online learning resources. Confronting such a large amount of learning data, it is important to develop effective clustering approaches for user group modeling and intelligent tutoring. The paper aims to discuss these issues.Design/methodology/approachIn this paper, a minimum spanning tree based approach is proposed for clustering of online learning resources. The novel clustering approach has two main stages, namely, elimination stage and construction stage. During the elimination stage, the Euclidean distance is adopted as a metrics formula to measure density of learning resources. Resources with quite low densities are identified as outliers and therefore removed. During the construction stage, a minimum spanning tree is built by initializing the centroids according to the degree of freedom of the resources. Online learning resources are subsequently partitioned into clusters by exploiting the structure of minimum spanning tree.FindingsConventional clustering algorithms have a number of shortcomings such that they cannot handle online learning resources effectively. On the one hand, extant partitional clustering methods use a randomly assigned centroid for each cluster, which usually cause the problem of ineffective clustering results. On the other hand, classical density-based clustering methods are very computationally expensive and time-consuming. Experimental results indicate that the algorithm proposed outperforms the traditional clustering algorithms for online learning resources.Originality/valueThe effectiveness of the proposed algorithms has been validated by using several data sets. Moreover, the proposed clustering algorithm has great potential in e-learning applications. It has been demonstrated how the novel technique can be integrated in various e-learning systems. For example, the clustering technique can classify learners into groups so that homogeneous grouping can improve the effectiveness of learning. Moreover, clustering of online learning resources is valuable to decision making in terms of tutorial strategies and instructional design for intelligent tutoring. Lastly, a number of directions for future research have been identified in the study.

Clustering methods play an important role in data mining and various other applications. This work investigates them based on swarm intelligence. It proposes a new clustering method by combining K-means clustering method and mussels wandering optimization algorithm. A single cluster method is well recognized to achieve limited performance when it is compared with a clustering ensemble (CE) that integrates several single ones. Hence, this work introduces a new CE method called weight-incorporated similarity- based CE. The commonly-used datasets with varying size are used to test the performance of the proposed methods. The simulation results illustrate the validity and performance advantages of the proposed ones over some of their peers.

Missing values are a common phenomenon when dealing with real-world data sets. Analysis of incomplete data sets has become an active area of research. In this paper, we focus on the problem of clustering incomplete data, which is intended to introduce some prior distribution information of the missing values into the algorithm of fuzzy clustering. First, non-parametric hypothesis testing is employed to describe the missing values adhering to a certain Gaussian distribution as probabilistic information granules based on the nearest neighbors of incomplete data. Second, we propose a novel clustering model, in which probabilistic information granules of missing values are incorporated into the Fuzzy C-Means clustering of incomplete data by involving the maximum likelihood criterion. Third, the clustering model is optimized by using a tri-level alternating optimization utilizing the method of Lagrange multipliers. The convergence and the time complexity of the clustering algorithm are also discussed. The experiments reported both on synthetic and real-world data sets demonstrate that the proposed approach can effectively realize clustering of incomplete data.

Existing clustering-based recommendation methods generally focus on the clustering of users, items or social trust relationships. Although demonstrated to be efficient and scalable to large-scale datasets, these methods are sensitive to the quality of clustering and still suffer from the problem of low accuracy. In order to solve this issue, in this paper, we propose a multi-type clustering based recommendation framework which systematically considers the trust-based user clustering, similarity-based user clustering and similarity-based item clustering to further improve the recommendation accuracy. A SVD (Singular Value Decomposition) signs-based community mining method is utilized to process the trust and distrust matrix in order to discover the trust-based user clusters. The PLSA (Probabilistic Latent Semantic Analysis)-based model is employed to explore the similarity-based user and item clusters. Then a clustering-based trust regularization term is proposed to incorporate the trust-based user clusters into the matrix factorization model. Comparative experiments on two real-world datasets demonstrate that our approach can better address the issues of data sparsity and cold start, and outperforms other state-of-the-art methods in terms of RMSE and MAE.

In recent years, combining multiple sources or views of datasets for data clustering has been a popular practice for improving clustering accuracy. As different views are different representations of the same set of instances, we can simultaneously use information from multiple views to improve the clustering results generated by the limited information from a single view. Previous studies mainly focus on the relationships between distinct data views, which would get some improvement over the single-view clustering. However, in the case of high-dimensional data, where each view of data is of high dimensionality, feature selection is also a necessity for further improving the clustering results. To overcome this problem, this paper proposes a novel algorithm termed Weighted Multi-view Clustering with Feature Selection (WMCFS) that can simultaneously perform multi-view data clustering and feature selection. Two weighting schemes are designed that respectively weight the views of data points and feature representations in each view, such that the best view and the most representative feature space in each view can be selected for clustering. Experimental results conducted on real-world datasets have validated the effectiveness of the proposed method. HighlightsThis paper proposes a new multi-view data clustering algorithm.The new method considers both view weighting and feature weighting.An EM-like method is designed to get the local optimum solution.Extensive experiments have been conducted to show the effectiveness.

The availability of many heterogeneous but related views of data has arisen in numerous clustering problems. Different views encode distinct representations of the same data, which often admit the same underlying cluster structure. The goal of multi-view clustering is to properly combine information from multiple views so as to generate high quality clustering results that are consistent across different views. Based on max-product belief propagation, we propose a novel multi-view clustering algorithm termed multi-view affinity propagation (MVAP). The basic idea is to establish a multi-view clustering model consisting of two components, which measure the within-view clustering quality and the explicit clustering consistency across different views, respectively. Solving this model is NP-hard, and a multi-view affinity propagation is proposed, which works by passing messages both within individual views and across different views. However, the exemplar consistency constraint makes the optimization almost impossible. To this end, by using some previously designed mathematical techniques, the messages as well as the cluster assignment vector computations are simplified to get simple yet functionally equivalent computations. Experimental results on several real-world multi-view datasets show that MVAP outperforms existing multi-view clustering algorithms. It is especially suitable for clustering more than two views.

Real world datasets often consist of data expressed through multiple modalities. Clustering such datasets is in most cases a challenging task as the involved modalities are often heterogeneous. In this paper we propose a graph-based multimodal clustering approach. The proposed approach utilizes an example relevant clustering in order to learn a model of the "same cluster" relationship between a pair of items. This model is subsequently used in order to organize the items of the collection to be clustered in a graph, where the nodes represent the items and a link between a pair of nodes exists if the model predicted that the corresponding pair of items belong to the same cluster. Eventually, a graph clustering algorithm is applied on the graph in order to produce the final clustering. The proposed approach is applied on two problems that are typically treated using clustering techniques; in particular, it is applied on the problem of detecting social events and to the problem of discovering different landmark views in collections of social multimedia.

Traditional cluster ensemble approaches have three limitations: ( $1$ ) They do not make use of prior knowledge of the datasets given by experts. ( $2$ ) Most of the conventional cluster ensemble methods cannot obtain satisfactory results when handling high dimensional data. ( $3$ ) All the ensemble members are considered, even the ones without positive contributions. In order to address the limitations of conventional cluster ensemble approaches, we first propose an incremental semi-supervised clustering ensemble framework (ISSCE) which makes use of the advantage of the random subspace technique, the constraint propagation approach, the proposed incremental ensemble member selection process, and the normalized cut algorithm to perform high dimensional data clustering. The random subspace technique is effective for handling high dimensional data, while the constraint propagation approach is useful for incorporating prior knowledge. The incremental ensemble member selection process is newly designed to judiciously remove redundant ensemble members based on a newly proposed local cost function and a global cost function, and the normalized cut algorithm is adopted to serve as the consensus function for providing more stable, robust, and accurate results. Then, a measure is proposed to quantify the similarity between two sets of attributes, and is used for computing the local cost function in ISSCE. Next, we analyze the time complexity of ISSCE theoretically. Finally, a set of nonparametric tests are adopted to compare multiple semi-supervised clustering ensemble approaches over different datasets. The experiments on 18 real-world datasets, which include six UCI datasets and 12 cancer gene expression profiles, confirm that ISSCE works well on datasets with very high dimensionality, and outperforms the state-of-the-art semi-supervised clustering ensemble approaches.

Uncertain data exist in many application fields, and there are numerous recent efforts in processing uncertain data to get more reliable results, especially uncertainty processing in clustering method. However, it is one of the urgent challenges to discover clusters with specific shape features. So we present a clustering method for data with uncertainties, and it is called multivariate wavelet principal component analysis-improved clustering using references and density (MWPCA-ICURD), which utilizes feature extraction and density-based clustering. To cluster uncertain data with specific shape original features, the original features are extracted by MWPCA method, which combines digital wavelet decomposition and principal component analysis organically. Then, a density-based clustering method ICURD is explored to discover specific shape clusters. Experimental results illustrate its validation and feasibility.

In this Paper the focus is given on data clustering using Modified Teaching–Learning Based Optimization (MTLBO) a hybridization technique of TLBO. Unlike TLBO, this population based method works on the effect of influence of a teacher on learners to find the optimum solution and it has been used for clustering. The motivation behind the data clustering is to find inherent structure in the data items and grouping then on the basis of their mutual similarity. The effectiveness of the method is tested on many benchmark problems with different characteristics and the results are compared with other population based methods and finally it is implemented on clustering using neural network in data mining.

We describe the algorithm underlying PKOM, a methodology for clustering, analysis and visualization of multi-dimensional data onto a two-dimensional map. PKOM is based on the mixture of two very popular methods that have been widely used by the pharmaceutical industry for the clustering of genomic or SAR (Structure Activity Relationship) chemical information. The first method at the origin of PKOM is SOM (Self-Organizing Maps), a clustering technique based on neural networks. The second method is TREE MAPS, a visualization method based on hierarchical clustering by dendrograms. We initially describe herein the two methods and the reasons why we have taken the best of both to merge them into PKOM. We then describe in detail the PKOM algorithm and its advantages compared to the two former. Examples are given on how to apply this kind of 2-D topological clustering technique to the organization of big pharmaceutical collections in practical cases.