Search

Clustering methods play an important role in data mining and various other applications. This work investigates them based on swarm intelligence. It proposes a new clustering method by combining K-means clustering method and mussels wandering optimization algorithm. A single cluster method is well recognized to achieve limited performance when it is compared with a clustering ensemble (CE) that integrates several single ones. Hence, this work introduces a new CE method called weight-incorporated similarity- based CE. The commonly-used datasets with varying size are used to test the performance of the proposed methods. The simulation results illustrate the validity and performance advantages of the proposed ones over some of their peers.

Missing values are a common phenomenon when dealing with real-world data sets. Analysis of incomplete data sets has become an active area of research. In this paper, we focus on the problem of clustering incomplete data, which is intended to introduce some prior distribution information of the missing values into the algorithm of fuzzy clustering. First, non-parametric hypothesis testing is employed to describe the missing values adhering to a certain Gaussian distribution as probabilistic information granules based on the nearest neighbors of incomplete data. Second, we propose a novel clustering model, in which probabilistic information granules of missing values are incorporated into the Fuzzy C-Means clustering of incomplete data by involving the maximum likelihood criterion. Third, the clustering model is optimized by using a tri-level alternating optimization utilizing the method of Lagrange multipliers. The convergence and the time complexity of the clustering algorithm are also discussed. The experiments reported both on synthetic and real-world data sets demonstrate that the proposed approach can effectively realize clustering of incomplete data.

Existing clustering approaches are usually restricted to crisp clustering, where objects just belong to one cluster; meanwhile there are some applications where objects could belong to more than one cluster. In addition, existing clustering approaches usually analyze static datasets in which objects are kept unchanged after being processed; however many practical datasets are dynamically modified which means some previously learned patterns have to be updated accordingly. In this paper, we propose a new tree-based incremental overlapping clustering method using the three-way decision theory. The tree is constructed from representative points introduced by this paper, which can enhance the relevance of the search result. The overlapping cluster is represented by the three-way decision with interval sets, and the three-way decision strategies are designed to updating the clustering when the data increases. Furthermore, the proposed method can determine the number of clusters during the processing. The experimental results show that it can identifies clusters of arbitrary shapes and does not sacrifice the computing time, and more results of comparison experiments show that the performance of proposed method is better than the compared algorithms in most of cases.

In recent years, a great deal of manifold clustering algorithms was presented to identify the subsets of the manifolds data. Meanwhile, numerous classification algorithms were also developed to classified data shaped in the form of manifold. However, nearly none of them pay attention to the statistical relationship between the manifold structures and class labels, thus failing to discover the knowledge concealed in data. In this paper, a manifold learning framework for both clustering and classification is presented, which involves two steps. In the first step, the clustering through ranking on manifolds is executed to explore structures in data; in the second step, the class posterior probability is calculated by using the Bayesian rule. The core of this framework lies in employing the Bayesian theory to establish the relationship between manifolds and classes thus creates a bridge between clustering learning and classification learning. Our new manifold learning framework is interesting from a number of perspectives: (1) our algorithm can perform manifold clustering learning which can auto-determine the clustering parameters without manual determining; (2) our algorithm can perform manifold classification learning which models the posterior probabilities p ( ω l | x i ) by using the Bayesian rule; (3) our algorithm can provide the statistical relationship between the manifold structure and the given classes. Encouraging experimental results are obtained on 2 artificial and 16 real-life benchmark datasets.

Pattern-based clustering algorithms return a set of patterns that describe the objects of each cluster. The most recent algorithms proposed in this approach extract patterns on numerical datasets by applying an a priori discretization process, which may cause information loss. In this paper, we introduce a new pattern-based clustering algorithm for numerical datasets, which does not need an a priori discretization on numerical features. The new algorithm extracts, from a collection of trees generated through a new induction procedure, a small subset of patterns useful for clustering. Experimental results show that the patterns extracted by the proposed algorithm allows to build a pattern-based clustering algorithm, which obtains better clustering results than recent pattern-based clustering algorithms. In addition, the proposed algorithm obtains similar clustering results, in quality, than traditional clustering algorithms.

Most of kmeans-type clustering algorithms rely on only intra-cluster compactness, i.e. the dispersions of a cluster. Inter-cluster separation which is widely used in classification algorithms, however, is rarely considered in a clustering process. In this paper, we present a new discriminative subspace kmeans-type clustering algorithm (DSKmeans), which integrates the intra-cluster compactness and the inter-cluster separation simultaneously. Different to traditional weighting kmeans-type algorithms, a 3-order tensor is constructed to evaluate the importance of different features in order to integrate the aforementioned two types of information. First, a new objective function for clustering is designed. To optimize the objective function, the corresponding updating rules for the algorithm are then derived analytically. The properties and performance of DSKmeans are investigated on several numerical and categorical data sets. Experimental results corroborate that our proposed algorithm outperforms the state-of-the-art kmeans-type clustering algorithms with respects to four metrics: Accuracy, RandIndex, Fscore and Normal Mutual Information(NMI).

Categorical data clustering has attracted much attention recently due to the fact that much of the data contained in today's databases is categorical in nature. While many algorithms for clustering categorical data have been proposed, some have low clustering accuracy while others have high computational complexity. This research proposes mean gain ratio (MGR), a new information theory based hierarchical divisive clustering algorithm for categorical data. MGR implements clustering from the attributes viewpoint which includes selecting a clustering attribute using mean gain ratio and selecting an equivalence class on the clustering attribute using entropy of clusters. It can be run with or without specifying the number of clusters while few existing clustering algorithms for categorical data can be run without specifying the number of clusters. Experimental results on nine University of California at Irvine (UCI) benchmark and ten synthetic data sets show that MGR performs better as compared to baseline algorithms in terms of its performance and efficiency of clustering.

Several clustering analysis techniques for categorical data exist to divide similar objects into groups. Some are able to handle uncertainty in the clustering process, whereas others have stability issues. In this paper, we propose a new technique called TMDP (Total Mean Distribution Precision) for selecting the partitioning attribute based on probabilistic rough set theory. On the basis of this technique, with the concept of granularity, we derive a new clustering algorithm, MTMDP (Maximum Total Mean Distribution Precision), for categorical data. The MTMDP algorithm is a robust clustering algorithm that handles uncertainty in the process of clustering categorical data. We compare the MTMDP algorithm with the MMR (Min–Min–Roughness) algorithm which is the most relevant clustering algorithm, and also compared it with other unstable clustering algorithms, such as k-modes, fuzzy k-modes and fuzzy centroids. The experimental results indicate that the MTMDP algorithm can be successfully used to analyze grouped categorical data because it produces better clustering results.

The discovery, extraction and analysis of knowledge from data rely generally upon the use of unsupervised learning methods, in particular clustering approaches. Much recent research in clustering and data engineering has focused on the consideration of finite mixture models which allow to reason in the face of uncertainty and to learn by example. The adoption of these models becomes a challenging task in the presence of outliers and in the case of high-dimensional data which necessitates the deployment of feature selection techniques. In this paper we tackle simultaneously the problems of cluster validation (i.e. model selection), feature selection and outliers rejection when clustering positive data. The proposed statistical framework is based on the generalized inverted Dirichlet distribution that offers a more practical and flexible alternative to the inverted Dirichlet which has a very restrictive covariance structure. The learning of the parameters of the resulting model is based on the minimization of a message length objective incorporating prior knowledge. We use synthetic data and real data generated from challenging applications, namely visual scenes and objects clustering, to demonstrate the feasibility and advantages of the proposed method.

Temporal data have many distinct characteristics, including high dimensionality, complex time dependency, and large volume, all of which make the temporal data clustering more challenging than conventional static datasets. In this paper, we propose a HMM-based partitioning ensemble based on hierarchical clustering refinement to solve the problems of initialization and model selection for temporal data clustering. Our approach results four major benefits, which can be highlighted as: (i) the model initialization problem is solved by associating the ensemble technique; (ii) the appropriate cluster number can be automatically determined by applying proposed consensus function on the multiple partitions obtained from the target dataset during clustering ensemble phase; (iii) no parameter re-estimation is required for the new merged pair of cluster, which significantly reduces the computing cost of its final refinement process based on HMM agglomerative clustering; and finally (iv) the composite model is better in characterizing the complex structure of clusters. Our approach has been evaluated on synthetic data and time series benchmark, and yields promising results for clustering tasks.

Since the advent of data clustering, the original formulation of the clustering problem has been enriched to incorporate a number of twists to widen its range of application. In particular, recent heuristic approaches have proposed to incorporate restrictions on the size of the clusters, while striving to minimize a measure of dissimilarity within them. Such size constraints effectively constitute a way to exploit prior knowledge, readily available in many scenarios, which can lead to an improved performance in the clustering obtained. In this paper, we build upon a modification of the celebrated k-means method resorting to a similar alternating optimization procedure, endowed with additive partition weights controlling the size of the partitions formed, adjusted by means of the Levenberg-Marquardt algorithm. We propose several further variations on this modification, in which different kinds of additional information are present. We report experimental results on various standardized datasets, demonstrating that our approaches outperform existing heuristics for size-constrained clustering. The running-time complexity of our proposal is assessed experimentally by means of a power-law regression analysis.

The Web is overcrowded with news articles, an overwhelming information source both with its amount and diversity. Document clustering is a powerful technique that has been widely used for organizing data into smaller and manageable information kernels. Several approaches have been proposed which, however, suffer from problems like synonymy, ambiguity and lack of a descriptive content marking of the generated clusters. In this work, we are investigating the application of a great spectrum of clustering algorithms, as well as similarity measures, to news articles that originate from the Web. Also, we are proposing the enhancement of standard k-means algorithm using the external knowledge from WordNet hypernyms in a twofold manner: enriching the ''bag of words'' used prior to the clustering process and assisting the label generation procedure following it. Furthermore, we are examining the effect that text preprocessing has on clustering. Operating on a corpus of news articles derived from major news portals, our comparison of the existing clustering methodologies revealed that k-means, gives better aggregate results when it comes to efficiency. This is amplified when the algorithm is accompanied with preliminary steps for data cleaning and normalizing, despite its simple nature. Moreover, the proposed WordNet-enabled W-k means clustering algorithm significantly improves standard k-means generating also useful and high quality cluster tags by using the presented cluster labeling process.

Data clustering is an important and frequently used unsupervised learning method. Recent research has demonstrated that incorporating instance-level background information to traditional clustering algorithms can increase the clustering performance. In this paper, we extend traditional clustering by introducing additional prior knowledge such as the size of each cluster. We propose a heuristic algorithm to transform size constrained clustering problems into integer linear programming problems. Experiments on both synthetic and UCI datasets demonstrate that our proposed approach can utilize cluster size constraints and lead to the improvement of clustering accuracy.

Identification of meaningful clusters from categorical data is one key problem in data mining. Recently, Average Normalized Mutual Information (ANMI) has been used to define categorical data clustering as an optimization problem. To find globally optimal or near-optimal partition determined by ANMI, a genetic clustering algorithm (G-ANMI) is proposed in this paper. Experimental results show that G-ANMI is superior or comparable to existing algorithms for clustering categorical data in terms of clustering accuracy.

Traditional clustering models based on distance similarity are not always effective in capturing correlation among data objects, while pattern-based clustering can do well in identifying correlation hidden among data objects. However, the state-of-the-art pattern-based clustering methods are inefficient and provide no metric to measure the clustering quality. This paper presents a new pattern-based subspace clustering method, which can tackle the problems mentioned above. Observing the analogy between mining frequent itemsets and discovering subspace clusters, we apply pattern tree - a structure used in frequent itemsets mining to determining the target subspaces by scanning the database once, which can be done efficiently in large datasets. Furthermore, we introduce a general clustering quality evaluation model to guide the identifying of meaningful clusters. The proposed new method enables the users to set flexibly proper quality-control parameters to meet different needs. Experimental results on synthetic and real datasets show that our method outperforms the existing methods in both efficiency and effectiveness.

Data pruning works with identifying noisy instances of a data set and removing them from the data set in order to improve the generalization of a learning algorithm. It has been well studied in supervised classification where the identification and removal of noisy instances are guided by available labels of instances. However, to the best knowledge of the authors', very few work has been done on data pruning for unsupervised clustering. This paper deals with the problem of data pruning for unsupervised clustering under the condition that labels of instances are unknown beforehand. We claim that unsupervised data pruning can benefit for the clustering of the data with noise. We propose a feasible approach, termed as unsupervised Data Pruning using Ensembles of multiple Clusterers (DPEC), to identify noisy instances of a data set. DPEC checks all instances of a data set and identifies noisy instances by using ensembles of multiple clustering results provided by different clusterers on the same data set. We test the performance of DPEC on several real data sets with artificial noise. Experimental results demonstrate that DPEC is often able to improve the accuracy and robustness of the clustering algorithm.

The paper presents a methodology for improving the organization of knowledge bases and demonstrates its application for generating the content of explanations. The DyKOr (Dynamic Knowledge Organization) method combines information that is usually available through execution traces with existing domain knowledge using techniques from machine learning including knowledge compilation, explanation-based learning, and conceptual clustering. These techniques allow the separation of the knowledge needed to solve a problem from that which is not required, and the identification of information that is related to the problem but is not explicitly stated. Thus, the analysis performed through the methodology can considerably improve the quality and content of explanations. The paper describes the implementation of the methodology and how it can be integrated into typical rule-based expert systems. Illustrations of how the method can be used to produce the content for explanations are presented in the context of typical consultation and problem solving expert systems. A discussion of how the information produced by the method can be used to prepare explanations for users with different levels of expertise is also presented.

Clustering techniques have been used for data abstraction. Dara abstraction has many applications in the contect of data-bases. Conceptual models are used to bridge the gap between the user's view of a database and the physical view of the database. Semantic models evolved to overcome the limitations of classical data models such as network and relational models. The paper uses a knowledge-based clustering algorithm to extend the abstractions, such as classification and association, which are employed in the semantic modeling of databases. The complexity of the proposed clustering algorithm is analysed. The extended semantic model can be used to design databases in which useful and interesting queries can be answered. The efficacy of the proposed knowledge-based clustering approach is examined in the context of a library database.