Search

Deep clustering has gained prominence due to its impressive capability to handle high-dimensional real-world data. However, in the absence of ground-truth labels, existing clustering methods struggle to discern false positives that resemble the target cluster and false negatives that visually differ but maintain semantic consistency. The unreliable projections caused by visual ambiguity disrupt representation learning, leading to sub-optimal clustering outcomes. To address this challenge, we propose a novel method called uncertainty-based learning for deep clustering (ULDC), which aims to discover more optimal cluster structures within data from an uncertainty perspective. Specifically, we utilize the Dirichlet distribution to quantify the uncertainty of feature projections in the latent space, providing a probabilistic framework for modeling uncertainty during the clustering process. We then develop uncertainty-based learning to mitigate the interference caused by false positives and negatives in the clustering tasks. Additionally, a semantic calibration module is introduced to achieve a global alignment of cross-instance semantics, facilitating the learning of clustering-favorite representations. Extensive experiments on five widely-used benchmarks demonstrate the effectiveness of ULDC. The source code is available from . [ABSTRACT FROM AUTHOR]

In recent years, there has been a growing interest in deep learning-based clustering. A recently introduced technique called DipDECK has shown effective performance on large and high-dimensional datasets. DipDECK utilises Hartigan's dip test, a statistical test, to merge small non-viable clusters. Notably, DipDECK was the first deep learning-based clustering technique to incorporate the dip test. However, the number of initial clusters of DipDECK is overestimated and the algorithm then randomly selects the initial seeds to produce the final clusters for a dataset. Therefore, in this paper, we presented a technique called UNSDipDECK , which is an improved version of DipDECK and does not require user input for datasets with an unknown number of clusters. UNSDipDECK produces high-quality initial seeds and the initial number of clusters through a deterministic process. UNSDipDECK uses the unique closest neighbourhood and unique neighbourhood set approaches to determine high-quality initial seeds for a dataset. In our study, we compared the performance of UNSDipDECK with fifteen baseline clustering techniques, including DipDECK, using NMI and ARI metrics. The experimental results indicate that UNSDipDECK outperforms the baseline techniques, including DipDECK. Additionally, we demonstrated that the initial seed selection process significantly contributes to UNSDipDECK 's ability to produce high-quality clusters. [ABSTRACT FROM AUTHOR]

Multi-view clustering aims to explore the feature information of objects from multiple perspectives to obtain accurate clustering results. However, existing research often fails to handle the information conflicts that arise during view fusion and does not fully utilize the complementary information between multiple views. To address these issues, this paper proposed a self-supervised multi-view clustering model guided by multi-angle semantic labels. The model first mapped the latent representations of each view to independent low-dimensional feature spaces, focusing on optimizing the consistency between views in one space to maintain the local structure of the feature space and the relative relationships between samples. At the same time, in another space, clustering information was directly extracted from the view level to capture richer and more diverse semantic features. Finally, pseudo-labels generated from multi-angle semantic features guided the clustering assignment at the object level, achieving collaborative optimization of the two representations. Extensive experimental results demonstrate that this approach can comprehensively explore both common and complementary information in multi-view data and exhibit good clustering performance. Moreover, compared to other methods, this approach has advantages in scenarios with a larger number of views. [ABSTRACT FROM AUTHOR]

BackgroundPhysiological motion, such as respiratory motion, has become a limiting factor in the spatial resolution of positron emission tomography (PET) imaging as the resolution of PET detectors continue to improve. Motion-induced misregistration between PET and CT images can also cause attenuation correction artifacts. Respiratory gating can be used to freeze the motion and to reduce motion induced artifacts.PurposeIn this study, we propose a robust data-driven approach using an unsupervised deep clustering network that employs an autoencoder (AE) to extract latent features for respiratory gating.MethodsWe first divide list-mode PET data into short-time frames. The short-time frame images are reconstructed without attenuation, scatter, or randoms correction to avoid attenuation mismatch artifacts and to reduce image reconstruction time. The deep AE is then trained using reconstructed short-time frame images to extract latent features for respiratory gating. No additional data are required for the AE training. K-means clustering is subsequently used to perform respiratory gating based on the latent features extracted by the deep AE. The effectiveness of our proposed Deep Clustering method was evaluated using physical phantom and real patient datasets. The performance was compared against phase gating based on an external signal (External) and image based principal component analysis (PCA) with K-means clustering (Image PCA).ResultsThe proposed method produced gated images with higher contrast and sharper myocardium boundaries than those obtained using the External gating method and Image PCA. Quantitatively, the gated images generated by the proposed Deep Clustering method showed larger center of mass (COM) displacement and higher lesion contrast than those obtained using the other two methods.ConclusionsThe effectiveness of our proposed method was validated using physical phantom and real patient data. The results showed our proposed framework could provide superior gating than the conventional External method and Image PCA.

Deep spectral clustering techniques are considered one of the most efficient clustering algorithms in data mining field. The similarity between instances and the disparity among classes are two critical factors in clustering fields. However, most current deep spectral clustering approaches do not sufficiently take them both into consideration. To tackle the above issue, we propose Semantic Spectral clustering with Contrastive learning and Neighbor mining (SSCN) framework, which performs instance-level pulling and cluster-level pushing cooperatively. Specifically, we obtain the semantic feature embedding using an unsupervised contrastive learning model. Next, we obtain the nearest neighbors partially and globally, and the neighbors along with data augmentation information enhance their effectiveness collaboratively on the instance level as well as the cluster level. The spectral constraint is applied by orthogonal layers to satisfy conventional spectral clustering. Extensive experiments demonstrate the superiority of our proposed frame of spectral clustering. [ABSTRACT FROM AUTHOR]

Deep learning has been recently used to extract the relevant features for representing input data also in the unsupervised setting. However, state-of-the-art techniques focus mostly on algorithmic efficiency and accuracy rather than mimicking the input manifold. On the contrary, competitive learning is a powerful tool for replicating the input distribution topology. It is cognitive/biologically inspired as it is founded on Hebbian learning, a neuropsychological theory claiming that neurons can increase their specialization by competing for the right to respond to/represent a subset of the input data. This paper introduces a novel perspective by combining these two techniques: unsupervised gradient-based and competitive learning. The theory is based on the intuition that neural networks can learn topological structures by working directly on the transpose of the input matrix. At this purpose, the vanilla competitive layer and its dual are presented. The former is representative of a standard competitive layer for deep clustering, while the latter is trained on the transposed matrix. The equivalence of the layers is extensively proven both theoretically and experimentally. The dual competitive layer has better properties. Unlike the vanilla layer, it directly outputs the prototypes of the data inputs, while still allowing learning by backpropagation. More importantly, this paper proves theoretically that the dual layer is better suited for handling high-dimensional data (e.g., for biological applications), because the estimation of the weights is driven by a constraining subspace which does not depend on the input dimensionality, but only on the dataset cardinality. This paper has introduced a novel approach for unsupervised gradient-based competitive learning. This approach is very promising both in the case of small datasets of high-dimensional data and for better exploiting the advantages of a deep architecture: the dual layer perfectly integrates with the deep layers. A theoretical justification is also given by using the analysis of the gradient flow for both vanilla and dual layers. [ABSTRACT FROM AUTHOR]

Recent studies show that the use of multimodality can effectively enhance the understanding of social media content. The relations between texts and images become an important basis for developing multimodal data and models. Some studies have attempted to label image-text relation (ITR) and build supervised learning models. However, manually labeling ITR is a challenging task and incurs many controversial labels because of disagreements among the annotators. In this paper, we present a novel unsupervised multimodal method called ITR pseudo-labeling (ITRp) that learns multimodal representations for various ITR types using different finetuning strategies. Our ITRp method generates pseudo-labels by clustering and uses them as supervision to train the classifier and encoders. We evaluate the ITRp method on the ITR dataset and the effects of the samples with incorrect labels on both the supervised and unsupervised models. The code and data are available on the website https://github.com/SuYindu/ITRp. [ABSTRACT FROM AUTHOR]

Network traffic classification (NTC) has attracted great attention in many applications such as secure communications, intrusion detection systems. The existing NTC methods based on supervised learning rely on sufficient labeled datasets in the training phase, but for most traffic datasets, it is difficult to obtain label information in practical applications. Although unsupervised learning does not rely on labels, its classification accuracy is not high, and the number of data classes is difficult to determine. This paper proposes an unsupervised NTC method based on adversarial training and deep clustering with improved network traffic classification (NTC) and lower computational complexity in comparison with the traditional clustering algorithms. Here, the training process does not require data labels, which greatly reduce the computational complexity of the network traffic classification through pretraining. In the pretraining stage, an autoencoder (AE) is used to reduce the dimension of features and reduce the complexity of the initial high-dimensional network traffic data features. Moreover, we employ the adversarial training model and a deep clustering structure to further optimize the extracted features. The experimental results show that our proposed method has robust performance, with a multiclassification accuracy of 92.2%, which is suitable for classification with a large number of unlabeled data in actual application scenarios. This paper only focuses on breakthroughs in the algorithm stage, and future work can be focused on the deployment and adaptation in practical environments. [ABSTRACT FROM AUTHOR]

An effective way to improve the performance of deep neural networks in most computer vision tasks is to improve the quantity of labeled data and the quality of labels. However, in the analysis and processing of medical images, high-quality annotation depends on the experience and professional knowledge of experts, which makes it very difficult to obtain a large number of high-quality annotations. Therefore, we propose a new semi-supervised framework for medical image classification. It combines semi-supervised classification with unsupervised deep clustering. Spreading label information to unlabeled data by alternately running two tasks helps the model to extract semantic information from unlabeled data, and prevents the model from overfitting to a small amount of labeled data. Compared with current methods, our framework enhances the robustness of the model and reduces the influence of outliers. We conducted a comparative experiment on the public benchmark medical image dataset to verify our method. On the ISIC 2018 Dataset, our method surpasses other methods by more than 0.85% on AUC and 1.08% on Sensitivity. On the ICIAR BACH 2018 dataset, our method achieved 94.12% AUC, 77.92% F1-score, 77.69% Recall, and 78.16% Precision. The error rate is at least 1.76% lower than that of other methods. The result shows the effectiveness of our method in medical image classification. [ABSTRACT FROM AUTHOR]

Extended vision techniques are ubiquitous in physics. However, the data cubes steaming from such analysis often pose a challenge in their interpretation, due to the intrinsic difficulty in discerning the relevant information from the spectra composing the data cube. Furthermore, the huge dimensionality of data cube spectra poses a complex task in its statistical interpretation; nevertheless, this complexity contains a massive amount of statistical information that can be exploited in an unsupervised manner to outline some essential properties of the case study at hand, e.g. it is possible to obtain an image segmentation via (deep) clustering of data-cube’s spectra, performed in a suitably defined low-dimensional embedding space. To tackle this topic, we explore the possibility of applying unsupervised clustering methods in encoded space, i.e. perform deep clustering on the spectral properties of datacube pixels. A statistical dimensional reduction is performed by an ad hoc trained (variational) AutoEncoder, in charge of mapping spectra into lower dimensional metric spaces, while the clustering process is performed by a (learnable) iterative K-means clustering algorithm. We apply this technique to two different use cases, of different physical origins: a set of macro mapping x-ray fluorescence (MA-XRF) synthetic data on pictorial artworks, and a dataset of simulated astrophysical observations.

Abstract Background Chronic cough affects approximately 10% of adults. The lack of ICD codes for chronic cough makes it challenging to apply supervised learning methods to predict the characteristics of chronic cough patients, thereby requiring the identification of chronic cough patients by other mechanisms. We developed a deep clustering algorithm with auto-encoder embedding (DCAE) to identify clusters of chronic cough patients based on data from a large cohort of 264,146 patients from the Electronic Medical Records (EMR) system. We constructed features using the diagnosis within the EMR, then built a clustering-oriented loss function directly on embedded features of the deep autoencoder to jointly perform feature refinement and cluster assignment. Lastly, we performed statistical analysis on the identified clusters to characterize the chronic cough patients compared to the non-chronic cough patients. Results The experimental results show that the DCAE model generated three chronic cough clusters and one non-chronic cough patient cluster. We found various diagnoses, medications, and lab tests highly associated with chronic cough patients by comparing the chronic cough cluster with the non-chronic cough cluster. Comparison of chronic cough clusters demonstrated that certain combinations of medications and diagnoses characterize some chronic cough clusters. Conclusions To the best of our knowledge, this study is the first to test the potential of unsupervised deep learning methods for chronic cough investigation, which also shows a great advantage over existing algorithms for patient data clustering.

Network traffic classification (NTC) has attracted great attention in many applications such as secure communications, intrusion detection systems. The existing NTC methods based on supervised learning rely on sufficient labeled datasets in the training phase, but for most traffic datasets, it is difficult to obtain label information in practical applications. Although unsupervised learning does not rely on labels, its classification accuracy is not high, and the number of data classes is difficult to determine. This paper proposes an unsupervised NTC method based on adversarial training and deep clustering with improved network traffic classification (NTC) and lower computational complexity in comparison with the traditional clustering algorithms. Here, the training process does not require data labels, which greatly reduce the computational complexity of the network traffic classification through pretraining. In the pretraining stage, an autoencoder (AE) is used to reduce the dimension of features and reduce the complexity of the initial high-dimensional network traffic data features. Moreover, we employ the adversarial training model and a deep clustering structure to further optimize the extracted features. The experimental results show that our proposed method has robust performance, with a multiclassification accuracy of 92.2%, which is suitable for classification with a large number of unlabeled data in actual application scenarios. This paper only focuses on breakthroughs in the algorithm stage, and future work can be focused on the deployment and adaptation in practical environments.

This paper proposes to perform online clustering by conducting twin contrastive learning (TCL) at the instance and cluster level. Specifically, we find that when the data is projected into a feature space with a dimensionality of the target cluster number, the rows and columns of its feature matrix correspond to the instance and cluster representation, respectively. Based on the observation, for a given dataset, the proposed TCL first constructs positive and negative pairs through data augmentations. Thereafter, in the row and column space of the feature matrix, instance- and cluster-level contrastive learning are respectively conducted by pulling together positive pairs while pushing apart the negatives. To alleviate the influence of intrinsic false-negative pairs and rectify cluster assignments, we adopt a confidence-based criterion to select pseudo-labels for boosting both the instance- and cluster-level contrastive learning. As a result, the clustering performance is further improved. Besides the elegant idea of twin contrastive learning, another advantage of TCL is that it could independently predict the cluster assignment for each instance, thus effortlessly fitting online scenarios. Extensive experiments on six widely-used image and text benchmarks demonstrate the effectiveness of TCL. The code is released on https://pengxi.me. [ABSTRACT FROM AUTHOR]

Clustering data effectively remains a significant challenge in machine learning, particularly when the optimal number of clusters is unknown. Traditional deep clustering methods often struggle with balancing local and global search, leading to premature convergence and inefficiency. To address these issues, we introduce ADSC-DPE-RT (Automatic Deep Sparse Clustering with a Dynamic Population-based Evolutionary Algorithm using Reinforcement Learning and Transfer Learning), a novel deep clustering approach. ADSC-DPE-RT builds on Multi-Trial Vector-based Differential Evolution (MTDE), an algorithm that integrates sparse auto-encoding and manifold learning to enable automatic clustering without prior knowledge of cluster count. However, MTDE's fixed population size can lead to either prolonged computation or premature convergence. Our approach introduces a dynamic population generation technique guided by Reinforcement Learning (RL) and Markov Decision Process (MDP) principles. This allows for flexible adjustment of population size, preventing premature convergence and reducing computation time. Additionally, we incorporate Generative Adversarial Networks (GANs) to facilitate dynamic knowledge transfer between MTDE strategies, enhancing diversity and accelerating convergence towards the global optimum. This is the first work to address the dynamic population issue in deep clustering through RL, combined with Transfer Learning to optimize evolutionary algorithms. Our results demonstrate significant improvements in clustering performance, positioning ADSC-DPE-RT as a competitive alternative to state-of-the-art deep clustering methods. • Introduces a novel clustering method integrating Auto-Encoders, Evolutionary Algorithms, and Reinforcement Learning. • Adjusts population size and strategy dynamically with Reinforcement Learning to enhance exploration. • Uses Generative Adversarial Networks to share elite populations among strategies, boosting diversity and solution quality. • Addresses feature extraction inefficiencies and local optima issues through dynamic, adaptive techniques. • Outperforms existing methods and works effectively without needing prior knowledge of the number of clusters. [ABSTRACT FROM AUTHOR]

Deep image clustering networks have the capability to categorize unlabeled images, thereby effectively utilizing them. This paper synthesizes recent researches about deep image clustering network and summarizes them within a general framework. Image Preprocessing part transforms collected images into a dataset accepted by the network, then Image Embedding part embeds images from the dataset into vectors which represents image features. After Feature Processing part further reduces the dimensionality or enhances these features, the following Feature Clustering part divides the images into several categories. Cluster Result Processing part treats the clustering outcomes to implement weighted, multi-view, subspace, multimodal, or self-supervised methods. Downstream Applications part employ the clustering results to address various real-world problems. The performance of common baseline clustering methods and several feature extraction architectures are also compared and analyzed. The results indicate that within deep image clustering networks, the choice of clustering algorithm has a relatively minor impact on clustering performance, whereas the selection of the feature extraction network architecture is decisive for clustering metrics. The choice of architecture should be customized based on the characteristics of the dataset. Finally, we provide suggestions for potential research directions in deep image clustering networks. • Analyzes deep image clustering theory, its performance benefits, and limitations. • Summarizes recent deep image clustering improvements with experimental data analysis. • Introduces deep image clustering applications and future improvement prospects. [ABSTRACT FROM AUTHOR]

Outlier detection is to separate anomalous data from inliers in the dataset. Recently, the most deep learning methods of outlier detection leverage an auxiliary reconstruction task by assuming that outliers are more difficult to recover than normal samples (inliers). However, it is not always true in deep auto-encoder (AE) based models. The auto-encoder based detectors may recover certain outliers even if outliers are not in the training data, because they do not constrain the feature learning. Instead, we think outlier detection can be done in the feature space by measuring the distance between outliers’ features and the consistency feature of inliers. To achieve this, we propose an unsupervised outlier detection method using a memory module and a contrastive learning module (MCOD). The memory module constrains the consistency of features, which merely represent the normal data. The contrastive learning module learns more discriminative features, which boosts the distinction between outliers and inliers. Extensive experiments on four benchmark datasets show that our proposed MCOD performs well and outperforms eleven state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Background: Chronic cough affects approximately 10% of adults. The lack of ICD codes for chronic cough makes it challenging to apply supervised learning methods to predict the characteristics of chronic cough patients, thereby requiring the identification of chronic cough patients by other mechanisms. We developed a deep clustering algorithm with auto-encoder embedding (DCAE) to identify clusters of chronic cough patients based on data from a large cohort of 264,146 patients from the Electronic Medical Records (EMR) system. We constructed features using the diagnosis within the EMR, then built a clustering-oriented loss function directly on embedded features of the deep autoencoder to jointly perform feature refinement and cluster assignment. Lastly, we performed statistical analysis on the identified clusters to characterize the chronic cough patients compared to the non-chronic cough patients. Results: The experimental results show that the DCAE model generated three chronic cough clusters and one non-chronic cough patient cluster. We found various diagnoses, medications, and lab tests highly associated with chronic cough patients by comparing the chronic cough cluster with the non-chronic cough cluster. Comparison of chronic cough clusters demonstrated that certain combinations of medications and diagnoses characterize some chronic cough clusters. Conclusions: To the best of our knowledge, this study is the first to test the potential of unsupervised deep learning methods for chronic cough investigation, which also shows a great advantage over existing algorithms for patient data clustering. [ABSTRACT FROM AUTHOR]

An effective way to improve the performance of deep neural networks in most computer vision tasks is to improve the quantity of labeled data and the quality of labels. However, in the analysis and processing of medical images, high-quality annotation depends on the experience and professional knowledge of experts, which makes it very difficult to obtain a large number of high-quality annotations. Therefore, we propose a new semi-supervised framework for medical image classification. It combines semi-supervised classification with unsupervised deep clustering. Spreading label information to unlabeled data by alternately running two tasks helps the model to extract semantic information from unlabeled data, and prevents the model from overfitting to a small amount of labeled data. Compared with current methods, our framework enhances the robustness of the model and reduces the influence of outliers. We conducted a comparative experiment on the public benchmark medical image dataset to verify our method. On the ISIC 2018 Dataset, our method surpasses other methods by more than 0.85% on AUC and 1.08% on Sensitivity. On the ICIAR BACH 2018 dataset, our method achieved 94.12% AUC, 77.92% F1-score, 77.69% Recall, and 78.16% Precision. The error rate is at least 1.76% lower than that of other methods. The result shows the effectiveness of our method in medical image classification.

• The feature representation consists of two parts: data and its local-topology information. • We propose a replacement strategy to find local-topology representation of data. • Data augmentation is applied to improve the generalization performance of the model. • A two-stage image deep clustering algorithm is presented based on local-topology embedding called ITEC. Reasonable feature representation plays an important role in improving the performance of clustering algorithms. However, recent deep clustering studies only focusing on feature representation at the pixel level leads to feature representation with low discrimination. Our key insight is that considering local-topology information between images would help to get a highly discriminative representation, and therefore we design a replacement strategy to find local-topology representation of data, and propose a two-stage image deep clustering algorithm based on local-topology embedding called ITEC. Specifically, we take advantage of data augmentation technique to improve the generalization performance of the learning models; then local-topology representation of data is embedded into the representation of data itself, so as to better complete tasks of image clustering. Extensive experiments demonstrate that local-topology information effectively promotes the performance of deep clustering significantly. [ABSTRACT FROM AUTHOR]

In recent years, with the great success of deep learning and especially deep unsupervised learning, many deep architectural clustering methods, collectively known as deep clustering, have emerged. Deep clustering shows the potential to outperform traditional methods, especially in handling complex high-dimensional data, taking full advantage of deep learning. To achieve a comprehensive overview of the field of deep clustering, this review systematically explores deep clustering methods and their various applications. First, the basic network architecture of deep clustering is described in detail, including the common network frameworks, and loss functions. Subsequently, deep clustering is divided into several categories based on the network architecture, and benchmark datasets and evaluation metrics in the field are introduced. Next, the real-world applications of deep clustering are explored in depth, providing successful cases in the fields of bioinformatics, medicine, anomaly detection, and image processing, highlighting the broad applicability of deep clustering in solving real-world challenges. Finally, the paper summarizes its contributions and explores potential directions for future research in deep clustering. • Analyze the theory of deep clustering and its performance advantages and disadvantages. • Summarizes the improvement of deep clustering in recent years, analyzes the improvement effect, and shows it with experimental data. • Finally, the related application research of deep clustering in different fields is introduced. At the same time, we summarize and prospect the improvement and development of deep clustering. [ABSTRACT FROM AUTHOR]

Deep clustering methods have found successful applications in single-sensor data fault diagnosis. However, most of these methods employ separate optimization strategies that overlook the interaction between feature learning and clustering. Moreover, conventional deep learning methods for fault diagnosis often disregard the consistent and complementary information inherent in the multisensor data, resulting in unsatisfactory multisensor fault diagnosis performance. In this study, we introduce a novel End-to-end Deep Clustering Method with Consistency and Complementarity Attention Mechanism, termed EDCM-CCAM, tailored for multisensor fault diagnosis. Firstly, multiple deep autoencoder networks are utilized to concurrently extract the deep representation features from various sensor inputs. Secondly, we introduce a Consistency and Complementarity Attention Mechanism (CCAM) to facilitate multisensor feature fusion, accompanied by the design of two distinct loss functions to fully exploit the consistent and complementary information within multisensor data. Finally, fault pattern recognition in multisensor data is accomplished through Kullback-Leibler (KL) divergence-based clustering, while a joint optimization strategy is employed to simultaneously optimize all components of the EDCM-CCAM. The efficacy of the proposed method is validated on a gearbox dataset, demonstrating superior performance in multisensor fault diagnosis compared to alternative methods. • The deep clustering method with joint optimization strategy is presented. • The new attention CCAM and two losses are designed for feature fusion. • The proposed method outperforms other comparative fault diagnosis methods. [ABSTRACT FROM AUTHOR]

Deep subspace clustering based on autoencoder and self-expression layer has become popular for clustering method. However, these models only focus on feature extraction by minimizing reconstruction error rather than specific clustering task, leading to unsatisfactory performance. To overcome the above shortcoming, we propose a deep subspace clustering framework which jointly extracts features via the embedding neural network and performs subspace learning. The model contains three modules, an autoencoder, a self-expression layer and a supervised competitive feature learning module. The proposed model is highly capable of capturing characteristic features to guide the clustering task by using relative entropy to minimize probabilistic cluster assignments and the target variables. The three modules are consolidated to be jointly trained and optimized competitively. Experimental results on five benchmark datasets demonstrate the effectiveness of the proposed deep subspace clustering by comparing with eleven baseline methods. [ABSTRACT FROM AUTHOR]

• A novel deep evolutionary clustering (ADSMTDE) to overcome clustering drawbacks. • Improving the auto-encoder by applying sparsity constraint and manifold learning. • To enhance clustering, evolutionary algorithm is adopted to optimize solutions. • Employing an incremental clustering technique to perform clustering dynamically. • ADSMTDE is competitive and superior to over the latest deep clustering methods. Most deep clustering methods despite utilizing complex networks to learn better from data, use a shallow clustering method. These methods have difficulty in finding good clusters due to the lack of ability to handle between local search and global search to prevent premature convergence. In other words, they do not consider different aspects of the search and it causes them to get stuck in the local optimum. In addition, the majority of existing deep clustering approaches perform clustering with the knowledge of the number of clusters, which is not practical in most real scenarios where such information is not available. To address these problems, this paper presents a novel automatic deep sparse clustering approach based on an evolutionary algorithm called Multi-Trial Vector-based Differential Evolution (MTDE). Sparse auto-encoder is first applied to extract embedded features. Manifold learning is then adopted to obtain representation and extract the spatial structure of features. Afterward, MTDE clustering is performed without prior information on the number of clusters to find the optimal clustering solution. The proposed approach was evaluated on various datasets, including images and time-series. The results demonstrate that the proposed method improved MTDE by 18.94% on average and compared to the most recent deep clustering algorithms, is consistently among the top three in the majority of datasets. [ABSTRACT FROM AUTHOR]

• We exploit the Siamese Network to find a clustering-friendly embedding space to mine highly-reliable pseudo-supervised information for the application of VAT and Conditional-GAN to synthesize cluster-specific samples in the setting of unsupervised learning. • We proposed adopting VAT to synthesize samples with different levels of perturbations that can enhance the robustness of Feature Extractor to noise and improve the lower-dimensional latent coding space discovered by the Feature Extractor. • We conducted experiments to verify that the latent space discovered by the Feature Extractor can facilitate the Siamese Network to find a clustering-friendly embedding space and extract pseudo-supervised information for VAT and Conditional-GAN. • The training of our EDCN involves the adversarial gaming between three players, which not only boosts performance improvement of the clustering but also preserves the cluster-specific information from the Siamese Network in synthesizing samples. In this paper, we propose an enhanced deep clustering network (EDCN), which is composed of a Feature Extractor, a Conditional Generator, a Discriminator and a Siamese Network. Specifically, we will utilize two kinds of generated data based on adversarial training, as well as the original data, to train the Feature Extractor for learning effective latent representations. In addition, we adopt the Siamese network to find an embedding space, where a better affinity similarity matrix is obtained as the key to success of spectral clustering in providing reliable pseudo-labels. Particularly, the obtained pseudo-labels will be used to generate realistic data by the Generator. Finally, the discriminator is used to model the real joint distribution of data and corresponding latent representations for Feature Extractor enhancement. To evaluate our proposed EDCN, we conduct extensive experiments on multiple data sets including MNIST, USPS, FRGC, CIFAR-10, STL-10, and Fashion-MNIST by comparing our method with a number of state-of-the-art deep clustering methods, and experimental results demonstrate its effectiveness and superiority. [ABSTRACT FROM AUTHOR]

Unsupervised clustering, which is extensively employed in deep learning and computer vision as a fundamental technique, has attracted much attention in recent years. Deep embedding clustering often uses auto-encoders to learn representations for clustering. However, auto-encoders tend to corrupt the learning representations when simultaneously learning embedded representations and performing clustering. In this paper, we propose a Deep Clustering via Variational Auto-Encoder (DC-VAE) of mutual information maximization. First, we formulate the deep clustering problem as learning soft cluster assignments within the framework of variational auto-encoder. Second, we impose mutual information maximization on the observed data and the representations to prevent soft cluster assignments from distorting learning representations. Third, we derive a new generalization evidence lower bound objects related to several previous models and introduce parameters to balance learning informative representations and clustering. It is shown that the proposed model can significantly boost the performance of clustering by learning effective and reliable representations for downstream machine learning tasks. Through experimental results on several datasets, we demonstrate that the proposed model is competitive with existing state-of-the-arts on multiple performance metrics. [ABSTRACT FROM AUTHOR]