Fuzzy Linear Discriminant Analysis-guided maximum entropy fuzzy clustering algorithm

Linear Discriminant Analysis (LDA) is a classical statistical approach for supervised feature extraction and dimensionality reduction, hard c-means (HCM) is a classical unsupervised learning algorithm for clustering. Based on the analysis of the relationship between LDA and HCM, Linear Discriminant Analysis-guided adaptive subspace hard c-means clustering algorithm (LDA-HCM) had been proposed. LDA-HCM combines LDA and HCM into a coherent framework and can adaptively reduce the dimension of data while performing data clustering simultaneously. Seeing that LDA-HCM is still a hard clustering algorithm, we consider the fuzzy extension version of LDA-HCM in this paper. To this end, firstly, we propose a new optimization criterion of Fuzzy Linear Discriminant Analysis (FLDA) by extending the value of membership function in classical LDA from binary 0 or 1 into closed interval [0, 1]. In the meantime, we present an efficient algorithm for the proposed FLDA. Secondly, we show the close relationship between FLDA and Maximum Entropy Fuzzy Clustering Algorithm (MEFCA): they both are maximizing fuzzy between-class scatter and minimizing within-class scatter simultaneously. Finally, based on the above analysis, combining FLDA and MEFCA into a joint framework, we propose fuzzy Linear Discriminant Analysis-guided maximum entropy fuzzy clustering algorithm (FLDA-MEFCA). LDA-MEFCA is a natural and effective fuzzy extension of LDA-HCM. Due to the introduction of soft decision strategy, FLDA-MEFCA can yield fuzzy partition of data set and is more flexible than LDA-HCM. We also give the convergence proof of FLDA-MEFCA. Extensive experiments on a collection of benchmark data sets are presented to show the effectiveness of the proposed algorithm.