Unsupervised Rank-deficient Density Estimation via Multi-Class Independent Component Analysis

One of the most effective ways of modeling vector data for unsupervised pattern classification or coding, is to assume that the observations are the result of picking randomly out of a fixed set of different distributions. In this paper we propose to perform the unsupervised estimation of the mixture density underlying the data as the problem of separating multiclass sources. Assuming in each class independent components, standard linear independent component analysis (ICA) can be adopted in the recently extended mode which provides signal reconstruction for a multiclass mixture. Unfortunately, in practical problems the class densities necessary to match the experimental distributions must be degenerate or poorly conditioned. In this paper we approach the problem by assuming from the beginning sources which have either rank-deficient distributions or show very concentrated eigenvalues. The class membership of each point is based on a distance measure from the hyperplanes and on the likelihood on each hyperplane. The independent components are then searched within each subspace. We present results of the algorithm on synthetic distributions with various degrees of degeneracy. Our results are promising for feature extraction applications.