Search

Laplacian mixture models have been used to deal with heavy-tailed distributions in data modeling problems. We consider an extension of Laplacian mixture models, which consists of ε -insensitive component distributions. An EM-type learning algorithm is derived for the maximum likelihood estimation of the proposed mixture model. The E-step is formulated in the usual way, while the M-step is formulated as the dual optimization problem instead of the primal optimization problem. Additionally, the convergence proof for ε =0 is accomplished. As an analogy to the k-means algorithm, we obtain what we call the ei-means algorithm in a certain limit of the learning algorithm. The derived algorithm is applied to approximate computation of rate-distortion functions associated with the ε -insensitive loss function. Then, it is demonstrated by synthetic data and real-world Spambase data that with appropriate selection of the ε value, the model is able to tolerate small percentage of noisy data. [ABSTRACT FROM AUTHOR]

Abstract: The local variational method is a technique to approximate an intractable posterior distribution in Bayesian learning. This article formulates a general framework for local variational approximation and shows that its objective function is decomposable into the sum of the Kullback information and the expected Bregman divergence from the approximating posterior distribution to the Bayesian posterior distribution. Based on a geometrical argument in the space of approximating posteriors, we propose an efficient method to evaluate an upper bound of the marginal likelihood. Moreover, we demonstrate that the variational Bayesian approach for the latent variable models can be viewed as a special case of this general framework. [Copyright &y& Elsevier]

Exponential principal component analysis (e-PCA) has been proposed to reduce the dimension of the parameters of probability distributions using Kuliback information as a distance between two distributions. It also provides a framework for dealing with various data types such as binary and integer for which the Gaussian assumption on the data distribution is inappropriate. In this paper, we introduce a latent variable model for the e-PCA. Assuming the discrete distribution on the latent variable leads to mixture models with constraint on their parameters. This provides a framework for clustering on the lower dimensional subspace of exponential family distributions. We derive a learning algorithm for those mixture models based on the variational Bayes (VB) method. Although intractable integration is required to implement the algorithm for a subspace, an approximation technique using Laplace's method allows us to carry out clustering on an arbitrary subspace. Combined with the estimation of the subspace, the resulting algorithm performs simultaneous dimensionality reduction and clustering. Numerical experiments on synthetic and real data demonstrate its effectiveness for extracting the structures of data as a visualization technique and its high generalization ability as a density estimation model. [ABSTRACT FROM AUTHOR]

The Variational Bayesian learning, proposed as an approximation of the Bayesian learning, has provided computational tractability and good generalization performance in many applications. However, little has been done to investigate its theoretical properties. In this paper, we discuss the Variational Bayesian learning of the mixture of exponential families and derive the asymptotic form of the stochastic complexities in a generalized setting of the prior distribution. We show that the stochastic complexities become smaller than those of regular statistical models, which implies that the advantage of the Bayesian learning still remains in the Variational Bayesian learning. Stochastic complexity, which is called the marginal likelihood or the free energy, not only becomes important in addressing the model selection problem but also enables us to discuss the accuracy of the Variational Bayesian approach as an approximation of the true Bayesian learning. The main result also shows the effects of the prior distribution under the generalized setting. [Copyright &y& Elsevier]