Selective smoothing of the generative topographic mapping

Generative topographic mapping is a nonlinear latent variable model introduced by Bishop et al. as a probabilistic reformulation of self-organizing maps. The complexity of this model is mostly determined by the number and form of basis functions generating the nonlinear mapping from latent space to data space, but it can be further controlled by adding a regularization term to increase the stiffness of the mapping and avoid data over-fitting. In this paper, we improve the map smoothing by introducing multiple regularization terms, one associated with each of the basis functions. A similar technique to that of automatic relevance determination, our selective map smoothing locally controls the stiffness of the mapping depending on length scales of the underlying manifold, while optimizing the effective number of active basis functions. Index Terms--Automatic relevance determination (ARD), Bayesian probability theory, generative topographic mapping (GTM), regularization, self-organizing map (SOM).