Approximate Manifold Sampling: Robust Bayesian Inference for Machine Learning

Efficient sampling from probability densities concentrated around a lower-dimensional submanifold is crucial in numerous applications arising in machine learning, statistics, and statistical physics. This task is particularly challenging due to the extreme anisotropy and high-dimensionality of the problem, and the correlation between the variables. We propose a novel family of bespoke MCMC algorithms designed to sample efficiently from these densities and show their computational superiority to general purpose and specialized samplers. Furthermore, we contribute to the development of integrator and Markov snippets, which are a particular class of general-purpose sequential algorithms for Bayesian inference and machine learning that can leverage the geometry of the space with integrators, is highly robust to the choice of the step size and the number of integration steps, and naturally lends itself to parallelisation. Building on these foundations, we present a sequential algorithm that is particularly well-suited to approximate manifold sampling.