Search

We introduce MGP-VAE (Multi-disentangled-features Gaussian Processes Variational AutoEncoder), a variational autoencoder which uses Gaussian processes (GP) to model the latent space for the unsupervised learning of disentangled representations in video sequences. We improve upon previous work by establishing a framework by which multiple features, static or dynamic, can be disentangled. Specifically we use fractional Brownian motions (fBM) and Brownian bridges (BB) to enforce an inter-frame correlation structure in each independent channel, and show that varying this structure enables one to capture different factors of variation in the data. We demonstrate the quality of our representations with experiments on three publicly available datasets, and also quantify the improvement using a video prediction task. Moreover, we introduce a novel geodesic loss function which takes into account the curvature of the data manifold to improve learning. Our experiments show that the combination of the improved representations with the novel loss function enable MGP-VAE to outperform the baselines in video prediction.

Given a solution $Y$ to a rough differential equation (RDE), a recent result [8] extends the classical It\"{o}-Stratonovich formula and provides a closed-form expression for $\int Y \circ \mathrm{d} \mathbf{X} - \int Y \, \mathrm{d} X$, i.e. the difference between the rough and Skorohod integrals of $Y$ with respect to $X$, where $X$ is a Gaussian process with finite $p$-variation less than 3. In this paper, we extend this result to Gaussian processes with finite $p$-variation such that $3 \leq p < 4$. The constraint this time is that we restrict ourselves to Volterra Gaussian processes with kernels satisfying a natural condition, which however still allows the result to encompass many standard examples, including fractional Brownian motion with $H > \frac{1}{4}$. Analogously to [8], we first show that the Riemann-sum approximants of the Skorohod integral converge in $L^2(\Omega)$ by adopting a suitable characterization of the Cameron-Martin norm, before appending the approximants with higher-level compensation terms without altering the limit. Lastly, the formula is obtained after a re-balancing of terms, and we also show how to recover the standard It\"{o} formulas in the case where the vector fields of the RDE governing $Y$ are commutative.