Search

In large domains, Monte-Carlo tree search (MCTS) is required to estimate the values of the states as efficiently and accurately as possible. However, the standard update rule in backpropagation assumes a stationary distribution for the returns, and particularly in min-max trees, convergence to the true value can be slow because of averaging. We present two methods, Softmax MCTS and Monotone MCTS, which generalize previous attempts to improve upon the backpropagation strategy. We demonstrate that both methods reduce to finding optimal monotone functions, which we do so by performing Bayesian optimization with a Gaussian process (GP) prior. We conduct experiments on computer Go, where the returns are given by a deep value neural network, and show that our proposed framework outperforms previous methods.

We introduce MGP-VAE (Multi-disentangled-features Gaussian Processes Variational AutoEncoder), a variational autoencoder which uses Gaussian processes (GP) to model the latent space for the unsupervised learning of disentangled representations in video sequences. We improve upon previous work by establishing a framework by which multiple features, static or dynamic, can be disentangled. Specifically we use fractional Brownian motions (fBM) and Brownian bridges (BB) to enforce an inter-frame correlation structure in each independent channel, and show that varying this structure enables one to capture different factors of variation in the data. We demonstrate the quality of our representations with experiments on three publicly available datasets, and also quantify the improvement using a video prediction task. Moreover, we introduce a novel geodesic loss function which takes into account the curvature of the data manifold to improve learning. Our experiments show that the combination of the improved representations with the novel loss function enable MGP-VAE to outperform the baselines in video prediction.

Given a solution $Y$ to a rough differential equation (RDE), a recent result [8] extends the classical It\"{o}-Stratonovich formula and provides a closed-form expression for $\int Y \circ \mathrm{d} \mathbf{X} - \int Y \, \mathrm{d} X$, i.e. the difference between the rough and Skorohod integrals of $Y$ with respect to $X$, where $X$ is a Gaussian process with finite $p$-variation less than 3. In this paper, we extend this result to Gaussian processes with finite $p$-variation such that $3 \leq p < 4$. The constraint this time is that we restrict ourselves to Volterra Gaussian processes with kernels satisfying a natural condition, which however still allows the result to encompass many standard examples, including fractional Brownian motion with $H > \frac{1}{4}$. Analogously to [8], we first show that the Riemann-sum approximants of the Skorohod integral converge in $L^2(\Omega)$ by adopting a suitable characterization of the Cameron-Martin norm, before appending the approximants with higher-level compensation terms without altering the limit. Lastly, the formula is obtained after a re-balancing of terms, and we also show how to recover the standard It\"{o} formulas in the case where the vector fields of the RDE governing $Y$ are commutative.

Complementary regularity between the integrand and integrator is a well known condition for the integral $\int_0^T f(r) \, \mathrm{d} g(r)$ to exist in the Riemann-Stieltjes sense. This condition also applies to the multi-dimensional case, in particular the 2D integral $\int_{[0, T]^2} f(s,t) \, \mathrm{d} g(s,t)$. In the paper, we give a new condition for the existence of the integral under the assumption that the integrator $g$ is a Volterra covariance function. We introduce the notion of strong H\"{o}lder bi-continuity, and show that if the integrand possess this property, the assumption on complementary regularity can be relaxed for the Riemann-Stieltjes sums of the integral to converge.

Given a Gaussian process $X$, its canonical geometric rough path lift $\mathbf{X}$, and a solution $Y$ to the rough differential equation (RDE) $\mathrm{d}Y_{t} = V\left (Y_{t}\right ) \circ \mathrm{d} \mathbf{X}_t$, we present a closed-form correction formula for $\int Y \circ \mathrm{d} \mathbf{X} - \int Y \, \mathrm{d} X$, i.e. the difference between the rough and Skorohod integrals of $Y$ with respect to $X$. When $X$ is standard Brownian motion, we recover the classical Stratonovich-to-It{\^o} conversion formula, which we generalize to Gaussian rough paths with finite $p$-variation, $p < 3$, and satisfying an additional natural condition. This encompasses many familiar examples, including fractional Brownian motion with $H > \frac{1}{3}$. To prove the formula, we first show that the Riemann-sum approximants of the Skorohod integral converge in $L^2(\Omega)$ by using a novel characterization of the Cameron-Martin norm in terms of higher-dimensional Young-Stieltjes integrals. Next, we append the approximants of the Skorohod integral with a suitable compensation term without altering the limit, and the formula is finally obtained after a re-balancing of terms.