Search

This thesis is organised into two main parts, which are both preceded by a joint introduction and a brief chapter on the signature analysis of time-ordered data. In the first part, we study the classical problem of recovering a multidimensional source signal from observations of nonlinear mixtures of this signal. We show that this recovery is possible (up to a permutation and monotone scaling of the source's original component signals) if the mixture is due to a sufficiently differentiable and invertible but otherwise arbitrarily nonlinear function and the component signals of the source are statistically independent with 'nondegenerate' second-order statistics. The latter assumption requires the source signal to meet one of three regularity conditions which essentially ensure that the source is sufficiently far away from the non-recoverable extremes of being deterministic or constant in time. These assumptions, which cover many popular time series models and stochastic processes, allow us to reformulate the initial problem of nonlinear blind source separation as a simple-to-state problem of optimisation-based function approximation. We propose to solve this approximation problem by minimizing a novel type of objective function that efficiently quantifies the mutual statistical dependence between multiple stochastic processes via cumulant-like statistics. This yields a scalable and direct new method for nonlinear Independent Component Analysis with widely applicable theoretical guarantees and for which our experiments indicate good performance. In the second part, we revisit the problem of blind source separation from the perspective of statistical robustness. Blind source separation (BSS) aims to recover an unobserved signal S from its mixture X = f (S) under the condition that the effecting transformation f is invertible but unknown. This being a basic problem with numerous practical applications, a fundamental issue is to understand how the solutions to this problem behave when their supporting statistical prior assumptions are violated. In the classical context of linear mixtures, we present a general framework to analyse such violations and quantify the effect they have on the blind recovery of S from X. Modelling S as a multidimensional stochastic process, we introduce an informative topology on the space of possible causes underlying a mixture X and show that the behaviour of a generic BSS-solution in response to general deviations from its defining structural assumptions can be profitably analysed in the form of explicit continuity guarantees with respect to this topology. This enables a flexible and convenient quantification of general model uncertainty scenarios and amounts to the first comprehensive robustness framework for BSS. Our theory is entirely constructive, and we demonstrate its utility with a number of statistical applications.

The thesis goal is to explore the relations between Formal Verification techniques in Computer Science and Stochastic Analysis in Mathematics. They both deal with probabilistic dynamical systems, implying that the number of connections is significant. The Formal Verification studies dynamical systems' properties, which are specified in terms of logic statements. This discipline has been successfully applied to multiple industry problems, such as ``verification of'' Software, Microprocessors or Neural Networks. The models studied have a countable states space and this can be considered the main obstacle to extend Formal Verification techniques for the computation of likelihoods in the case of Stochastic Differential Equations. However, the most studied properties in Formal Verification have a clear characterization in Mathematics. In this thesis, we focus on the safety property ``always'', which can be characterized as an exit time in the theory of Stochastic Analysis. The main difficulties are twofold. Firstly, if we are particularly interested in discretizing the states space - going from a continuous, uncountable space to a finite countable one - it is very likely to have an explosion of the cardinality of the states space. This leads to problems both in time and memory complexity. We find a new characterization of the barycenter of a discrete probability measure, which allows us to introduce a new algorithm for the recombination problem. Then, we use the algorithm to build recombining trees in order to approximate weakly stochastic processes, solving the exponential explosion of the cardinality of the states space. Secondly, one has to consider the approximation error: if the probability to exit from a safety region has not an analytical closed-form, a numerical scheme must be considered and therefore it is important to analyse the error. We introduce a grid-free approach for the computation of Probabilistic Safety Regions using Malliavin Calculus.

In this thesis, we consider problems related to the numerical simulation of stochastic differential equations (SDEs). In particular, we are interested in methods that use both increments and areas of the Brownian path. For example, time integrals of Brownian motion carry useful information whilst being normally distributed and thus straightforward to generate. We will make precise the "path information" contained by such integrals and present a polynomial approximation theorem for Brownian motion. Since the Gaussian coefficients given by this expansion are independent, we can express Brownian motion as a polynomial with additional noise. We shall then use this simple observation to develop high order methods. The majority of these methods approximate SDEs through the use of ODEs, which can be accurately discretized using state-of-the-art solvers. To numerically demonstrate these approaches, we will simulate four SDEs: Inhomogeneous Geometric Brownian Motion, Cox-Ingersoll-Ross model, underdamped Langevin equation and the Lévy area of Brownian motion. The code for these examples can be found at github.com/james-m-foster. We also study problems related to variable step size methods for SDEs. Most notably, we investigate which numerical methods and variable step size strategies can ensure pathwise convergence to the true SDE solution. Our analysis is based on rough path theory and requires methods to be locally close to an ODE driven by a piecewise "Brownian polynomial". This result can therefore be applied to the Heun and midpoint methods. In order to prove convergence, we will require variable step size strategies to produce a nested sequence of partitions with mesh size tending to zero.