Search

In this paper, we study the error in first order Sobolev norm in the approximation of solutions to linear parabolic PDEs. We use a Monte Carlo Euler scheme obtained from combining the Feynman--Kac representation with a Euler discretization of the underlying stochastic process. We derive approximation rates depending on the time-discretization, the number of Monte Carlo simulations, and the dimension. In particular, we show that the Monte Carlo Euler scheme breaks the curse of dimensionality with respect to the first order Sobolev norm. Our argument is based on new estimates on the weak error of the Euler approximation of a diffusion process together with its derivative with respect to the initial condition. As a consequence, we obtain that neural networks are able to approximate solutions of linear parabolic PDEs in first order Sobolev norm without the curse of dimensionality if the coefficients of the PDEs admit an efficient approximation with neural networks.

Dynamical systems theory has recently been applied in optimization to prove that gradient descent algorithms avoid so-called strict saddle points of the loss function. However, in many modern machine learning applications, the required regularity conditions are not satisfied. In particular, this is the case for rectified linear unit (ReLU) networks. In this paper, we prove a variant of the relevant dynamical systems result, a center-stable manifold theorem, in which we relax some of the regularity requirements. Then, we verify that shallow ReLU networks fit into the new framework. Building on a classification of critical points of the square integral loss of shallow ReLU networks measured against an affine target function, we deduce that gradient descent avoids most saddle points. We proceed to prove convergence to global minima if the initialization is sufficiently good, which is expressed by an explicit threshold on the limiting loss.

In this paper we develop a new machinery to study the capacity of artificial neural networks (ANNs) to approximate high-dimensional functions without suffering from the curse of dimensionality. Specifically, we introduce a concept which we refer to as approximation spaces of artificial neural networks and we present several tools to handle those spaces. Roughly speaking, approximation spaces consist of sequences of functions which can, in a suitable way, be approximated by ANNs without curse of dimensionality in the sense that the number of required ANN parameters to approximate a function of the sequence with an accuracy $\varepsilon > 0$ grows at most polynomially both in the reciprocal $1/\varepsilon$ of the required accuracy and in the dimension $d \in \mathbb{N} = \{1, 2, 3, \ldots \}$ of the function. We show under suitable assumptions that these approximation spaces are closed under various operations including linear combinations, formations of limits, and infinite compositions. To illustrate the power of the machinery proposed in this paper, we employ the developed theory to prove that ANNs have the capacity to overcome the curse of dimensionality in the numerical approximation of certain first order transport partial differential equations (PDEs). Under suitable conditions we even prove that approximation spaces are closed under flows of first order transport PDEs., 32 pages

In this article we introduce and study a deep learning based approximation algorithm for solutions of stochastic partial differential equations (SPDEs). In the proposed approximation algorithm we employ a deep neural network for every realization of the driving noise process of the SPDE to approximate the solution process of the SPDE under consideration. We test the performance of the proposed approximation algorithm in the case of stochastic heat equations with additive noise, stochastic heat equations with multiplicative noise, stochastic Black--Scholes equations with multiplicative noise, and Zakai equations from nonlinear filtering. In each of these SPDEs the proposed approximation algorithm produces accurate results with short run times in up to 50 space dimensions.

In this paper we study a class of combined regular and singular stochastic control problems that can be expressed as constrained BSDEs. In the Markovian case, this reduces to a characterization through a PDE with gradient constraint. But the BSDE formulation makes it possible to move beyond Markovian models and consider path-dependent problems. We also provide an approximation of the original control problem with standard BSDEs that yield a characterization of approximately optimal values and controls.

This paper provides versions of classical results from linear algebra, real analysis and convex analysis in a free module of finite rank over the ring $L^0$ of measurable functions on a $\sigma$-finite measure space. We study the question whether a submodule is finitely generated and introduce the more general concepts of $L^0$-affine sets, $L^0$-convex sets, $L^0$-convex cones, $L^0$-hyperplanes, $L^0$-half-spaces and $L^0$-convex polyhedral sets. We investigate orthogonal complements, orthogonal decompositions and the existence of orthonormal bases. We also study $L^0$-linear, $L^0$-affine, $L^0$-convex and $L^0$-sublinear functions and introduce notions of continuity, differentiability, directional derivatives and subgradients. We use a conditional version of the Bolzano-Weierstrass theorem to show that conditional Cauchy sequences converge and give conditions under which conditional optimization problems have optimal solutions. We prove results on the separation of $L^0$-convex sets by $L^0$-hyperplanes and study $L^0$-convex conjugate functions. We provide a result on the existence of $L^0$-subgradients of $L^0$-convex functions, prove a conditional version of the Fenchel-Moreau theorem and study conditional inf-convolutions.

We provide existence results and comparison principles for solutions of backward stochastic difference equations (BS$��$Es) and then prove convergence of these to solutions of backward stochastic differential equations (BSDEs) when the mesh size of the time-discretizaton goes to zero. The BS$��$Es and BSDEs are governed by drivers $f^N(t,��,y,z)$ and $f(t,��,y,z),$ respectively. The new feature of this paper is that they may be non-Lipschitz in z. For the convergence results it is assumed that the BS$��$Es are based on d-dimensional random walks $W^N$ approximating the d-dimensional Brownian motion W underlying the BSDE and that $f^N$ converges to f. Conditions are given under which for any bounded terminal condition $��$ for the BSDE, there exist bounded terminal conditions $��^N$ for the sequence of BS$��$Es converging to $��$, such that the corresponding solutions converge to the solution of the limiting BSDE. An important special case is when $f^N$ and f are convex in z. We show that in this situation, the solutions of the BS$��$Es converge to the solution of the BSDE for every uniformly bounded sequence $��^N$ converging to $��$. As a consequence, one obtains that the BSDE is robust in the sense that if $(W^N,��^N)$ is close to $(W,��)$ in distribution, then the solution of the Nth BS$��$E is close to the solution of the BSDE in distribution too., Published in at http://dx.doi.org/10.3150/12-BEJ445 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

We introduce a class of second order backward stochastic differential equations and show relations to fully non-linear parabolic PDEs. In particular, we provide a stochastic representation result for solutions of such PDEs and discuss Monte Carlo methods for their numerical treatment., Comment: 26 pages

Dai and Singleton (2000) study a class of term structure models for interest rates that specify the short rate as an affine combination of the components of an N-dimensional affine diffusion process. Observable quantities in such models are invariant under regular affine transformations of the underlying diffusion process. In their canonical form, the models in Dai and Singleton (2000) are based on diffusion processes with diagonal diffusion matrices. This motivates the following question: Can the diffusion matrix of an affine diffusion process always be diagonalized by means of a regular affine transformation? We show that if the state space of the diffusion is of the form D = R-+(m) x RN-m for integers 0 = N - 1, there exists a regular affine transformation of D onto itself that diagonalizes the diffusion matrix. So in this case, the Dai-Singleton canonical representation is exhaustive. On the other hand, we provide examples of affine diffusion processes with state space R-+(2) x R-2 whose diffusion matrices cannot be diagonalized through regular affine transformation. This shows that for 2

We propose a general framework for studying last passage times, suprema, and drawdowns of a large class of continuous-time stochastic processes. Our approach is based on processes of class Sigma and the more general concept of two processes, one of which moves only when the other is at the origin. After investigating certain transformations of such processes and their convergence properties, we provide three general representation results. The first allows the recovery of a process of class Sigma from its final value and the last time it visited the origin. In many situations this gives access to the distribution of the last time a stochastic process attains a certain level or is equal to its running maximum. It also leads to recently discovered formulas expressing option prices in terms of last passage times. Our second representation result is a stochastic integral representation that will allow us to price and hedge options on the running maximum of an underlying that are triggered when the underlying drops to a given level or, alternatively, when the drawdown or relative drawdown of the underlying attains a given height. The third representation gives conditional expectations of certain functionals of processes of class Sigma. It can be used to deduce the distributions of a variety of interesting random variables such as running maxima, drawdowns, and maximum drawdowns of suitably stopped processes. Copyright © 2012 by SIAM.