Search

We consider the control of McKean-Vlasov dynamics (or mean-field control) with probabilistic state constraints. We rely on a level-set approach which provides a representation of the constrained problem in terms of an unconstrained one with exact penalization and running maximum or integral cost. The method is then extended to the common noise setting. Our work extends (Bokanowski, Picarelli, and Zidani, SIAM J. Control Optim. 54.5 (2016), pp. 2568--2593) and (Bokanowski, Picarelli, and Zidani, Appl. Math. Optim. 71 (2015), pp. 125--163) to a mean-field setting. The reformulation as an unconstrained problem is particularly suitable for the numerical resolution of the problem, that is achieved from an extension of a machine learning algorithm from (Carmona, Lauri{\`e}re, arXiv:1908.01613 to appear in Ann. Appl. Prob., 2019). A first application concerns the storage of renewable electricity in the presence of mean-field price impact and another one focuses on a mean-variance portfolio selection problem with probabilistic constraints on the wealth. We also illustrate our approach for a direct numerical resolution of the primal Markowitz continuous-time problem without relying on duality., Comment: To appear in Numerical Algebra, Control and Optimization

We prove a rate of convergence for the $N$-particle approximation of a second-order partial diffe\-rential equation in the space of probability measures, like the Master equation or Bellman equation of mean-field control problem under common noise. The rate is of order $1/N$ for the pathwise error on the solution $v$ and of order $1/\sqrt{N}$ for the $L^2$-error on its $L$-derivative $\partial_\mu v$. The proof relies on backward stochastic differential equations techniques.

This paper presents machine learning techniques and deep reinforcement learningbased algorithms for the efficient resolution of nonlinear partial differential equations and dynamic optimization problems arising in investment decisions and derivative pricing in financial engineering. We survey recent results in the literature, present new developments, notably in the fully nonlinear case, and compare the different schemes illustrated by numerical tests on various financial applications. We conclude by highlighting some future research directions., arXiv admin note: substantial text overlap with arXiv:2006.01496

to appear in Machine Learning And Data Sciences For Financial Markets: A Guide To Contemporary Practices. Ed. by A. Capponi. and C.A. Lehalle. Cambridge University Press, 2021. Chap. New Frontiers for Stochastic Control in Finance; International audience; This paper presents machine learning techniques and deep reinforcement learningbased algorithms for the efficient resolution of nonlinear partial differential equations and dynamic optimization problems arising in investment decisions and derivative pricing in financial engineering. We survey recent results in the literature, present new developments, notably in the fully nonlinear case, and compare the different schemes illustrated by numerical tests on various financial applications. We conclude by highlighting some future research directions.

42 pages; We develop multistep machine learning schemes for solving nonlinear partial differential equations (PDEs) in high dimension. The method is based on probabilistic representation of PDEs by backward stochastic differential equations (BSDEs) and its iterated time discretization. In the case of semilinear PDEs, our algorithm estimates simultaneously by backward induction the solution and its gradient by neural networks through sequential minimizations of suitable quadratic loss functions that are performed by stochastic gradient descent. The approach is extended to the more challenging case of fully nonlinear PDEs, and we propose different approximations of the Hessian of the solution to the PDE, i.e., the $\Gamma$-component of the BSDE, by combining Malliavin weights and neural networks. Extensive numerical tests are carried out with various examples of semilinear PDEs including viscous Burgers equation and examples of fully nonlinear PDEs like Hamilton-Jacobi-Bellman equations arising in portfolio selection problems with stochastic volatilities, or Monge-Ampère equations in dimension up to 15. The performance and accuracy of our numerical results are compared with some other recent machine learning algorithms in the literature, see \cite{HJE17}, \cite{HPW19}, \cite{BEJ19}, \cite{BBCJN19} and \cite{phawar19}. Furthermore, we provide a rigorous approximation error analysis of the deep backward multistep scheme as well as the deep splitting method for semilinear PDEs, which yields convergence rate in terms of the number of neurons for shallow neural networks.

The STochastic OPTimization library (StOpt) aims at providing tools in C++ for solving somestochastic optimization problems encountered in finance or in the industry.A python binding is available for some C++ objects provided permitting to easily solve an optimization problem by regression.Different methods are available : dynamic programming methods based on Monte Carlo with regressions (global, local and sparse regressors), for underlying states following some uncontrolled Stochastic Differential Equations (python binding provided). Semi-Lagrangian methods for Hamilton Jacobi Bellman general equations for underlying states following some controlled Stochastic Differential Equations (C++ only) Stochastic Dual Dynamic Programming methods to deal with stochastic stocks management problems in high dimension. A SDDP module in python is provided. To use this module, the transitional optimization problem has to written in C++ and mapped to python (examples provided). Some methods are provided to solve by Monte Carlo some problems where the underlying stochastic state is controlled. Some pure Monte Carlo Methods are proposed to solve some non linear PDEsFor each method, a framework is provided to optimize the problem and then simulate it out of the sample using the optimal commands previously calculated.Parallelization methods based on OpenMP and MPI are provided in this framework permitting to solve high dimensional problems on clusters.The library should be flexible enough to be used at different levels depending on the user's willingness.

Power producers are interested in valuing their power plant production. By trading into forward contracts, we propose to reduce the contingency of the associated income considering the fixed costs and using an asymmetric risk criterion. In an asymptotic framework, we provide an optimal hedging strategy through a solution of a nonlinear partial differential equation. As a numerical experiment, we analyze the impact of the fixed costs structure on the hedging policy and the value of the assets.

33 pages; We propose new machine learning schemes for solving high dimensional nonlinear partial differential equations (PDEs). Relying on the classical backward stochastic differential equation (BSDE) representation of PDEs, our algorithms estimate simultaneously the solution and its gradient by deep neural networks. These approximations are performed at each time step from the minimization of loss functions defined recursively by backward induction. The methodology is extended to variational inequalities arising in optimal stopping problems. We analyze the convergence of the deep learning schemes and provide error estimates in terms of the universal approximation of neural networks. Numerical results show that our algorithms give very good results till dimension 50 (and certainly above), for both PDEs and variational inequalities problems. For the PDEs resolution, our results are very similar to those obtained by the recent method in \cite{weinan2017deep} when the latter converges to the right solution or does not diverge. Numerical tests indicate that the proposed methods are not stuck in poor local minimaas it can be the case with the algorithm designed in \cite{weinan2017deep}, and no divergence is experienced. The only limitation seems to be due to the inability of the considered deep neural networks to represent a solution with a too complex structure in high dimension.

The STochastic OPTimization library (StOpt) aims at providing tools in C++ for solving somestochastic optimization problems encountered in finance or in the industry.A python binding is available for some C++ objects provided permitting to easily solve an optimization problem by regression.Different methods are available : dynamic programming methods based on Monte Carlo with regressions (global, local and sparse regressors), for underlying states following some uncontrolled Stochastic Differential Equations (python binding provided). Semi-Lagrangian methods for Hamilton Jacobi Bellman general equations for underlying states following some controlled Stochastic Differential Equations (C++ only) Stochastic Dual Dynamic Programming methods to deal with stochastic stocks management problems in high dimension. A SDDP module in python is provided. To use this module, the transitional optimization problem has to written in C++ and mapped to python (examples provided). Some methods are provided to solve by Monte Carlo some problems where the underlying stochastic state is controlled. Some pure Monte Carlo Methods are proposed to solve some non linear PDEsFor each method, a framework is provided to optimize the problem and then simulate it out of the sample using the optimal commands previously calculated.Parallelization methods based on OpenMP and MPI are provided in this framework permitting to solve high dimensional problems on clusters.The library should be flexible enough to be used at different levels depending on the user's willingness.

We propose a method for pricing American options whose pay-off depends on the moving average of the underlying asset price. The method uses a finite dimensional approximation of the infinite-dimensional dynamics of the moving average process based on a truncated Laguerre series expansion. The resulting problem is a finite-dimensional optimal stopping problem, which we propose to solve with a least squares Monte Carlo approach. We analyze the theoretical convergence rate of our method and present numerical results in the Black-Scholes framework.

International audience; Management of electricity production to control cost while satisfying demand, leads to solve a stochastic optimization problem where the main sources of uncertainty are the demand load, the electricity and fuel market prices, the hydraulicity, and the availability of the thermal production assets. A stochastic dynamic programming method is an interesting solution for non convex optimization, but is both CPU and memory consuming. It requires parallelization to achieve speedup and size up, and to deal with a big number of stocks (N) and a big number of uncertainty factors. This talk will introduce a collaboration between EDF (a French electricity producer) and SUPELEC (a French engineering school and research laboratory) that aimed to distribute N-dimension stochastic dynamic programming applications on large distributed architectures, like PC clusters and IBM Blue Gene supercomputers. This collaboration was initiated in a French ANR project about Distributed and Grid computing applied to financial mathematic problems (the “GCPMF” ANR project). From an applicative point of view, the goal of this research was to be able to deal with at least three or four uncertainty factors, and at least six or seven stocks in optimization, while being able to efficiently use in simulation the commands calculated. The simulations are used after optimization in order to generate gain estimations on different periods and in order to estimate the associated risks. The methodology developed in this research project will bring some reference calculations that will help to derive some simplified versions to use in production. From a computer science point of view, three different parallelization strategies have been carried out in order to access input and output files from thousands of processors, to distribute a N dimensional cube of data used at each time step of an optimization algorithm, and to compute independent simulations requiring data spread in many separate files managed by different processors. All designed parallel algorithms have been experimented on a 7- stocks problem (7-dimensions problem) on different parallel architectures. We successfully used up to 256 processors of a PC cluster and up to 8192 processors of a Blue Gene/L supercomputer, achieving scalability with regular decrease of the execution time. We started distributing a 1-dimension stochastic control algorithm (applied to a gas storage valuation) in February 2007, and we extended our distribution to a N- dimension algorithm in 2008 (applied to electricity production management). In the next months this industrial and large scale distributed application will be used: – by EDF to study and optimize its energetic stock management and electricity production, using its new Blue Gene/P supercomputer up to 32000 processors; – by SUPELEC (IMS group) and INRIA (AlGorille and Reso teams) to run large experiments on Grid'5000, analyze communications and performances, and optimize task distribution when using several sites of Grid'5000. A global collaboration between EDF, SUPELEC and INRIA will allow comparing performances of this real and not embarrassingly parallel application, on supercomputers, different large PC-clusters and one multi-site Grid.

International audience; Management of French electricity production to control cost while satisfying demand, leads to solve a stochastic optimization problem where the main sources of uncertainty are the demand load, the electricity and fuel market prices, the hydraulicity, and the availability of the thermal production assets. A stochastic dynamic programming method is an interesting solution, but is both CPU and memory consuming. It requires parallelization to achieve speedup and size up, and to deal with a big number of stocks (N) and a big number of uncertainty factors. This paper introduces a distribution of a N-dimension stochastic dynamic programming application, on PC clusters and IBM Blue Gene/L super-computer. It has needed to parallelize input and output file accesses from thousands of processors, to load balance a N-dimension cube of data and computation evolving at each time step, and to compute Monte-Carlo simulations requiring data spread in many separate files managed by different processors. Finally, a successful experiment of a 7-stock problem using up to 8192 processors validates this distribution strategy.

International audience; This paper introduces the distribution of a stochastic control algorithm which is applied to gas storage valuation, and presents its experimental performances on two PC clusters and an IBM Blue Gene/L supercomputer. This research is part of a French national project which gathers people from the academic world (computer scientists, mathematicians, ...) as well as people from the industry of energy and finance in order to provide concrete answers on the use of computational clusters, grids and supercomputers applied to problems of financial mathematics. The designed distribution allows to run gas storage valuation models which require considerable amounts of computational power and memory space while achieving both speedup and size-up: it has been successfully implemented and experimented on PC clusters (up to 144 processors) and on a Blue Gene supercomputer (up to 1024 processors). Finally, our distributed algorithm allows to use more computing resources in order to maintain constant the execution time while increasing the calculation accuracy.