Search

In this paper we explore optimal liquidation in a market populated by a number of heterogeneous market makers that have limited inventory-carrying and risk-bearing capacity. We derive a reduced form model for the dynamics of their aggregated inventory considering a proper scaling limit. The resulting price impact profile is shown to depend on the characteristics and relative importance of their inventories. The model is flexible enough to reproduce the empirically documented power law behavior of the price impact function. For any choice of the market makers characteristics, optimal execution within this modeling approach can be recast as a linear-quadratic stochastic control problem. The value function and the associated optimal trading rate can be obtained semi-explicitly subject to solving a differential matrix Riccati equation. Numerical simulations are conducted to illustrate the performance of the resulting optimal liquidation strategy in relation to standard benchmarks. Remarkably, they show that the increase in performance is determined by a substantial reduction of higher order moment risk. [ABSTRACT FROM AUTHOR]

This paper is concerned with the turnpike property for a class of stochastic linear-quadratic (LQ, for short) optimal control problems with periodic coefficients. The stability and stabilizability of the control system are studied, followed by the discussion of the existence and uniqueness of periodic solutions. A deterministic periodic LQ problem is introduced and solved, whose optimal pair, together with a pair of correction processes, serves as the turnpike limit of the stochastic problem. It is shown that the turnpike limit is periodic in the distribution sense. In the special case of constant coefficients, the turnpike limit turns out to have stationary distributions, with the expectation being the solution to a static optimization problem. [ABSTRACT FROM AUTHOR]

This paper deals with finite‐horizon minimum‐variance and covariance steering problems subject to constraints. The goal of the minimum variance problem is to steer the state mean of an uncertain system to a prescribed vector while minimizing the trace of its terminal state covariance whereas the goal in the covariance steering problem is to steer the covariance matrix of the terminal state to a prescribed positive definite matrix. The paper proposes a solution approach that relies on a stochastic version of the affine disturbance feedback control parametrization. In this control policy parametrization, the control input at each stage is expressed as an affine function of the history of disturbances that have acted upon the system. It is shown that this particular parametrization reduces the stochastic optimal control problems considered in this paper into tractable convex programs or difference of convex functions programs with essentially the same decision variables. In addition, the paper proposes a variation of this control parametrization that relies on truncated histories of past disturbances, which allows for sub‐optimal controllers to be designed that strike a balance between performance and computational cost. The suboptimality of the truncated policies is formally analyzed and closed form expressions are provided for the performance loss due to the use of the truncation scheme. Finally, the paper concludes with a comparative analysis of the truncated versions of the proposed policy parametrization and other standard policy parametrizations through numerical simulations. [ABSTRACT FROM AUTHOR]

We consider optimal control problems for a system governed by a stochastic differential equation driven by a d-dimensional Brownian motion where both the drift and the diffusion coefficient are controlled. It is well known that without additional convexity conditions the strict control problem does not admit an optimal control. To overcome this difficulty, we consider the relaxed model, in which admissible controls are measure-valued processes and the relaxed state process is governed by a stochastic differential equation driven by a continuous orthogonal martingale measure. This relaxed model admits an optimal control that can be approximated by a sequence of strict controls by the so-called chattering lemma. We establish optimality necessary conditions, in terms of two adjoint processes, extending Peng's maximum principle to relaxed control problems. We show that relaxing the drift and diffusion martingale parts directly as in deterministic control does not lead to a true relaxed model as the obtained controlled dynamics is not continuous in the control variable. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

Sampling rare events in metastable dynamical systems is often a computationally expensive task and one needs to resort to enhanced sampling methods such as importance sampling. Since we can formulate the problem of finding optimal importance sampling controls as a stochastic optimization problem, this then brings additional numerical challenges and the convergence of corresponding algorithms might suffer from metastabilty. In this article, we address this issue by combining systematic control approaches with the heuristic adaptive metadynamics method. Crucially, we approximate the importance sampling control by a neural network, which makes the algorithm in principle feasible for high-dimensional applications. We can numerically demonstrate in relevant metastable problems that our algorithm is more effective than previous attempts and that only the combination of the two approaches leads to a satisfying convergence and therefore to an efficient sampling in certain metastable settings. [ABSTRACT FROM AUTHOR]

We study the optimal consumption, investment and lifeinsurance purchase and selection strategies for a wage-earner with an uncertain lifetime with access to a financial market comprised of one risk-free security and one risky-asset whose prices evolve according to linear diffusions modulated by a continuous-time stochastic process determined by an additional diffusive nonlinear stochastic differential equation. The process modulating the linear diffusions may be regarded as an indicator describing the state of the economy in a given instant of time. Additionally, we allow the Brownian motions driving each of these equations to be correlated. The lifeinsurance market under consideration herein consists of a fixed number of providers offering pairwise distinct contracts. We use dynamic programming techniques to characterize the solutions to the problem described above for a general family of utility functions, studying the case of discounted constant relative risk aversion utilities with more detail. [ABSTRACT FROM AUTHOR]

This paper is concerned with an optimal control problem for a mean-field linear stochastic differential equation with a quadratic functional in the infinite time horizon. Under suitable conditions, including the stabilizability, the (strong) exponential, integral, and mean-square turnpike properties for the optimal pair are established. The keys are to correctly formulate the corresponding static optimization problem and find the equations determining the correction processes. These have revealed the main feature of the stochastic problems which are significantly different from the deterministic version of the theory. [ABSTRACT FROM AUTHOR]

We consider a class of stochastic optimal control problems with partial observation, and study their approximation by discrete-time control problems. We establish a convergence result by using the weak convergence technique of Kushner and Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York], together with the notion of relaxed control rule introduced by El Karoui, Huú Nguyen and Jeanblanc-Picqué [SIAM J. Control Optim., 26 (1988), pp. 1025--1061]. In particular, with a well chosen discrete-time control system, we obtain a first implementable numerical algorithm (with convergence) for the partially observed control problem. Moreover, our discrete-time approximation result would open the door to study convergence of more general numerical approximation methods, such as machine learning based methods. Finally, we illustrate our convergence result by numerical experiments on a partially observed control problem in a linear quadratic setting. [ABSTRACT FROM AUTHOR]

This paper focuses on the application of meta-reinforcement learning to the robust design of low-thrust interplanetary trajectories in the presence of multiple uncertainties. A closed-loop control policy is used to optimally steer the spacecraft to a final target state despite the considered perturbations. The control policy is approximated by a deep recurrent neural network, trained by policy-gradient reinforcement learning on a collection of environments featuring mixed sources of uncertainty, namely dynamic uncertainty and control execution errors. The recurrent network is able to build an internal representation of the distribution of environments, thus better adapting the control to the different stochastic scenarios. The results in terms of optimality, constraint handling, and robustness on a fuel-optimal low-thrust transfer between Earth and Mars are compared with those obtained via a traditional reinforcement learning approach based on a feed-forward neural network. • In interplanetary space missions, the spacecraft trajectory is affected by multiple uncertainties. • Meta-reinforcement learning can be applied seamlessly to any uncertainty and dynamic model. • A recurrent neural network is used as a history-dependent closed-loop control policy. • The network is trained on a low-thrust transfer featuring dynamic uncertainties and control execution errors. • Meta-reinforcement learning shows improved performance and robustness compared to standard reinforcement learning. [ABSTRACT FROM AUTHOR]

This paper focuses on a stochastic goodwill model involving a single manufacturer and a single retailer in a supply chain, in which both quality level and goodwill level are governed by stochastic differential equations. In both the linear-quadratic Stackelberg game and linear-quadratic cooperative game, explicit expressions are obtained for the optimal quality improvement efforts level of the manufacturer, the optimal advertising efforts level of the retailer as well as the optimal profit of the supply chain. We also derive the expectation and variance values for quality and goodwill levels, respectively. The distribution of incremental profit and subsidy are analysed in the cooperative game via the utility theory. Finally, several numerical experiments are reported to show the influence of decay rates and white noise disturbance of the quality and goodwill levels on the expectation of quality and goodwill levels. [ABSTRACT FROM AUTHOR]

The optimal control of non-instantaneous impulsive second-order stochastic McKean–Vlasov evolution system with Clarke subdifferential and mixed fractional Brownian motion is investigated in this article. The deterministic nonlinear second-order controlled partial differential system is enriched with stochastic perturbations, non-instantaneous impulses, and Clarke subdifferential. In particular, the nonlinearities in the system that rely on the state of the solution are allowed to rely on the corresponding probability distribution of the state. The solvability of the considered system is discussed with the help of stochastic analysis, multivalued analysis, and multivalued fixed point theorem. Further, the existence of optimal control is established with the aid of Balder's theorem. Finally, an example is provided to illustrate the developed theory. [ABSTRACT FROM AUTHOR]

To overcome the time-consuming training caused by the sparse reward function in reinforcement learning, an efficient dense reward framework for robust low-thrust trajectory optimization is proposed. The dense reward functions are designed separately for the deterministic and considered uncertain scenarios, including state uncertainties, observation uncertainties and execution uncertainties. For the uncertainties, the dense reward function is designed to diminish the deviation with respect to the isochronous nominal state along the corresponding deterministic optimal trajectory at each step, rendering the reward function no longer sparse and suitable for complex multirevolution problems. In addition, a multistage reward function of the terminal constraints for the rendezvous missions is designed by incorporating some exponential acceleration terms, enabling significant improvement in training efficiency as the terminal errors become low. In addition, a dense reward function for the deterministic scenario is also proposed via the introduction of empirical forbidden zones and an exponential term. The effectiveness and efficiency of the proposed method is demonstrated in a simple Earth-Mars mission and a complex Earth-Venus multirevolution mission. The promising results verify the significant effect of the proposed method in speeding up the process of training an initial incapable agent to an 'expert' while guaranteeing or even improving the performance. [ABSTRACT FROM AUTHOR]

We develop an approach for two-player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different types of constraints, which lead us to find real-valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with the stochastic maximum principle technique. Our application is an example of cooperation between a bank and an insurance company, which is a popular, well-known business agreement type called Bancassurance. [ABSTRACT FROM AUTHOR]

The Pontryagin type maximum principle and Bellman's dynamic programming principle serve as two of the most important tools in solving optimal control problems. There is a huge literature on the study of relationship between them. The main purpose of this paper is to investigate the relationship between the Pontryagin type maximum principle and the dynamic programming principle for control systems governed by stochastic evolution equations in infinite dimensional space, with the control variables entering into both the drift and the diffusion terms. To do so, we first prove the dynamic programming principle for those systems without employing the martingale solutions. Then we establish the desired relationships in both cases that value function associated is smooth or nonsmooth. For the nonsmooth case, in particular, by employing the relaxed transposition solution, we discover the connection between the superdifferentials and subdifferentials of value function and the first-order and second-order adjoint equations. [ABSTRACT FROM AUTHOR]

This study applies the theory of stochastic control to an environmental project planning to counteract against the sediment starvation problem in river environments. This can be considered as a time‐average inventory problem to time‐discretely control a continuous‐time system driven by a non‐smooth jump process under habit formation disturbing project implementation. The system is modeled such that the sediment storage dynamics are physically consistent with certain experimental results. Further, the habit formation is modeled as simple linear dynamics and serves as a constraint related to the replenishment amount of the sediment. We show that the time‐average control problem is not necessarily ergodic. Consequently, the effective Hamiltonian may become a non‐constant. Thereafter, ratcheting cases as extreme cases of the irreversible habit formation are considered, owing to them being unique exactly solvable non‐ergodic control problems. The optimality equation associated with a regularized and hence well‐defined control problem is verified. Furthermore, a finite difference scheme is examined against the exactly solvable case and then applied to more complicated cases. [ABSTRACT FROM AUTHOR]

This paper studies the numerical algorithm of stochastic control problems in investment optimization. Investors choose the optimal investment to maximize the expected return under uncertainty. The optimality condition, the Hamilton–Jacobi–Bellman (HJB) equation, satisfied by the value function and obtained by the dynamic programming method, is a partial differential equation coupled with optimization. One of the major computational difficulties is the irregular boundary conditions presented in the HJB equation. In this paper, two mesh-free algorithms are proposed to solve two different cases of HJB equations with regular and irregular boundary conditions. The model of optimal investment under uncertainty developed by Abel is used to study the efficacy of the proposed algorithms. Extensive numerical studies are conducted to test the impact of the key parameters on the numerical efficacy. By comparing the numerical solution with the exact solution, the proposed numerical algorithms are validated. [ABSTRACT FROM AUTHOR]

In this paper, we solve an optimal reinsurance problem in the mathematical finance area. We assume that the surplus process of the insurance company follows a controlled diffusion process and the constant interest rate is involved in the financial model. During the whole optimization period, the company has a choice to buy reinsurance contract and decide the reinsurance retention level. Meanwhile, the bankruptcy at the terminal time is not allowed. The aim of the optimization problem is to minimize the distance between the terminal wealth and a given goal by controlling the reinsurance proportion. Using the stochastic control theory, we derive the Hamilton-Jacobi-Bellman equation for the optimization problem. Via adopting the technique of changing variable as well as the dual transformation, an explicit solution of the value function and the optimal policy are shown. Finally, several numerical examples are shown, from which we find several main factors that affect the optimal reinsurance policy. [ABSTRACT FROM AUTHOR]

The optimization and control of spacecraft trajectories in presence of uncertainties and unmodeled disturbances represents a challenging problem. Future missions would benefit from robust control techniques to compute their trajectory in a complex, nonlinear environment and in presence of stochasticity. In this work the Stochastic Optimal Control problem in the Circular Restricted Three Body Problem is addressed and a method is proposed based on Nonlinear Model Predictive control (NMPC). This method presents the challenge of solving a nonlinear optimization problem on-board. A Quantum-inspired meta-heuristic optimization algorithm is tested in this context: the Diffusion Monte Carlo (DMC) method is based on a numerical techniques used in Quantum Mechanics, adapted to solve optimization problems. The NMPC with DMC is implemented, taking into account an estimate of the computational time required through the introduction of delayed actuation of the control. Two applications are presented featuring transfer trajectories between planar periodic orbits in the vicinity of the Moon; the proposed method proved able to successfully generate trajectories when disturbance accelerations and uncertain initial conditions are present. • The Circular Restricted Three Body Problem with uncertainties is considered. • A NMPC is used to drive the spacecraft to the desired orbit despite the uncertainty. • A Quantum-inspired meta-heuristic optimizer is used in the MPC. • Applications are presented regarding transfers between Lunar periodic orbits. [ABSTRACT FROM AUTHOR]

To address the issue of inaccurate distributions in discrete-time stochastic systems, a minimax linear quadratic control method using theWasserstein metric is proposed. Our method aims to construct a control policy that is robust against errors in an empirical distribution of underlying uncertainty by adopting an adversary that selects the worst-case distribution at each time. The opponent receives a Wasserstein penalty proportional to the amount of deviation from the empirical distribution. As a tractable solution, a closed-form expression of the optimal policy pair is derived using a Riccati equation. We identify nontrivial stabilizability and observability conditions under which the Riccati recursion converges to the unique positive semidefinite solution of an algebraic Riccati equation. Our method is shown to possess several salient features, including closed-loop stability, a guaranteed-cost property, and a probabilistic out-of-sample performance guarantee. [ABSTRACT FROM AUTHOR]

A stochastic epidemic model with random noise transmission is taken into account, describing the dynamics of the measles viral infection. The basic reproductive number is calculated corresponding to the stochastic model. It is determined that, given initial positive data, the model has bounded, unique, and positive solution. Additionally, utilizing stochastic Lyapunov functional theory, we study the extinction of the disease. Stationary distribution and extinction of the infection are examined by providing sufficient conditions. We employed optimal control principles and examined stochastic control systems to regulate the transmission of the virus using environmental factors. Graphical representations have been offered for simplicity of comprehending in order to further verify the acquired analytical findings. [ABSTRACT FROM AUTHOR]

We study singular perturbations of a class of two-scale stochastic control systems with unbounded data. The assumptions are designed to cover some relaxation problems for deep neural networks. We construct effective Hamiltonian and initial data and prove the convergence of the value function to the solution of a limit (effective) Cauchy problem for a parabolic equation of HJB type. We use methods of probability, viscosity solutions and homogenization. [ABSTRACT FROM AUTHOR]

We study an exit contract design problem, where one provides a universal exit contract to multiple heterogeneous agents, with which each agent chooses an optimal (exit) stopping time. The problem consists in optimizing the universal exit contract w.r.t. some criterion depending on the contract as well as the agents' exit times. Under a technical monotonicity condition, and by using Bank-El Karoui's representation of stochastic processes, we are able to transform the initial contract optimization problem into an optimal control problem. The latter is also equivalent to an optimal multiple stopping problem and the existence of the optimal contract is proved. We next show that the problem in the continuous-time setting can be approximated by a sequence of discrete-time ones, which would induce a natural numerical approximation method. We finally discuss the optimization problem over the class of all Markovian and/or continuous exit contracts. [ABSTRACT FROM AUTHOR]

This paper considers an optimal control problem for a time-invariant linear stochastic system with discrete time, scalar unbounded control, additive noise, and a probabilistic criterion for retaining its trajectories in a given neighborhood of zero. We use dynamic programming and two-sided Bellman function estimates to derive analytical expressions for the optimal control at two time steps and a suboptimal control on any control horizon. The effectiveness of these controls is illustrated on a numerical example. [ABSTRACT FROM AUTHOR]

We consider the diffusive limit of a typical pure-jump Markovian control problem as the intensity of the driving Poisson process tends to infinity. We show that the convergence speed is provided by the Hölder exponent of the Hessian of the limit problem, and explain how correction terms can be constructed. This provides an alternative efficient method for the numerical approximation of the optimal control of a pure-jump problem in situations with very high intensity of jumps. We illustrate this approach in the context of a display advertising auction problem. [ABSTRACT FROM AUTHOR]

This paper is concerned with a class of mean-feld type coupled forward-backward stochastic diferential equations (MF-FBSDEs), in which the coupling appears in integral terms, terminal terms, and initial terms. Inspired by various mean-feld type linear-quadratic (MF-LQ) optimal control problems, we propose a type of randomized domination-monotonicity conditions, under which and the usual Lipschitz condition we obtain a well-posedness result on MF-FBSDEs in the sense of square integrability including the unique solvability, an estimate of the solution, and the related continuous dependence property of the solution on the coefcients. The result of MF- FBSDEs in turn extends MF-LQ problems in the literature to a general situation where the initial states or the terminal states are also controlled at the same time, and gives explicit expressions of the related unique optimal controls. [ABSTRACT FROM AUTHOR]

This paper is devoted to the stochastic optimal control problem of ordinary differential equations allowing for both path-dependence and measurable randomness. As opposed to the deterministic path-dependent cases, the value function turns out to be a random field on the path space and it is characterized by a stochastic path-dependent Hamilton–Jacobi (SPHJ) equation. A notion of viscosity solution is proposed and the value function is proved to be the unique viscosity solution to the associated SPHJ equation. [ABSTRACT FROM AUTHOR]

In this paper, we consider the optimal reinsurance problem for a risk model with a thinning-dependence structure, where the stochastic sources related to claim occurrence are classified into different groups, and each group may cause a claim in each insurance class with some probability. We assume that the insurer can manage the risk by purchasing per-loss reinsurance, and their aim is to maximize the expected utility of the terminal wealth. By using the technique of stochastic control, we obtain the corresponding Hamilton–Jaccobi–Bellman equation. From the perspective of game theory, we derive the closed-form expression of the optimal strategy for each class of business, which is actually the best response to other given strategies. We also investigate the necessary conditions for optimal strategies and transfer the original optimization problem into a system of equations. Furthermore, we prove that the solution of the system of equations always exists, but may not be unique, and we also study some features of the optimal strategies in special cases and derive several interesting results. Finally, some numerical examples are given to show the impacts of some important parameters on the optimal strategies. [ABSTRACT FROM AUTHOR]

Numerous applied models used in the study of optimal control problems, inverse problems, shape optimization, machine learning, fractional programming, neural networks, image registration and so on lead to stochastic optimization problems in Hilbert spaces. Under a suitable convexity assumption on the objective function, a necessary and sufficient optimality condition for stochastic optimization problems is a stochastic variational inequality. This article presents a new stochastic regularized second-order iterative scheme for solving a variational inequality in a stochastic environment where the primary operator is accessed by employing sampling techniques. The proposed iterative scheme, which fits within the general framework of the stochastic approximation approach, has its almost-sure convergence analysis given in a Hilbert space. We test the feasibility and the efficacy of the proposed stochastic approximation approach for a stochastic optimal control problem and a stochastic inverse problem, both associated with a second-order stochastic partial differential equation. This article is part of the theme issue 'Non-smooth variational problems and applications'. [ABSTRACT FROM AUTHOR]

We present a pair of adjoint optimal control problems characterizing a class of time-symmetric stochastic processes defined on random time intervals. The associated PDEs are of free-boundary type. The particularity of our approach is that it involves two adjoint optimal stopping times adapted to a pair of filtrations, the traditional increasing one and another, decreasing. They are the keys of the time symmetry of the construction, which can be regarded as a generalization of 'Schrödinger's problem' (1931–1932) to space-time domains. The relation with the notion of 'Hidden diffusions' is also described. [ABSTRACT FROM AUTHOR]

In this work, we provide discrete optimality conditions of the optimal control problems of stochastic differential equations. Euler and Runge–Kutta methods are used for discretization. A Lagrange multiplier method for a discrete-time stochastic optimal control problem is formulated. The discrete adjoint process p n is obtained in terms of conditional expectations E [ p n + 1 ] and E [ p n + 1 Δ W ] for both methods. To estimate these nested conditional expectations at each time step via simulation, we use a very powerful new approach, least-squares Monte-Carlo method, developed by Longstaff–Schwartz. This is the first time to solve a stochastic optimal control problem by calculating the nested conditional expectations numerically with the help of a least-squares Monte-Carlo method. Some examples are studied to test and demonstrate the efficiency of the Lagrange multiplier combined with the least-squares Monte-Carlo method. [ABSTRACT FROM AUTHOR]

A controlled discrete-time stochastic process is proposed as a model for the state of a machine. In the proposed model, normal and random wear of the machine are considered. The random wear of the machine during a time unit can be either a discrete or a continuous random variable. The objective is to find a control policy that maximises the profits generated by the machine over its useful lifetime. In this problem, the optimiser must decide whether to do or not to do the maintenance work on the machine at each time unit. The significance of the paper is that the final time in the optimal control problem is a random variable denoting the first time the machine must be replaced. To obtain the optimal control, one can try to solve the dynamic programming equation, which reduces to a difference or an integral equation, satisfied by the value function. Finally, particular cases are solved exactly, or approximately, and explicitly to demonstrate and validate the proposed model. [ABSTRACT FROM AUTHOR]

This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time-horizon [0, T] as T → ∞. The so-called turnpike properties are established for such problems, under stabilizability condition which is weaker than the controllability, normally imposed in the similar problem for ordinary differential systems. In dealing with the turnpike problem, a crucial issue is to determine the corresponding static optimization problem. Intuitively mimicking the deterministic situations, it seems to be natural to include both the drift and the diffusion expressions of the state equation to be zero as constraints in the static optimization problem. However, this would lead us to a wrong direction. It is found that the correct static problem should contain the diffusion as a part of the objective function, which reveals a deep feature of the stochastic turnpike problem. [ABSTRACT FROM AUTHOR]

This article studies the formation mission design problem for commercial aircraft in the presence of uncertainties. Specifically, it considers uncertainties in the departure times of the aircraft and in the fuel burn savings for the trailing aircraft. Given several commercial flights, the problem consists in arranging them in formation or solo flights and finding the trajectories that minimize the expected value of the DOC of the flights. The formation mission design problem is formulated as an optimal control problem of a stochastic switched dynamical system and solved using nonintrusive gPC-based stochastic collocation. The stochastic collocation method converts the SSOCP into an augmented deterministic switched optimal control problem. With this approach, a small number of sample points of the random parameters are used to jointly solve particular instances of the switched optimal control problem. The obtained solutions are then expressed as orthogonal polynomial expansions in terms of the random parameters using these sample points. This technique allows statistical and global sensitivity analysis of the stochastic solutions to be conducted at a low computational cost. The aim of this article is to establish if, in the presence of uncertainties, a formation mission is beneficial with respect to solo flight in terms of the expected value of the direct operating costs. Several numerical experiments have been conducted in which uncertainties on the departure times and on the fuel saving during formation flight have been considered. The obtained results demonstrate that benefits can be achieved even in the presence of these uncertainties. [ABSTRACT FROM AUTHOR]

We develop a new iterative method based on the Pontryagin principle to solve stochastic control problems. This method is nothing other than the Newton method extended to the framework of stochastic optimal control, where the state dynamics are given by an ODEs with stochastic coefficients and the cost is random. Each iteration of the method is made of two ingredients: computing the Newton direction, and finding an adapted step length. The Newton direction is obtained by solving an affine-linear Forward-Backward Stochastic Differential Equation (FBSDE) with random coefficients. This is done in the setting of a general filtration. Solving such an FBSDE reduces to solving a Riccati Backward Stochastic Differential Equation (BSDE) and an affine-linear BSDE, as expected in the framework of linear FBSDEs or Linear-Quadratic stochastic control problems. We then establish convergence results for this Newton method. In particular, Lipschitz-continuity of the second-order derivative of the cost functional is established with an appropriate choice of norm and under boundedness assumptions, which is sufficient to prove (local) quadratic convergence of the method in the space of uniformly bounded processes. To choose an appropriate step length while fitting our choice of space of processes, an adapted Backtracking line search method is developed. We then prove global convergence of the Newton method with the proposed line search procedure, which occurs at a quadratic rate after finitely many iterations. An implementation with regression techniques to solve BSDEs arising in the computation of the Newton step is developed. We apply it to the control problem of a large number of batteries providing ancillary services to an electricity network. [ABSTRACT FROM AUTHOR]

The aim of the paper is to apply the stochastic optimal control problem in order to optimize the number of individual which will have the pre-exposure prophylaxis (PReP) treatment in the stochastic model for HIV/AIDS with PReP. By using the stochastic maximum principle, we derive the stochastic optimal control of PReP for the unconstrained control problem. Furthermore, by combining the stochastic maximum principle with a version of the Lagrange multiplier method, we solve the PReP problem for two different types of budget constrains with a given constrain for the costs (possible of different kind, transportation, price of the treatment, etc.). Obtained results for the different percentage of the individuals who got the vaccine, as well as results for unconstrained and constrained problems, are illustrated by a numerical example. [ABSTRACT FROM AUTHOR]

The aim of this paper is to derive a maximum principle for a control problem governed by a stochastic partial differential equation (SPDE) with locally monotone coefficients. To reach our goal we adapt the method which uses the relation between backward stochastic partial differential equation (BSPDE) and the maximum principle. In particular, necessary conditions for optimality for this stochastic optimal control problem are obtained. In spite of the fact that the method used here was used by several authors before, our adaptation is not immediate. It applies a trick which is used to get estimates for solutions of SPDE with Locally Monotone Coefficients as in the proof of the Lemmas 5.1 and 5.3. This adaptation permits us to apply our results to get a maximum principle for the optimal control to the cases when the system is governed by the 2D stochastic Navier-Stokes equation and by a stochastic Burgers' equation. [ABSTRACT FROM AUTHOR]

We describe the growth dynamics of a stock using stochastic differential equations with a generalized logistic growth model which encompasses several well-known growth functions as special cases. For each model, we compute the optimal variable effort policy and compare the expected net present value of the total profit earned by the harvester among policies. In addition, we further extend the study to include parameters sensitivity, such as the costs and volatility, and present an explicitly Crank–Nicolson discretization scheme necessary to obtain optimal policies. [ABSTRACT FROM AUTHOR]

We consider a stochastic optimal control problem with state variable dynamics described by a stochastic differential equation of diffusive type modulated by a semi-Markov process with a finite state space. The time horizon is both deterministic and finite. Within such setup, we provide a detailed proof of the dynamic programming principle and use it to characterize the value function as a viscosity solution of the corresponding Hamilton-Jacobi-Bellman equation. We illustrate our results with an application to Mathematical Finance: the generalization of Merton's optimal consumption-investment problem to financial markets with semi-Markov switching. [ABSTRACT FROM AUTHOR]

Stochastic flows of an advective-diffusive nature are ubiquitous in biology and the physical sciences. Of particular interest is the problem of reconciling observed marginal distributions with a given prior posed by Schrödinger in 1932 and known as the Schrödinger Bridge Problem (SBP). It turns out that Schrödinger's problem can be viewed as both a modeling problem and a control problem. Due to their fundamental significance, the SBP and its deterministic (zero-noise limit) counterpart of Optimal Mass Transport (OMT) have recently received interest within a broad spectrum of disciplines, including physics, stochastic control, computer science, probability theory, and geometry. Yet, while the mathematics and applications of the SBP/OMT have been developing at a considerable pace, accounting for marginals of unequal mass has received scant attention; the problem of interpolating between "unbalanced" marginals has been approached by introducing source/sink terms into the transport equations, in an ad hoc manner, chiefly driven by applications in image registration. Nevertheless, losses are inherent in many physical processes, and thereby models that account for lossy transport may also need to be reconciled with observed marginals following Schrödinger's dictum; that is, it is necessary to adjust the probability of trajectories of particles, including those that do not make it to the terminal observation point, so that the updated law represents the most likely way that particles may have been transported, or have vanished, at some intermediate point. Thus, the purpose of this work is to develop such a natural generalization of the SBP for stochastic evolution with losses, whereupon particles are "killed" (jump into a coffin/extinction state) according to a probabilistic law, and thereby mass is gradually lost along their stochastically driven flow. Through a suitable embedding we turn the problem into an SBP for stochastic processes that combine diffusive and jump characteristics. Then, following a large-deviations formalism in the style of Schrödinger, given a prior law that allows for losses, we ask for the most probable evolution of particles along with the most likely killing rate as the particles transition between the specified marginals. Our approach differs sharply from previous work involving a Feynman-Kac multiplicative reweighing of the reference measure: The latter, as we argue, is far from Schrödinger's quest. An iterative scheme, generalizing the celebrated Fortet-IPF-Sinkhorn algorithm, permits us to compute the new drift and the new killing rate of the path-space solution measure. We finally formulate and solve a related fluid-dynamic control problem for the flow of one-time marginals where both the drift and the new killing rate play the role of control variables. [ABSTRACT FROM AUTHOR]

In this paper, we study the maximum principle for stochastic optimal control problems of forward–backward stochastic difference systems (FBSΔSs). Two types of FBSΔSs are investigated. The first one is described by a partially coupled forward–backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. By adopting an appropriate representation of the product rule and an appropriate formulation of the adjoint process, we deduce the adjoint difference equation. Finally, the maximum principle for this optimal control problem with the control domain being convex is established. [ABSTRACT FROM AUTHOR]

We propose an online algorithm for solving a class of continuous-state Markov decision processes. The algorithm combines classical Q-learning with an asynchronous averaging procedure, which allows Q-function estimates at sampled state–action pairs to be adaptively updated based on observations collected along a single sample trajectory. These estimates are then used to iteratively construct an interpolation-based function approximator of the Q-function. We prove the convergence of the algorithm and provide numerical results to illustrate its performance. • Proposed a model-free Q-learning algorithm for solving a class of infinite horizon Markov decision processes with continuous state spaces. • Proved the strong convergence of the algorithm when used in conjunction with a class of non-linear function approximators. • Illustrated the performance of the proposed algorithm through simulation comparison studies. [ABSTRACT FROM AUTHOR]

The design of fast charging protocols is fundamental to improving the performance and lifetime of lithium-ion batteries. It is well-known that charging operations consistently performed at very high current will negatively impact operational safety and battery lifetime, although a quantitative understanding of these relationships remains lacking. The protocol design problem is typically formulated as a model-based dynamic optimization, where safety of operations can be encoded by constraining relevant battery states. However, all models are affected by uncertainty, which in turn propagates to state predictions. In this case, charging protocols based on nominal predictions may not satisfy the operating constraints. To overcome this issue, this work proposes a stochastic optimal control approach for the efficient computation of safe, fast charging protocols, able to explicitly account for parametric uncertainties affecting the battery model and guarantee probabilistically robust constraint satisfaction. Given a description of uncertainty affecting model parameters, linearized sensitivity analysis is exploited to propagate uncertainty to the battery states, and suitable backoff values for safety constraints are computed for each time instant. The effectiveness of the methodology is demonstrated in silico, by computing five different protocols, with a detailed Multiphase Porous Electrode Theory-based model of commercially available lithium-iron-phosphate batteries. • A stochastic optimal control method is proposed for design of safe, fast protocols. • The method explicitly accounts for uncertainties in model parameters. • Uncertainties are propagated to the battery states using sensitivity analysis. • Backoff values for safety constraints are satisfied at each time instance. • The methodology is demonstrated for lithium-ion phosphate batteries. [ABSTRACT FROM AUTHOR]

We consider an investor that trades continuously and wants to liquidate an initial asset position within a prescribed time interval. As a consequence of his trading activity, during the execution of the liquidation order, the investor has no guarantees that the placed order is executed immediately; it may go unfilled, partially filled or filled in excess. The uncertainty in the execution affects the trading activity of the investor and the asset share price dynamics generating additional sources of noise: the execution risk and the price impact risk, respectively. Assuming the two sources of noise correlated and driven by the cumulative effect of the investor trading strategy, we study the problem of finding the optimal liquidation strategy adopted by the investor in order to maximize the expected revenue resulting from the liquidation. The mathematical model of the liquidation problem presented here extends the model of Almgren and Chriss [Optimal execution of portfolio transactions. J. Risk, 2000, 3(2), 5–39] to include execution and price impact risks. The liquidation problem is modeled as a linear quadratic stochastic optimal control problem with the finite horizon and, under some assumptions about the functional form for the magnitude of execution and price impact risks, is solved explicitly. The derived solution coincides with the optimal trading strategy obtained in the absence of execution uncertainty for an asset price with a modified growth rate. This suggests that the uncertainty in the execution modifies the directional view of the investor about the future growth rate of the asset price. [ABSTRACT FROM AUTHOR]

Let X (t) be a one-dimensional controlled diffusion process, and Y (t) be a deterministic process. Various models are considered for (X ˙ (t) , Y ˙ (t)). The optimizer tries to bring the ratio X (t) / Y (t) to a certain value d as soon as possible. There is also a second optimizer whose aim is to prevent the ratio X (t) / Y (t) from reaching d as long as possible. Explicit solutions are obtained. The particular case when the control variables are assumed to be constants and the general case are treated. [ABSTRACT FROM AUTHOR]

This paper is to establish a sufficient maximum principle for one kind of stochastic optimal control problem with three types of delays: a discrete delay, a moving-average delay and a noisy memory. The main features of this research include the introduction of a unified adjoint equation and a simple method to get the adjoint process. One kind of optimal consumption problem and its special cases are studied as illustrative examples, for which the adjoint equations are solved with two different approaches and the optimal consumption strategies are obtained. [ABSTRACT FROM AUTHOR]