Search

Information drives trading and hence affects price dynamics. We study how an informed trader optimally trades, how multiple traders compete with each other, and how their competition affects price dynamics as well as their inventories in the linear price impact model framework. We formulate the informed trading problem as a stochastic optimal control problem and a stochastic differential game in different cases. By virtue of different methods, problems are solved explicitly in all cases. In the case of a single trader, the risk-averse informed trader trades less in the beginning and then speeds up to reduce exposure to the volatility of dynamic information. In addition, the risk-averse trader reveals information more slowly and thus contributes less to price efficiency. When multiple risk-neutral traders compete with information, regardless of the competition structure, they tend to trade more rapidly in the early stage to fill orders at a good price. Competition and the presence of a leader help promote price efficiency, and competition also leads to the empirically observed mean-reverting behavior of price dynamics and order flows. Furthermore, among risk-averse informed traders, inventory management triggers predatory trading and active information leakage during informed trading, and the positions of traders with the same level of risk aversion tend to be identical and mean-reverting to zero under multiple rounds of competition. [ABSTRACT FROM AUTHOR]

We present a two-country model (North and South) that describes the phenomenon of offshoring and reshoring. The model is a continuous time-controlled Markov chain with binary states. The main trade-off involves production costs and transaction costs between one country and another. In the first part of this paper, we identify the key parameters of the model: the difference in unit production costs between the two countries considered, the marginal cost of transitioning between countries, and the incentive paid by the North country to all companies that have not relocated at the end of the planning interval. The final goal of our paper is to understand how national tax incentives can influence this process. [ABSTRACT FROM AUTHOR]

In this paper we explore optimal liquidation in a market populated by a number of heterogeneous market makers that have limited inventory-carrying and risk-bearing capacity. We derive a reduced form model for the dynamics of their aggregated inventory considering a proper scaling limit. The resulting price impact profile is shown to depend on the characteristics and relative importance of their inventories. The model is flexible enough to reproduce the empirically documented power law behavior of the price impact function. For any choice of the market makers characteristics, optimal execution within this modeling approach can be recast as a linear-quadratic stochastic control problem. The value function and the associated optimal trading rate can be obtained semi-explicitly subject to solving a differential matrix Riccati equation. Numerical simulations are conducted to illustrate the performance of the resulting optimal liquidation strategy in relation to standard benchmarks. Remarkably, they show that the increase in performance is determined by a substantial reduction of higher order moment risk. [ABSTRACT FROM AUTHOR]

This paper is concerned with the turnpike property for a class of stochastic linear-quadratic (LQ, for short) optimal control problems with periodic coefficients. The stability and stabilizability of the control system are studied, followed by the discussion of the existence and uniqueness of periodic solutions. A deterministic periodic LQ problem is introduced and solved, whose optimal pair, together with a pair of correction processes, serves as the turnpike limit of the stochastic problem. It is shown that the turnpike limit is periodic in the distribution sense. In the special case of constant coefficients, the turnpike limit turns out to have stationary distributions, with the expectation being the solution to a static optimization problem. [ABSTRACT FROM AUTHOR]

This paper deals with finite‐horizon minimum‐variance and covariance steering problems subject to constraints. The goal of the minimum variance problem is to steer the state mean of an uncertain system to a prescribed vector while minimizing the trace of its terminal state covariance whereas the goal in the covariance steering problem is to steer the covariance matrix of the terminal state to a prescribed positive definite matrix. The paper proposes a solution approach that relies on a stochastic version of the affine disturbance feedback control parametrization. In this control policy parametrization, the control input at each stage is expressed as an affine function of the history of disturbances that have acted upon the system. It is shown that this particular parametrization reduces the stochastic optimal control problems considered in this paper into tractable convex programs or difference of convex functions programs with essentially the same decision variables. In addition, the paper proposes a variation of this control parametrization that relies on truncated histories of past disturbances, which allows for sub‐optimal controllers to be designed that strike a balance between performance and computational cost. The suboptimality of the truncated policies is formally analyzed and closed form expressions are provided for the performance loss due to the use of the truncation scheme. Finally, the paper concludes with a comparative analysis of the truncated versions of the proposed policy parametrization and other standard policy parametrizations through numerical simulations. [ABSTRACT FROM AUTHOR]

We consider optimal control problems for a system governed by a stochastic differential equation driven by a d-dimensional Brownian motion where both the drift and the diffusion coefficient are controlled. It is well known that without additional convexity conditions the strict control problem does not admit an optimal control. To overcome this difficulty, we consider the relaxed model, in which admissible controls are measure-valued processes and the relaxed state process is governed by a stochastic differential equation driven by a continuous orthogonal martingale measure. This relaxed model admits an optimal control that can be approximated by a sequence of strict controls by the so-called chattering lemma. We establish optimality necessary conditions, in terms of two adjoint processes, extending Peng's maximum principle to relaxed control problems. We show that relaxing the drift and diffusion martingale parts directly as in deterministic control does not lead to a true relaxed model as the obtained controlled dynamics is not continuous in the control variable. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

Sampling rare events in metastable dynamical systems is often a computationally expensive task and one needs to resort to enhanced sampling methods such as importance sampling. Since we can formulate the problem of finding optimal importance sampling controls as a stochastic optimization problem, this then brings additional numerical challenges and the convergence of corresponding algorithms might suffer from metastabilty. In this article, we address this issue by combining systematic control approaches with the heuristic adaptive metadynamics method. Crucially, we approximate the importance sampling control by a neural network, which makes the algorithm in principle feasible for high-dimensional applications. We can numerically demonstrate in relevant metastable problems that our algorithm is more effective than previous attempts and that only the combination of the two approaches leads to a satisfying convergence and therefore to an efficient sampling in certain metastable settings. [ABSTRACT FROM AUTHOR]

We study the optimal consumption, investment and lifeinsurance purchase and selection strategies for a wage-earner with an uncertain lifetime with access to a financial market comprised of one risk-free security and one risky-asset whose prices evolve according to linear diffusions modulated by a continuous-time stochastic process determined by an additional diffusive nonlinear stochastic differential equation. The process modulating the linear diffusions may be regarded as an indicator describing the state of the economy in a given instant of time. Additionally, we allow the Brownian motions driving each of these equations to be correlated. The lifeinsurance market under consideration herein consists of a fixed number of providers offering pairwise distinct contracts. We use dynamic programming techniques to characterize the solutions to the problem described above for a general family of utility functions, studying the case of discounted constant relative risk aversion utilities with more detail. [ABSTRACT FROM AUTHOR]

This paper is concerned with an optimal control problem for a mean-field linear stochastic differential equation with a quadratic functional in the infinite time horizon. Under suitable conditions, including the stabilizability, the (strong) exponential, integral, and mean-square turnpike properties for the optimal pair are established. The keys are to correctly formulate the corresponding static optimization problem and find the equations determining the correction processes. These have revealed the main feature of the stochastic problems which are significantly different from the deterministic version of the theory. [ABSTRACT FROM AUTHOR]

We consider a class of stochastic optimal control problems with partial observation, and study their approximation by discrete-time control problems. We establish a convergence result by using the weak convergence technique of Kushner and Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York], together with the notion of relaxed control rule introduced by El Karoui, Huú Nguyen and Jeanblanc-Picqué [SIAM J. Control Optim., 26 (1988), pp. 1025--1061]. In particular, with a well chosen discrete-time control system, we obtain a first implementable numerical algorithm (with convergence) for the partially observed control problem. Moreover, our discrete-time approximation result would open the door to study convergence of more general numerical approximation methods, such as machine learning based methods. Finally, we illustrate our convergence result by numerical experiments on a partially observed control problem in a linear quadratic setting. [ABSTRACT FROM AUTHOR]

This paper focuses on the application of meta-reinforcement learning to the robust design of low-thrust interplanetary trajectories in the presence of multiple uncertainties. A closed-loop control policy is used to optimally steer the spacecraft to a final target state despite the considered perturbations. The control policy is approximated by a deep recurrent neural network, trained by policy-gradient reinforcement learning on a collection of environments featuring mixed sources of uncertainty, namely dynamic uncertainty and control execution errors. The recurrent network is able to build an internal representation of the distribution of environments, thus better adapting the control to the different stochastic scenarios. The results in terms of optimality, constraint handling, and robustness on a fuel-optimal low-thrust transfer between Earth and Mars are compared with those obtained via a traditional reinforcement learning approach based on a feed-forward neural network. • In interplanetary space missions, the spacecraft trajectory is affected by multiple uncertainties. • Meta-reinforcement learning can be applied seamlessly to any uncertainty and dynamic model. • A recurrent neural network is used as a history-dependent closed-loop control policy. • The network is trained on a low-thrust transfer featuring dynamic uncertainties and control execution errors. • Meta-reinforcement learning shows improved performance and robustness compared to standard reinforcement learning. [ABSTRACT FROM AUTHOR]

This paper focuses on a stochastic goodwill model involving a single manufacturer and a single retailer in a supply chain, in which both quality level and goodwill level are governed by stochastic differential equations. In both the linear-quadratic Stackelberg game and linear-quadratic cooperative game, explicit expressions are obtained for the optimal quality improvement efforts level of the manufacturer, the optimal advertising efforts level of the retailer as well as the optimal profit of the supply chain. We also derive the expectation and variance values for quality and goodwill levels, respectively. The distribution of incremental profit and subsidy are analysed in the cooperative game via the utility theory. Finally, several numerical experiments are reported to show the influence of decay rates and white noise disturbance of the quality and goodwill levels on the expectation of quality and goodwill levels. [ABSTRACT FROM AUTHOR]

The optimal control of non-instantaneous impulsive second-order stochastic McKean–Vlasov evolution system with Clarke subdifferential and mixed fractional Brownian motion is investigated in this article. The deterministic nonlinear second-order controlled partial differential system is enriched with stochastic perturbations, non-instantaneous impulses, and Clarke subdifferential. In particular, the nonlinearities in the system that rely on the state of the solution are allowed to rely on the corresponding probability distribution of the state. The solvability of the considered system is discussed with the help of stochastic analysis, multivalued analysis, and multivalued fixed point theorem. Further, the existence of optimal control is established with the aid of Balder's theorem. Finally, an example is provided to illustrate the developed theory. [ABSTRACT FROM AUTHOR]

We consider a continuous lifetime model for investor whose lifetime is a random variable. We assume the investor has an access to the social welfare system, the financial market and the life insurance market. The investor aims to find the optimal strategies that maximize the expected utility obtained from consumption, investing in the financial market, buying life insurance, registering in the social welfare system, the size of his estate in the event of premature death and the size of his fortune at time of retirement if he lives that long. We use dynamic programming techniques to derive a second-order nonlinear partial differential equation whose solution is the maximum objective function. We use special case of discounted constant relative risk aversion utilities to find an explicit solutions for the optimal strategies. Finally, we have shown a numerical solution for the problem under consideration and study some properties for the optimal strategies. [ABSTRACT FROM AUTHOR]

To overcome the time-consuming training caused by the sparse reward function in reinforcement learning, an efficient dense reward framework for robust low-thrust trajectory optimization is proposed. The dense reward functions are designed separately for the deterministic and considered uncertain scenarios, including state uncertainties, observation uncertainties and execution uncertainties. For the uncertainties, the dense reward function is designed to diminish the deviation with respect to the isochronous nominal state along the corresponding deterministic optimal trajectory at each step, rendering the reward function no longer sparse and suitable for complex multirevolution problems. In addition, a multistage reward function of the terminal constraints for the rendezvous missions is designed by incorporating some exponential acceleration terms, enabling significant improvement in training efficiency as the terminal errors become low. In addition, a dense reward function for the deterministic scenario is also proposed via the introduction of empirical forbidden zones and an exponential term. The effectiveness and efficiency of the proposed method is demonstrated in a simple Earth-Mars mission and a complex Earth-Venus multirevolution mission. The promising results verify the significant effect of the proposed method in speeding up the process of training an initial incapable agent to an 'expert' while guaranteeing or even improving the performance. [ABSTRACT FROM AUTHOR]

In this paper, we discuss an optimal dividend and reinsurance problem for an insurance company facing two types of risks: unstable income and potential loses. The arrival of all loses is characterized as a compound Poisson process. We assumes that every possible loss can be reinsured for a part of it. The reserve is a combination of a diffusion process and a controllable compound Poisson process. We investigate the optimal dividend and reinsurance strategy by analyzing the corresponding variational inequality on the value function. A significant difference from the existing literature is that the HJB equation in this variational inequality is a partial integro-differential equation with a functional optimization problem appearing in the integral operator. We not only prove the existence of a classical solution to the problem and the continuity, strict monotonicity, boundedness of the dividend free boundary, but also discuss the properties of the optimal reinsurance policy, including the continuity, monotonicity of the optimal part covered by reinsurance for each possible loss, and the smoothness of the reinsurance free boundary. [ABSTRACT FROM AUTHOR]

We develop an approach for two-player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different types of constraints, which lead us to find real-valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with the stochastic maximum principle technique. Our application is an example of cooperation between a bank and an insurance company, which is a popular, well-known business agreement type called Bancassurance. [ABSTRACT FROM AUTHOR]

The Pontryagin type maximum principle and Bellman's dynamic programming principle serve as two of the most important tools in solving optimal control problems. There is a huge literature on the study of relationship between them. The main purpose of this paper is to investigate the relationship between the Pontryagin type maximum principle and the dynamic programming principle for control systems governed by stochastic evolution equations in infinite dimensional space, with the control variables entering into both the drift and the diffusion terms. To do so, we first prove the dynamic programming principle for those systems without employing the martingale solutions. Then we establish the desired relationships in both cases that value function associated is smooth or nonsmooth. For the nonsmooth case, in particular, by employing the relaxed transposition solution, we discover the connection between the superdifferentials and subdifferentials of value function and the first-order and second-order adjoint equations. [ABSTRACT FROM AUTHOR]

This study applies the theory of stochastic control to an environmental project planning to counteract against the sediment starvation problem in river environments. This can be considered as a time‐average inventory problem to time‐discretely control a continuous‐time system driven by a non‐smooth jump process under habit formation disturbing project implementation. The system is modeled such that the sediment storage dynamics are physically consistent with certain experimental results. Further, the habit formation is modeled as simple linear dynamics and serves as a constraint related to the replenishment amount of the sediment. We show that the time‐average control problem is not necessarily ergodic. Consequently, the effective Hamiltonian may become a non‐constant. Thereafter, ratcheting cases as extreme cases of the irreversible habit formation are considered, owing to them being unique exactly solvable non‐ergodic control problems. The optimality equation associated with a regularized and hence well‐defined control problem is verified. Furthermore, a finite difference scheme is examined against the exactly solvable case and then applied to more complicated cases. [ABSTRACT FROM AUTHOR]

This paper studies the numerical algorithm of stochastic control problems in investment optimization. Investors choose the optimal investment to maximize the expected return under uncertainty. The optimality condition, the Hamilton–Jacobi–Bellman (HJB) equation, satisfied by the value function and obtained by the dynamic programming method, is a partial differential equation coupled with optimization. One of the major computational difficulties is the irregular boundary conditions presented in the HJB equation. In this paper, two mesh-free algorithms are proposed to solve two different cases of HJB equations with regular and irregular boundary conditions. The model of optimal investment under uncertainty developed by Abel is used to study the efficacy of the proposed algorithms. Extensive numerical studies are conducted to test the impact of the key parameters on the numerical efficacy. By comparing the numerical solution with the exact solution, the proposed numerical algorithms are validated. [ABSTRACT FROM AUTHOR]

This paper investigates the optimal private health insurance contract design problem, considering the joint interests of a policyholder and an insurer. Both the policyholder and the insurer jointly determine the premium of private health insurance. In order to better reflect reality, the illness expenditure is modelled by an extended compound Poisson process depending on health status. Under the mean–variance criterion and by applying dynamic programming, control theory, and leader–follower game techniques, analytically time-consistent private health insurance strategies are derived, optimal private health insurance contracts are designed, and their implications toward insurance are analysed. Finally, we perform numerical experiments assuming that the policyholder and the insurer calculate their wealth every year and they deposit their disposable income into the Bank of China with the interest rate being r = 0.021 . The values of other model parameters are set by referring to the data in the related literature. We find that the worse the policyholder's health, the higher the premium that they pay for private health insurance, and buying private health insurance can effectively reduce the policyholder's economic losses caused by illnesses. [ABSTRACT FROM AUTHOR]

In this paper, we solve an optimal reinsurance problem in the mathematical finance area. We assume that the surplus process of the insurance company follows a controlled diffusion process and the constant interest rate is involved in the financial model. During the whole optimization period, the company has a choice to buy reinsurance contract and decide the reinsurance retention level. Meanwhile, the bankruptcy at the terminal time is not allowed. The aim of the optimization problem is to minimize the distance between the terminal wealth and a given goal by controlling the reinsurance proportion. Using the stochastic control theory, we derive the Hamilton-Jacobi-Bellman equation for the optimization problem. Via adopting the technique of changing variable as well as the dual transformation, an explicit solution of the value function and the optimal policy are shown. Finally, several numerical examples are shown, from which we find several main factors that affect the optimal reinsurance policy. [ABSTRACT FROM AUTHOR]

The optimization and control of spacecraft trajectories in presence of uncertainties and unmodeled disturbances represents a challenging problem. Future missions would benefit from robust control techniques to compute their trajectory in a complex, nonlinear environment and in presence of stochasticity. In this work the Stochastic Optimal Control problem in the Circular Restricted Three Body Problem is addressed and a method is proposed based on Nonlinear Model Predictive control (NMPC). This method presents the challenge of solving a nonlinear optimization problem on-board. A Quantum-inspired meta-heuristic optimization algorithm is tested in this context: the Diffusion Monte Carlo (DMC) method is based on a numerical techniques used in Quantum Mechanics, adapted to solve optimization problems. The NMPC with DMC is implemented, taking into account an estimate of the computational time required through the introduction of delayed actuation of the control. Two applications are presented featuring transfer trajectories between planar periodic orbits in the vicinity of the Moon; the proposed method proved able to successfully generate trajectories when disturbance accelerations and uncertain initial conditions are present. • The Circular Restricted Three Body Problem with uncertainties is considered. • A NMPC is used to drive the spacecraft to the desired orbit despite the uncertainty. • A Quantum-inspired meta-heuristic optimizer is used in the MPC. • Applications are presented regarding transfers between Lunar periodic orbits. [ABSTRACT FROM AUTHOR]

To address the issue of inaccurate distributions in discrete-time stochastic systems, a minimax linear quadratic control method using theWasserstein metric is proposed. Our method aims to construct a control policy that is robust against errors in an empirical distribution of underlying uncertainty by adopting an adversary that selects the worst-case distribution at each time. The opponent receives a Wasserstein penalty proportional to the amount of deviation from the empirical distribution. As a tractable solution, a closed-form expression of the optimal policy pair is derived using a Riccati equation. We identify nontrivial stabilizability and observability conditions under which the Riccati recursion converges to the unique positive semidefinite solution of an algebraic Riccati equation. Our method is shown to possess several salient features, including closed-loop stability, a guaranteed-cost property, and a probabilistic out-of-sample performance guarantee. [ABSTRACT FROM AUTHOR]

A stochastic epidemic model with random noise transmission is taken into account, describing the dynamics of the measles viral infection. The basic reproductive number is calculated corresponding to the stochastic model. It is determined that, given initial positive data, the model has bounded, unique, and positive solution. Additionally, utilizing stochastic Lyapunov functional theory, we study the extinction of the disease. Stationary distribution and extinction of the infection are examined by providing sufficient conditions. We employed optimal control principles and examined stochastic control systems to regulate the transmission of the virus using environmental factors. Graphical representations have been offered for simplicity of comprehending in order to further verify the acquired analytical findings. [ABSTRACT FROM AUTHOR]

We study singular perturbations of a class of two-scale stochastic control systems with unbounded data. The assumptions are designed to cover some relaxation problems for deep neural networks. We construct effective Hamiltonian and initial data and prove the convergence of the value function to the solution of a limit (effective) Cauchy problem for a parabolic equation of HJB type. We use methods of probability, viscosity solutions and homogenization. [ABSTRACT FROM AUTHOR]

We study an exit contract design problem, where one provides a universal exit contract to multiple heterogeneous agents, with which each agent chooses an optimal (exit) stopping time. The problem consists in optimizing the universal exit contract w.r.t. some criterion depending on the contract as well as the agents' exit times. Under a technical monotonicity condition, and by using Bank-El Karoui's representation of stochastic processes, we are able to transform the initial contract optimization problem into an optimal control problem. The latter is also equivalent to an optimal multiple stopping problem and the existence of the optimal contract is proved. We next show that the problem in the continuous-time setting can be approximated by a sequence of discrete-time ones, which would induce a natural numerical approximation method. We finally discuss the optimization problem over the class of all Markovian and/or continuous exit contracts. [ABSTRACT FROM AUTHOR]

This paper considers an optimal control problem for a time-invariant linear stochastic system with discrete time, scalar unbounded control, additive noise, and a probabilistic criterion for retaining its trajectories in a given neighborhood of zero. We use dynamic programming and two-sided Bellman function estimates to derive analytical expressions for the optimal control at two time steps and a suboptimal control on any control horizon. The effectiveness of these controls is illustrated on a numerical example. [ABSTRACT FROM AUTHOR]

We consider the diffusive limit of a typical pure-jump Markovian control problem as the intensity of the driving Poisson process tends to infinity. We show that the convergence speed is provided by the Hölder exponent of the Hessian of the limit problem, and explain how correction terms can be constructed. This provides an alternative efficient method for the numerical approximation of the optimal control of a pure-jump problem in situations with very high intensity of jumps. We illustrate this approach in the context of a display advertising auction problem. [ABSTRACT FROM AUTHOR]

This paper is concerned with a class of mean-feld type coupled forward-backward stochastic diferential equations (MF-FBSDEs), in which the coupling appears in integral terms, terminal terms, and initial terms. Inspired by various mean-feld type linear-quadratic (MF-LQ) optimal control problems, we propose a type of randomized domination-monotonicity conditions, under which and the usual Lipschitz condition we obtain a well-posedness result on MF-FBSDEs in the sense of square integrability including the unique solvability, an estimate of the solution, and the related continuous dependence property of the solution on the coefcients. The result of MF- FBSDEs in turn extends MF-LQ problems in the literature to a general situation where the initial states or the terminal states are also controlled at the same time, and gives explicit expressions of the related unique optimal controls. [ABSTRACT FROM AUTHOR]

This paper is devoted to the stochastic optimal control problem of ordinary differential equations allowing for both path-dependence and measurable randomness. As opposed to the deterministic path-dependent cases, the value function turns out to be a random field on the path space and it is characterized by a stochastic path-dependent Hamilton–Jacobi (SPHJ) equation. A notion of viscosity solution is proposed and the value function is proved to be the unique viscosity solution to the associated SPHJ equation. [ABSTRACT FROM AUTHOR]

In this paper, we consider the optimal reinsurance problem for a risk model with a thinning-dependence structure, where the stochastic sources related to claim occurrence are classified into different groups, and each group may cause a claim in each insurance class with some probability. We assume that the insurer can manage the risk by purchasing per-loss reinsurance, and their aim is to maximize the expected utility of the terminal wealth. By using the technique of stochastic control, we obtain the corresponding Hamilton–Jaccobi–Bellman equation. From the perspective of game theory, we derive the closed-form expression of the optimal strategy for each class of business, which is actually the best response to other given strategies. We also investigate the necessary conditions for optimal strategies and transfer the original optimization problem into a system of equations. Furthermore, we prove that the solution of the system of equations always exists, but may not be unique, and we also study some features of the optimal strategies in special cases and derive several interesting results. Finally, some numerical examples are given to show the impacts of some important parameters on the optimal strategies. [ABSTRACT FROM AUTHOR]

Numerous applied models used in the study of optimal control problems, inverse problems, shape optimization, machine learning, fractional programming, neural networks, image registration and so on lead to stochastic optimization problems in Hilbert spaces. Under a suitable convexity assumption on the objective function, a necessary and sufficient optimality condition for stochastic optimization problems is a stochastic variational inequality. This article presents a new stochastic regularized second-order iterative scheme for solving a variational inequality in a stochastic environment where the primary operator is accessed by employing sampling techniques. The proposed iterative scheme, which fits within the general framework of the stochastic approximation approach, has its almost-sure convergence analysis given in a Hilbert space. We test the feasibility and the efficacy of the proposed stochastic approximation approach for a stochastic optimal control problem and a stochastic inverse problem, both associated with a second-order stochastic partial differential equation. This article is part of the theme issue 'Non-smooth variational problems and applications'. [ABSTRACT FROM AUTHOR]

We present a pair of adjoint optimal control problems characterizing a class of time-symmetric stochastic processes defined on random time intervals. The associated PDEs are of free-boundary type. The particularity of our approach is that it involves two adjoint optimal stopping times adapted to a pair of filtrations, the traditional increasing one and another, decreasing. They are the keys of the time symmetry of the construction, which can be regarded as a generalization of 'Schrödinger's problem' (1931–1932) to space-time domains. The relation with the notion of 'Hidden diffusions' is also described. [ABSTRACT FROM AUTHOR]

In this work, we provide discrete optimality conditions of the optimal control problems of stochastic differential equations. Euler and Runge–Kutta methods are used for discretization. A Lagrange multiplier method for a discrete-time stochastic optimal control problem is formulated. The discrete adjoint process p n is obtained in terms of conditional expectations E [ p n + 1 ] and E [ p n + 1 Δ W ] for both methods. To estimate these nested conditional expectations at each time step via simulation, we use a very powerful new approach, least-squares Monte-Carlo method, developed by Longstaff–Schwartz. This is the first time to solve a stochastic optimal control problem by calculating the nested conditional expectations numerically with the help of a least-squares Monte-Carlo method. Some examples are studied to test and demonstrate the efficiency of the Lagrange multiplier combined with the least-squares Monte-Carlo method. [ABSTRACT FROM AUTHOR]

A controlled discrete-time stochastic process is proposed as a model for the state of a machine. In the proposed model, normal and random wear of the machine are considered. The random wear of the machine during a time unit can be either a discrete or a continuous random variable. The objective is to find a control policy that maximises the profits generated by the machine over its useful lifetime. In this problem, the optimiser must decide whether to do or not to do the maintenance work on the machine at each time unit. The significance of the paper is that the final time in the optimal control problem is a random variable denoting the first time the machine must be replaced. To obtain the optimal control, one can try to solve the dynamic programming equation, which reduces to a difference or an integral equation, satisfied by the value function. Finally, particular cases are solved exactly, or approximately, and explicitly to demonstrate and validate the proposed model. [ABSTRACT FROM AUTHOR]

This paper presents a computation-efficient stochastic dynamic programming algorithm for solving energy storage price arbitrage considering variable charge and discharge efficiencies. We formulate the price arbitrage problem using stochastic dynamic programming and model real-time prices as a Markov process. Then we propose an analytical solution algorithm using a piecewise linear approximation of the value-to-go function. Our solution algorithm achieves extreme computation performance and solves the proposed arbitrage problem for one operating day in less than one second on a personal computer. We demonstrate our approach using historical price data from four price zones in New York Independent System Operator, with case studies comparing the performance of different stochastic models and storage settings. Our results show that the proposed method captures 50% to 90% of arbitrage profit compared to perfect price forecasts. In particular, our method captures more than 80% of arbitrage profit in three out of the four price zones when considering batteries with more than two-hour duration and realistic degradation cost. [ABSTRACT FROM AUTHOR]

This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time-horizon [0, T] as T → ∞. The so-called turnpike properties are established for such problems, under stabilizability condition which is weaker than the controllability, normally imposed in the similar problem for ordinary differential systems. In dealing with the turnpike problem, a crucial issue is to determine the corresponding static optimization problem. Intuitively mimicking the deterministic situations, it seems to be natural to include both the drift and the diffusion expressions of the state equation to be zero as constraints in the static optimization problem. However, this would lead us to a wrong direction. It is found that the correct static problem should contain the diffusion as a part of the objective function, which reveals a deep feature of the stochastic turnpike problem. [ABSTRACT FROM AUTHOR]

In this paper, we study an optimal investment and reinsurance problem in which the interest rate is driven by the Vasicek process and two dependent classes of insurance business are correlated through a common shock component. The goal of insurer is to find an optimal investment-reinsurance strategy to minimize the variance of terminal net wealth for a given expected terminal net wealth. By using the linear-quadratic optimal control theory and the corresponding Hamilton-Jacobi-Bellman (HJB) equation, the closed-form expressions for the value function and optimal strategies are obtained under the special case of perfect correlation between the bond and stock processes. We present that the solution of the HJB equation is no longer a classical solution, but a viscosity solution due to the non-negativity constraint of the reinsurance strategy. Furthermore, the efficient strategies and efficient frontier are derived explicitly. Finally, we explore some numerical examples to show the influence of model parameters on the optimal investment and reinsurance strategies. [ABSTRACT FROM AUTHOR]

This article studies the formation mission design problem for commercial aircraft in the presence of uncertainties. Specifically, it considers uncertainties in the departure times of the aircraft and in the fuel burn savings for the trailing aircraft. Given several commercial flights, the problem consists in arranging them in formation or solo flights and finding the trajectories that minimize the expected value of the DOC of the flights. The formation mission design problem is formulated as an optimal control problem of a stochastic switched dynamical system and solved using nonintrusive gPC-based stochastic collocation. The stochastic collocation method converts the SSOCP into an augmented deterministic switched optimal control problem. With this approach, a small number of sample points of the random parameters are used to jointly solve particular instances of the switched optimal control problem. The obtained solutions are then expressed as orthogonal polynomial expansions in terms of the random parameters using these sample points. This technique allows statistical and global sensitivity analysis of the stochastic solutions to be conducted at a low computational cost. The aim of this article is to establish if, in the presence of uncertainties, a formation mission is beneficial with respect to solo flight in terms of the expected value of the direct operating costs. Several numerical experiments have been conducted in which uncertainties on the departure times and on the fuel saving during formation flight have been considered. The obtained results demonstrate that benefits can be achieved even in the presence of these uncertainties. [ABSTRACT FROM AUTHOR]

We develop a new iterative method based on the Pontryagin principle to solve stochastic control problems. This method is nothing other than the Newton method extended to the framework of stochastic optimal control, where the state dynamics are given by an ODEs with stochastic coefficients and the cost is random. Each iteration of the method is made of two ingredients: computing the Newton direction, and finding an adapted step length. The Newton direction is obtained by solving an affine-linear Forward-Backward Stochastic Differential Equation (FBSDE) with random coefficients. This is done in the setting of a general filtration. Solving such an FBSDE reduces to solving a Riccati Backward Stochastic Differential Equation (BSDE) and an affine-linear BSDE, as expected in the framework of linear FBSDEs or Linear-Quadratic stochastic control problems. We then establish convergence results for this Newton method. In particular, Lipschitz-continuity of the second-order derivative of the cost functional is established with an appropriate choice of norm and under boundedness assumptions, which is sufficient to prove (local) quadratic convergence of the method in the space of uniformly bounded processes. To choose an appropriate step length while fitting our choice of space of processes, an adapted Backtracking line search method is developed. We then prove global convergence of the Newton method with the proposed line search procedure, which occurs at a quadratic rate after finitely many iterations. An implementation with regression techniques to solve BSDEs arising in the computation of the Newton step is developed. We apply it to the control problem of a large number of batteries providing ancillary services to an electricity network. [ABSTRACT FROM AUTHOR]

The aim of the paper is to apply the stochastic optimal control problem in order to optimize the number of individual which will have the pre-exposure prophylaxis (PReP) treatment in the stochastic model for HIV/AIDS with PReP. By using the stochastic maximum principle, we derive the stochastic optimal control of PReP for the unconstrained control problem. Furthermore, by combining the stochastic maximum principle with a version of the Lagrange multiplier method, we solve the PReP problem for two different types of budget constrains with a given constrain for the costs (possible of different kind, transportation, price of the treatment, etc.). Obtained results for the different percentage of the individuals who got the vaccine, as well as results for unconstrained and constrained problems, are illustrated by a numerical example. [ABSTRACT FROM AUTHOR]

The aim of this paper is to derive a maximum principle for a control problem governed by a stochastic partial differential equation (SPDE) with locally monotone coefficients. To reach our goal we adapt the method which uses the relation between backward stochastic partial differential equation (BSPDE) and the maximum principle. In particular, necessary conditions for optimality for this stochastic optimal control problem are obtained. In spite of the fact that the method used here was used by several authors before, our adaptation is not immediate. It applies a trick which is used to get estimates for solutions of SPDE with Locally Monotone Coefficients as in the proof of the Lemmas 5.1 and 5.3. This adaptation permits us to apply our results to get a maximum principle for the optimal control to the cases when the system is governed by the 2D stochastic Navier-Stokes equation and by a stochastic Burgers' equation. [ABSTRACT FROM AUTHOR]

We describe the growth dynamics of a stock using stochastic differential equations with a generalized logistic growth model which encompasses several well-known growth functions as special cases. For each model, we compute the optimal variable effort policy and compare the expected net present value of the total profit earned by the harvester among policies. In addition, we further extend the study to include parameters sensitivity, such as the costs and volatility, and present an explicitly Crank–Nicolson discretization scheme necessary to obtain optimal policies. [ABSTRACT FROM AUTHOR]

We consider a stochastic optimal control problem with state variable dynamics described by a stochastic differential equation of diffusive type modulated by a semi-Markov process with a finite state space. The time horizon is both deterministic and finite. Within such setup, we provide a detailed proof of the dynamic programming principle and use it to characterize the value function as a viscosity solution of the corresponding Hamilton-Jacobi-Bellman equation. We illustrate our results with an application to Mathematical Finance: the generalization of Merton's optimal consumption-investment problem to financial markets with semi-Markov switching. [ABSTRACT FROM AUTHOR]

Stochastic flows of an advective-diffusive nature are ubiquitous in biology and the physical sciences. Of particular interest is the problem of reconciling observed marginal distributions with a given prior posed by Schrödinger in 1932 and known as the Schrödinger Bridge Problem (SBP). It turns out that Schrödinger's problem can be viewed as both a modeling problem and a control problem. Due to their fundamental significance, the SBP and its deterministic (zero-noise limit) counterpart of Optimal Mass Transport (OMT) have recently received interest within a broad spectrum of disciplines, including physics, stochastic control, computer science, probability theory, and geometry. Yet, while the mathematics and applications of the SBP/OMT have been developing at a considerable pace, accounting for marginals of unequal mass has received scant attention; the problem of interpolating between "unbalanced" marginals has been approached by introducing source/sink terms into the transport equations, in an ad hoc manner, chiefly driven by applications in image registration. Nevertheless, losses are inherent in many physical processes, and thereby models that account for lossy transport may also need to be reconciled with observed marginals following Schrödinger's dictum; that is, it is necessary to adjust the probability of trajectories of particles, including those that do not make it to the terminal observation point, so that the updated law represents the most likely way that particles may have been transported, or have vanished, at some intermediate point. Thus, the purpose of this work is to develop such a natural generalization of the SBP for stochastic evolution with losses, whereupon particles are "killed" (jump into a coffin/extinction state) according to a probabilistic law, and thereby mass is gradually lost along their stochastically driven flow. Through a suitable embedding we turn the problem into an SBP for stochastic processes that combine diffusive and jump characteristics. Then, following a large-deviations formalism in the style of Schrödinger, given a prior law that allows for losses, we ask for the most probable evolution of particles along with the most likely killing rate as the particles transition between the specified marginals. Our approach differs sharply from previous work involving a Feynman-Kac multiplicative reweighing of the reference measure: The latter, as we argue, is far from Schrödinger's quest. An iterative scheme, generalizing the celebrated Fortet-IPF-Sinkhorn algorithm, permits us to compute the new drift and the new killing rate of the path-space solution measure. We finally formulate and solve a related fluid-dynamic control problem for the flow of one-time marginals where both the drift and the new killing rate play the role of control variables. [ABSTRACT FROM AUTHOR]

In this paper, we study the maximum principle for stochastic optimal control problems of forward–backward stochastic difference systems (FBSΔSs). Two types of FBSΔSs are investigated. The first one is described by a partially coupled forward–backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. By adopting an appropriate representation of the product rule and an appropriate formulation of the adjoint process, we deduce the adjoint difference equation. Finally, the maximum principle for this optimal control problem with the control domain being convex is established. [ABSTRACT FROM AUTHOR]

The paper models the maintenance of ecological networks in forest environments, built from bioreserves, patches and corridors, when these grids are subject to random processes such as extreme natural events. It also outlines a management plan to support the optimized results. After presenting the random graph-theoretic framework, we apply the stochastic optimal control to the graph dynamics. Our results show that the preservation of the network architecture cannot be achieved, under stochastic control, over the entire duration. It can only be accomplished, at the cost of sacrificing the links between the patches, by increasing the usage of the control devices. This would have a negative effect on the species migration by causing congestion among the channels left at their disposal. The optimal scenario, in which the shadow price is at its lowest and all connections are well-preserved, occurs at half of the course, be it the only optimal stopping moment found on the stochastic optimal trajectories. In such a scenario, the optimal forestry management policy has to integrate agility, integrated response, and quicker response time. [ABSTRACT FROM AUTHOR]