Search

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the entropic gradient descent in the optimization of deep neural networks via homogenization. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton--Jacobi--Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modeled as dynamic controls. [ABSTRACT FROM AUTHOR]

We consider optimal control problems for a system governed by a stochastic differential equation driven by a d-dimensional Brownian motion where both the drift and the diffusion coefficient are controlled. It is well known that without additional convexity conditions the strict control problem does not admit an optimal control. To overcome this difficulty, we consider the relaxed model, in which admissible controls are measure-valued processes and the relaxed state process is governed by a stochastic differential equation driven by a continuous orthogonal martingale measure. This relaxed model admits an optimal control that can be approximated by a sequence of strict controls by the so-called chattering lemma. We establish optimality necessary conditions, in terms of two adjoint processes, extending Peng's maximum principle to relaxed control problems. We show that relaxing the drift and diffusion martingale parts directly as in deterministic control does not lead to a true relaxed model as the obtained controlled dynamics is not continuous in the control variable. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

Summary: This paper deals with the second‐order necessary optimality conditions for discrete‐time stochastic optimal control problems under weakened convexity assumptions. Using a special variation of the control, and by virtue of a new discrete‐time backward stochastic equation, we establish a more general and constructive first‐order necessary optimality condition in the form of a global stochastic maximum principle. Moreover, by introducing a new discrete‐time backward stochastic matrix equation, the second‐order multipoint necessary optimality conditions of singular controls are derived, which covers and improves the classical second‐order necessary optimality conditions of discrete‐time stochastic systems. [ABSTRACT FROM AUTHOR]

In this paper, the stochastic control system under partial observation for singular optimal controls in the classical sense is studied. Note that both the system and the observation contain noise, in which the control variable is allowed to enter into all the coefficient terms. By adding some regularity assumptions, the pointwise second-order maximum principle under partial observation can be obtained by Malliavin calculus. Eventually, an example is given to demonstrate how to find the optimal control. [ABSTRACT FROM AUTHOR]

In this paper, we consider a joint dynamic pricing and production policy for a stochastic inventory system with perishable products. The demand is stochastic and dependent on the price and the level of the on-hand inventory. Combined dynamic pricing and production control, a stochastic dynamic optimization model that maximizes the total discounted profit is built. Applying the stochastic optimal control theory, we formulate the problem of finding the optimal joint dynamic pricing and production schedule as the problem of solving a Hamilton-Jacobi-Bellman (HJB) equation. By solving the HJB equation, analytical solutions for optimal pricing and production rate are obtained. In addition, a numerical example is given to illustrate the validness of the theoretical results and sensitivity analysis on major system parameters is carried out. [ABSTRACT FROM AUTHOR]

We study a stochastic control problem of mean‐field controlled stochastic differential systems driven by a fractional Brownian motion with Hurst parameter H∈(1/2,1)$$ H\in \left(1/2,1\right) $$. As a necessary condition of the optimal control we obtain a stochastic maximum principle. The associated adjoint mean‐field backward stochastic differential equation driven by a fractional Brownian motion and a classical Brownian motion. Applying the stochastic maximum principle to a mean‐field stochastic linear quadratic problem, we obtain the optimal control and prove that the necessary condition for the optimality of an admissible control is also sufficient under certain assumptions. [ABSTRACT FROM AUTHOR]

A class of fully coupled forward–backward stochastic difference equations with delay (FBSDDEs) over infinite horizon are considered in this article. By establishing a non‐homogeneous explicit relation between the forward and backward equations in terms of Riccati‐like difference equations, we derive the unique solution to the FBSDDEs under certain conditions. Then, we deduce that the FBSDDEs are solvable if and only if the corresponding stochastic delayed system is β$$ \beta $$‐degree open‐loop mean‐square exponentially stabilizable. Finally, as an application, the FBSDDEs are employed to demonstrate the maximum principle of the stochastic LQ optimal control problem. [ABSTRACT FROM AUTHOR]

The Pontryagin type maximum principle and Bellman's dynamic programming principle serve as two of the most important tools in solving optimal control problems. There is a huge literature on the study of relationship between them. The main purpose of this paper is to investigate the relationship between the Pontryagin type maximum principle and the dynamic programming principle for control systems governed by stochastic evolution equations in infinite dimensional space, with the control variables entering into both the drift and the diffusion terms. To do so, we first prove the dynamic programming principle for those systems without employing the martingale solutions. Then we establish the desired relationships in both cases that value function associated is smooth or nonsmooth. For the nonsmooth case, in particular, by employing the relaxed transposition solution, we discover the connection between the superdifferentials and subdifferentials of value function and the first-order and second-order adjoint equations. [ABSTRACT FROM AUTHOR]

To address the issue of inaccurate distributions in discrete-time stochastic systems, a minimax linear quadratic control method using theWasserstein metric is proposed. Our method aims to construct a control policy that is robust against errors in an empirical distribution of underlying uncertainty by adopting an adversary that selects the worst-case distribution at each time. The opponent receives a Wasserstein penalty proportional to the amount of deviation from the empirical distribution. As a tractable solution, a closed-form expression of the optimal policy pair is derived using a Riccati equation. We identify nontrivial stabilizability and observability conditions under which the Riccati recursion converges to the unique positive semidefinite solution of an algebraic Riccati equation. Our method is shown to possess several salient features, including closed-loop stability, a guaranteed-cost property, and a probabilistic out-of-sample performance guarantee. [ABSTRACT FROM AUTHOR]

A stochastic epidemic model with random noise transmission is taken into account, describing the dynamics of the measles viral infection. The basic reproductive number is calculated corresponding to the stochastic model. It is determined that, given initial positive data, the model has bounded, unique, and positive solution. Additionally, utilizing stochastic Lyapunov functional theory, we study the extinction of the disease. Stationary distribution and extinction of the infection are examined by providing sufficient conditions. We employed optimal control principles and examined stochastic control systems to regulate the transmission of the virus using environmental factors. Graphical representations have been offered for simplicity of comprehending in order to further verify the acquired analytical findings. [ABSTRACT FROM AUTHOR]

This paper considers an optimal control problem for a time-invariant linear stochastic system with discrete time, scalar unbounded control, additive noise, and a probabilistic criterion for retaining its trajectories in a given neighborhood of zero. We use dynamic programming and two-sided Bellman function estimates to derive analytical expressions for the optimal control at two time steps and a suboptimal control on any control horizon. The effectiveness of these controls is illustrated on a numerical example. [ABSTRACT FROM AUTHOR]

We study singular perturbations of a class of two-scale stochastic control systems with unbounded data. The assumptions are designed to cover some relaxation problems for deep neural networks. We construct effective Hamiltonian and initial data and prove the convergence of the value function to the solution of a limit (effective) Cauchy problem for a parabolic equation of HJB type. We use methods of probability, viscosity solutions and homogenization. [ABSTRACT FROM AUTHOR]

In this paper, we study the maximum principle for stochastic optimal control problems of forward–backward stochastic difference systems (FBSΔSs). Two types of FBSΔSs are investigated. The first one is described by a partially coupled forward–backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. By adopting an appropriate representation of the product rule and an appropriate formulation of the adjoint process, we deduce the adjoint difference equation. Finally, the maximum principle for this optimal control problem with the control domain being convex is established. [ABSTRACT FROM AUTHOR]

This article studies the maximum principle for stochastic optimal control problems of forward‐backward stochastic difference systems (FBSΔSs) where the uncertainty is modeled by a discrete time, finite state process, rather than white noises. Two distinct forms of FBSΔSs are investigated. The first one is described by a partially coupled forward‐backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. We deduce the adjoint difference equation by adopting an appropriate representation of the product rule and a proper formulation of the backward stochastic difference equation (BSΔE). Finally, the maximum principle for this optimal control problem with the convex control domain is established. [ABSTRACT FROM AUTHOR]

Modern robotic systems are expected to operate robustly in partially unknown environments. This article proposes an algorithm capable of controlling a wide range of high-dimensional robotic systems in such challenging scenarios. Our method is based on the path integral formulation of stochastic optimal control, which we extend with constraint-handling capabilities. Under our control law, the optimal input is inferred from a set of stochastic rollouts of the system dynamics. These rollouts are simulated by a physics engine, placing minimal restrictions on the types of systems and environments that can be modeled. Although sampling-based algorithms are typically not suitable for online control, we demonstrate in this work how importance sampling and constraints can be used to effectively curb the sampling complexity and enable real-time control applications. Furthermore, the path integral framework provides a natural way of incorporating existing control architectures as ancillary controllers for shaping the sampling distribution. Our results reveal that even in cases where the ancillary controller would fail, our stochastic control algorithm provides an additional safety and robustness layer. Moreover, in the absence of an existing ancillary controller, our method can be used to train a parametrized importance sampling policy using data from the stochastic rollouts. The algorithm may thereby bootstrap itself by learning an importance sampling policy offline and then refining it to unseen environments during online control. We validate our results on three robotic systems, including hardware experiments on a quadrupedal robot. [ABSTRACT FROM AUTHOR]

In this paper, inspired by various stochastic linear-quadratic (LQ, for short) problems, we develop the method of continuation to study nonlinear forward–backward stochastic differential equations (FBSDEs, for short) in a kind of domination-monotonicity frameworks. The coupling of FBSDEs is in a general form, i.e., it not only appears in integral terms and terminal terms, but also in initial terms. By virtue of introducing various matrices, matrix-valued random variables and matrix-valued stochastic processes, we present the domination-monotonicity framework carefully and rigorously. A unique solvability result and a pair of estimates for coupled FBSDEs are obtained (see Theorem 3.5 in the case of simple domination-monotonicity conditions and Theorem 5.2 in the case of multi-level self-similar domination-monotonicity structures). As applications of theoretical results, the related stochastic Hamiltonian systems of several LQ problems are discussed. [ABSTRACT FROM AUTHOR]

This paper presents a stochastic framework for the optimization of microgrids that has the functionality of providing flexibility services to System Operators (SOs) considering uncertainties in the energy forecast. The methodology is developed with the aim of being applied to complex microgrids composed of different distributed energy resources and hybrid energy storage systems (ESS). The associated optimization problem is operated in two stages: the first one performs a stochastic optimization of the microgrid in order to reserve an up/down regulation capacity with which to deal with the energy forecast uncertainties of the microgrid. The different microgrid devices are optimized by considering their operational costs in order to achieve their optimal operation in the Day-Ahead Market (DM). The second stage is used to re-schedule the initial planning according to the signal request and an economic offer from the SO. The control problem is developed using Stochastic Model Predictive Control (SMPC) techniques and Mixed-Integer Quadratic Programming (MIQP), owing to the presence of logic, integer, mixed and probabilistic variables. The simulation results show that the proposed methodology reduces the risk of undergoing up/down-penalty deviations in the Regulation Service Market (RM), also being able to provide flexibility services to the SOs, despite being subject to uncertainties in the energy forecast carried out for the microgrid. [ABSTRACT FROM AUTHOR]

An optimal bounded control strategy for strongly nonlinear vibro-impact systems under stochastic excitations with actuator saturation is proposed. First, the impact effect is incorporated in an equivalent equation by using a nonsmooth transformation. Under the assumption of light damping and weak random perturbation, the system energy is a slowly varying process. By using the stochastic averaging of envelope for strongly nonlinear systems, the partially averaged Itô stochastic differential equation for system energy can be derived. The optimal control problem is transformed from the original optimal control problem for the state variables to an equivalent optimal control problem for the system energy, which decreases the dimensions of the optimal control problem. Then, based on stochastic maximum principle, an adjoint equation for the adjoint variable and the maximum condition of partially averaged control problem are established. For infinite time-interval ergodic control, the adjoint variable is assumed to be a stationary process and the adjoint equation can be further simplified. Finally, the probability density function of the system energy and other statistics of the optimally controlled system are derived by calculating the associated Fokker–Plank–Kolmogorov equation. For comparison, the bang–bang control is also investigated and the control results are compared to show the advantages of the developed control strategy. [ABSTRACT FROM AUTHOR]

This paper considers an optimal control problem for a discrete-time stochastic system with the probability of first reaching the boundaries of a given domain as the optimality criterion. Dynamic programming-based sufficient conditions of optimality are formulated and proved. The isobells of levels 1 and 0 of the Bellman function are used for obtaining two-sided estimates of the right-hand side of the dynamic programming equation, two-sided estimates of the Bellman function, and two-sided estimates of the optimal-value function of the probabilistic criterion. A suboptimal control design method is proposed. The conditions of equivalence to an optimal control problem with a probabilistic terminal criterion are established. An illustrative example is given. [ABSTRACT FROM AUTHOR]

This paper examines an optimal control problem with a probabilistic criterion for retaining the trajectories of a discrete-time stochastic system in given sets. The dynamic programming method is employed for obtaining the isobells of levels 1 and 0 of the Bellman function, two-sided estimates for the right-hand side of the dynamic programming equation, two-sided estimates for the Bellman function, and the optimal-value function of the probabilistic criterion. These results are then used for deriving an approximate formula for the optimal control. As an illustrative example the problem of keeping an inverted pendulum in the neighborhood of an unstable equilibrium is considered. [ABSTRACT FROM AUTHOR]

Summary: In this article, we discuss an infinite horizon optimal control of the stochastic system with partial information, where the state is governed by a mean‐field stochastic differential delay equation driven by Teugels martingales associated with Lévy processes and an independent Brownian motion. First, we show the existence and uniqueness theorem for an infinite horizon mean‐field anticipated backward stochastic differential equation driven by Teugels martingales. Then applying different approaches for the underlying system, we establish two classes of stochastic maximum principles, which include two necessary conditions and two sufficient conditions for optimality, under a convex control domain. Moreover, compared with the finite horizon optimal control, we add the transversality conditions to the two kinds of stochastic maximum principles. Finally, using the stochastic maximum principle II, we settle an infinite horizon optimal consumption problem driven by Teugels martingales associated with Gamma processes. [ABSTRACT FROM AUTHOR]

A classical problem in stochastic ergodic control consists of studying the limit behavior of the optimal value of a discounted integral in infinite horizon (the so called Abel mean of an integral cost) as the discount factor tends to zero or the value defined with a Cesaro mean of an integral cost when the horizon T tends to +1. We investigate the possible limits in the norm of uniform convergence topology of values defined through Abel means or Cearo means when ! 0+ and T ! +1, respectively. Here we give two types of new representation formulas for the accumulation points of the values when the averaging parameter converges. We show that there is only one possible accumulation point which is the same for Abel means or Cesaro means. The first type of representation formula is based on probability measures on the product of the state space and the control state space, which are limits of occupational measures. The second type of representation formula is based on measures which are the projection of invariant measure on the space of relaxed controls. We also give a result comparing the both sets of measures involved in both classes of representation formulas. An important consequence of the representation formulas is the existence of the limit value when one has the equicontinuity property of Abel or Cesaro mean values. This is the case, for example, for nonexpansive stochastic control systems. In the end some insightful examples are given which help to better understand the results. [ABSTRACT FROM AUTHOR]

This paper is concerned with a finite‐time nonlinear stochastic optimal control problem with input saturation as a hard constraint on the control input. The stochastic Hamilton‐Jacobi‐Bellman (SHJB) equation associated with this problem is derived, and a concrete solution method is also presented by extending an iteration framework of path integral stochastic optimal control. This is the first result that provides an optimal feedback control rigorously satisfying the saturation constraint as a solution to the SHJB equation. © 2020 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc. [ABSTRACT FROM AUTHOR]

The aim of this paper is to investigate the existence of optimal controls for systems described by stochastic partial differential equations (SPDEs) with locally monotone coefficients controlled by external forces which are feedback controls. To attain our objective we adapt the argument of Lisei (2002) where the existence of optimal controls to the stochastic Navier–Stokes equation was studied. The results obtained in the present paper may be applied to demonstrate the existence of optimal controls to various types of controlled SPDEs such as: a stochastic nonlocal equation and stochastic semilinear equations which are locally monotone equations; we also apply the result to a monotone equation such as the stochastic reaction–diffusion equation and to a stochastic linear equation. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a new class of backward doubly stochastic differential equations (in short BDSDE) called mean-field backward doubly stochastic differential equations (in short MFBDSDE) driven by Itô-Lévy processes and study the partial information optimal control problems for backward doubly stochastic systems driven by Itô-Lévy processes of mean-field type, in which the coefficients depend on not only the solution processes but also their expected values. First, using the method of contraction mapping, we prove the existence and uniqueness of the solutions to this kind of MFBDSDE. Then, by the method of convex variation and duality technique, we establish a sufficient and necessary stochastic maximum principle for the stochastic system. Finally, we illustrate our theoretical results by an application to a stochastic linear quadratic optimal control problem of a mean-field backward doubly stochastic system driven by Itô-Lévy processes. [ABSTRACT FROM AUTHOR]

Starting from the Avellaneda–Stoikov framework, we consider a market maker who wants to optimally set bid/ask quotes over a finite time horizon, to maximize her expected utility. The intensities of the orders she receives depend not only on the spreads she quotes but also on unobservable factors modelled by a hidden Markov chain. We tackle this stochastic control problem under partial information with a model that unifies and generalizes many existing ones under full information, combining several risk metrics and constraints, and using general decreasing intensity functionals. We use stochastic filtering, control and piecewise-deterministic Markov processes theory, to reduce the dimensionality of the problem and characterize the reduced value function as the unique continuous viscosity solution of its dynamic programming equation. We then solve the analogous full information problem and compare the results numerically through a concrete example. We show that the optimal full information spreads are biased when the exact market regime is unknown, and the market maker needs to adjust for additional regime uncertainty in terms of P&L sensitivity and observed order flow volatility. This effect becomes higher, the longer the waiting time in between orders. [ABSTRACT FROM AUTHOR]

We consider the problem of governing systemic risk in a banking system model. The banking system model consists in an initial value problem for a system of stochastic differential equations whose dependent variables are the log-monetary reserves of the banks as functions of time. The banking system model considered generalizes previous models studied in Fouque and Sun (in: Fouque and Langsam (eds) Handbook of systemic risk, Cambridge University Press, Cambridge, pp 444–452, 2013), Carmona et al. (Commun Math Sci 13(4):911–933, 2015), Garnier et al. (SIAM J Financ Math 4:151–184, 2013) and describes an homogeneous population of banks. Two distinct mechanisms are used to model the cooperation among banks and the cooperation between banks and monetary authority. These mechanisms are regulated respectively by the parameters α and γ . A bank fails when its log-monetary reserves go below an assigned default level. We call systemic risk or systemic event in a bounded time interval the fact that in that time interval at least a given fraction of the banks fails. The probability of systemic risk in a bounded time interval is evaluated using statistical simulation. A method to govern the probability of systemic risk in a bounded time interval is presented. The goal of the governance is to keep the probability of systemic risk in a bounded time interval between two given thresholds. The governance is based on the choice of the log-monetary reserves of a kind of "ideal bank" as a function of time and on the solution of an optimal control problem for the mean field approximation of the banking system model. The solution of the optimal control problem determines the parameters α and γ as functions of time, that is defines the rules of the borrowing and lending activity among banks and between banks and monetary authority. Some numerical examples are discussed. In particular during a two year period we consider the governance of systemic risk in the next year when governance decisions are taken quarterly. The systemic risk governance is tested in absence and in presence of positive and negative shocks acting on the banking system. [ABSTRACT FROM AUTHOR]

We consider an optimal regulator problem for a class of nonlinear stochastic systems with a square-root nonlinearity and random coefficients, and using the quadratic-linear criterion. This represents a certain nonlinear generalisation of the stochastic linear-quadratic control problem with random coefficients. The solution if found in an explicit closed-form as an affine state-feedback control in terms of a Riccati and linear backward stochastic differential equations. As an application, we give the solution to an optimal investment problem in a market with random coefficients. [ABSTRACT FROM AUTHOR]

In this paper, the optimal control problem for a discrete-time stochastic system with a general-form probabilistic criterion is considered. Using dynamic programming and the properties of the Bellman function, new two-sided bounds of this function that refine the earlier results are constructed. The derived bounds are then adopted to justify the application of the modified strategy that is optimal in the two-step investment portfolio management problem under risk to the corresponding multistep problem. An example that illustrates the advantages of such a strategy over other well-known strategies is given. [ABSTRACT FROM AUTHOR]

An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is time-inconsistent in general. Therefore, instead of finding a global optimal control (which is not possible), we look for a time-consistent (approximately) locally optimal equilibrium strategy. Such a strategy can be represented through the solution to a system of partial differential equations, called an equilibrium Hamilton–Jacob–Bellman (HJB) equation which is constructed via a sequence of multi-person differential games. A verification theorem is proved and, under proper conditions, the well-posedness of the equilibrium HJB equation is established as well. [ABSTRACT FROM AUTHOR]

Consideration was given to the design of the optimal control of the general discrete stochastic system with a criterion as the probability of the state vector sojourn in the given sets at each time instant. Derived were relations of the dynamic programming enabling one to establish an optimal solution in the class of Markov strategies without extension of the state vector with subsequent reduction to an equivalent problem with the probabilistic terminal criterion. Consideration was given to the problem of one-parameter correction of the flying vehicle trajectory. An analytical solution was established. [ABSTRACT FROM AUTHOR]

In this study, the problem of designing a stochastic optimal controller for sampled‐data systems whose sampling interval is subjected to a certain probability distribution is addressed. To design the controller, the Kronecker product operation and the Vandermonde matrix were introduced. A design method of the stochastic optimal controller is proposed. It is shown that the controller guarantee that the closed‐loop system has exponentially mean square stability. Finally, the simulation results illustrate the effectiveness and practicability of the proposed method. [ABSTRACT FROM AUTHOR]

The solution of a stochastic optimal control problem may be associated with that of the Hamilton–Jacobi–Bellman (HJB) equation, which is a second order partial differential equation subject to a terminal condition. When this equation is semilinear and satisfies certain other constraints, it can be solved via a nonlinear version of the Feynman–Kac formula. According to this approach, the solution to the HJB equation can be obtained by simulating an associated pair of partly coupled forward–backward stochastic differential equations. Although an elegant way to interpret and solve a partial differential equation, simulating the system of forward–backward equations can be computationally inefficient. In this work, the HJB equation pertaining to the optimal control problem is reformulated such that instead of the given terminal condition, it is now subject to an appropriate initial condition. In the process, while the total cost associated with the control problem remains unchanged, pathwise solutions may not. Associated with the new partial differential equation, we then derive a set of stochastic differential equations whose solutions move only forward in time. This approach has a significant computational advantage over the original formulation. Moreover, since the forward–backward approach generally requires simulating stochastic differential equations over the current to the terminal time at every step, the integration errors may accumulate and carry forward in estimating the control. This error is particularly high initially since the time of integration is the longest there. The proposed method, numerically implemented for the control of stochastically excited oscillators, is free of such errors and hence more robust. Unsurprisingly, it also exhibits lower sampling variance. [Display omitted] • A Monte Carlo scheme with forward time integration for stochastic optimal control. • Based on a modified HJB equation, now supplied with an initial condition. • Achieved through a smearing of the terminal cost by Ito's formula. • Vastly reduced computational cost and sampling variance. • Advantages illustrated with numerical examples. [ABSTRACT FROM AUTHOR]

In this letter we provide several informative tight error bounds when using value function approximators for the risk-sensitive cost setting for a given policy represented using exponential utility. The novelty of our approach is that we make use of the irreducibility of the underlying Markov chain (resulting in better bounds using Perron–Frobenius eigenvectors) to derive new bounds whereas the earlier work used primarily the spectral variation bound which holds for any matrix, hence did not make use of the irreducibility. All our bounds have a perturbation term for large state spaces. We also present examples where we show that the new bounds perform 90-100% better than the earlier proposed spectral variation bound. [ABSTRACT FROM AUTHOR]

In this paper, an open problem is solved, for the stochastic optimal control problem with delay where the control domain is nonconvex and the diffusion term contains both control and its delayed term. Inspired by previous results about delayed stochastic control systems, Peng's global stochastic maximum principle is generalized to the time delayed case. A special backward stochastic differential equation is introduced to deal with the cross terms, when applying the duality technique. Comparing with the classical result, the maximum condition contains an indicator function, which in fact is the characteristic of the stochastic optimal control problem with delay. Furthermore, to illustrate the applications of our theoretical results, three dynamic optimization problems are addressed based on the global maximum principle. [ABSTRACT FROM AUTHOR]