Search

This paper deals with finite‐horizon minimum‐variance and covariance steering problems subject to constraints. The goal of the minimum variance problem is to steer the state mean of an uncertain system to a prescribed vector while minimizing the trace of its terminal state covariance whereas the goal in the covariance steering problem is to steer the covariance matrix of the terminal state to a prescribed positive definite matrix. The paper proposes a solution approach that relies on a stochastic version of the affine disturbance feedback control parametrization. In this control policy parametrization, the control input at each stage is expressed as an affine function of the history of disturbances that have acted upon the system. It is shown that this particular parametrization reduces the stochastic optimal control problems considered in this paper into tractable convex programs or difference of convex functions programs with essentially the same decision variables. In addition, the paper proposes a variation of this control parametrization that relies on truncated histories of past disturbances, which allows for sub‐optimal controllers to be designed that strike a balance between performance and computational cost. The suboptimality of the truncated policies is formally analyzed and closed form expressions are provided for the performance loss due to the use of the truncation scheme. Finally, the paper concludes with a comparative analysis of the truncated versions of the proposed policy parametrization and other standard policy parametrizations through numerical simulations. [ABSTRACT FROM AUTHOR]

Summary: This paper deals with the second‐order necessary optimality conditions for discrete‐time stochastic optimal control problems under weakened convexity assumptions. Using a special variation of the control, and by virtue of a new discrete‐time backward stochastic equation, we establish a more general and constructive first‐order necessary optimality condition in the form of a global stochastic maximum principle. Moreover, by introducing a new discrete‐time backward stochastic matrix equation, the second‐order multipoint necessary optimality conditions of singular controls are derived, which covers and improves the classical second‐order necessary optimality conditions of discrete‐time stochastic systems. [ABSTRACT FROM AUTHOR]

We study a stochastic control problem of mean‐field controlled stochastic differential systems driven by a fractional Brownian motion with Hurst parameter H∈(1/2,1)$$ H\in \left(1/2,1\right) $$. As a necessary condition of the optimal control we obtain a stochastic maximum principle. The associated adjoint mean‐field backward stochastic differential equation driven by a fractional Brownian motion and a classical Brownian motion. Applying the stochastic maximum principle to a mean‐field stochastic linear quadratic problem, we obtain the optimal control and prove that the necessary condition for the optimality of an admissible control is also sufficient under certain assumptions. [ABSTRACT FROM AUTHOR]

A class of fully coupled forward–backward stochastic difference equations with delay (FBSDDEs) over infinite horizon are considered in this article. By establishing a non‐homogeneous explicit relation between the forward and backward equations in terms of Riccati‐like difference equations, we derive the unique solution to the FBSDDEs under certain conditions. Then, we deduce that the FBSDDEs are solvable if and only if the corresponding stochastic delayed system is β$$ \beta $$‐degree open‐loop mean‐square exponentially stabilizable. Finally, as an application, the FBSDDEs are employed to demonstrate the maximum principle of the stochastic LQ optimal control problem. [ABSTRACT FROM AUTHOR]

This study applies the theory of stochastic control to an environmental project planning to counteract against the sediment starvation problem in river environments. This can be considered as a time‐average inventory problem to time‐discretely control a continuous‐time system driven by a non‐smooth jump process under habit formation disturbing project implementation. The system is modeled such that the sediment storage dynamics are physically consistent with certain experimental results. Further, the habit formation is modeled as simple linear dynamics and serves as a constraint related to the replenishment amount of the sediment. We show that the time‐average control problem is not necessarily ergodic. Consequently, the effective Hamiltonian may become a non‐constant. Thereafter, ratcheting cases as extreme cases of the irreversible habit formation are considered, owing to them being unique exactly solvable non‐ergodic control problems. The optimality equation associated with a regularized and hence well‐defined control problem is verified. Furthermore, a finite difference scheme is examined against the exactly solvable case and then applied to more complicated cases. [ABSTRACT FROM AUTHOR]

This paper deals with the optimal control of grid‐connected Battery Energy Storage Systems (BESSs) operating for energy arbitrage. An important issue is that BESSs degrade over time, according to their use, and thus they are usable only for a limited number of cycles. Therefore, the time horizon of the optimization depends on the actual operation of the BESS. We focus on Li‐ion batteries and use an empirical model to describe battery degradation. The BESS model includes an equivalent circuit for the battery and a simplified model for the power converter. In order to model the energy price variations, we use a linear stochastic model that includes the effect of the time‐of‐the‐day. The problem of maximizing the revenues obtained over the BESS lifetime is formulated as a stochastic optimal control problem with a long, operation‐dependent time horizon. First, we divide this problem into a finite set of subproblems, such that for each one of them, the State of Health (SoH) of the battery is approximately constant. Next, we reformulate approximately every subproblem into the minimization of the ratio of two long‐time average‐cost criteria and use a value‐iteration‐type algorithm to derive the optimal policy. Finally, we present some numerical results and investigate the effects of the energy loss parameters, degradation parameters, and price dynamics on the optimal policy. [ABSTRACT FROM AUTHOR]

In dealer markets, dealers provide prices at which they agree to buy and sell the assets and securities they have in their scope. With ever increasing trading volume, this quoting task has to be done algorithmically in most markets such as foreign exchange (FX) markets or corporate bond markets. Over the last 10 years, many mathematical models have been designed that can be the basis of quoting algorithms in dealer markets. Nevertheless, in most (if not all) models, the dealer is a pure internalizer, setting quotes and waiting for clients. However, on many dealer markets, dealers also have access to an interdealer market or even public trading venues where they can hedge part of their inventory. In this paper, we propose a model taking this possibility into account therefore allowing dealers to externalize part of their risk. The model displays an important feature well known to practitioners that within a certain inventory range, the dealer internalizes the flow by appropriately adjusting the quotes and starts externalizing outside of that range. The larger the franchise, the wider is the inventory range suitable for pure internalization. The model is illustrated numerically with realistic parameters for USDCNH spot market. [ABSTRACT FROM AUTHOR]

In this paper, we explore the dynamics of working hours and wages in a model economy where a firm and its workforce are linked to each other by an implicit contract. Specifically, we develop a deterministic and a stochastic framework in which the firm sets its level of labor utilization by considering that workers' earnings tend to adjust in the direction of a fixed level. Without any uncertainty about firm's profitability, we show that the existence and the properties of stationary solutions rely on the factors that usually determine the enforceability of contracts and we demonstrate that wages move countercyclically towards the allocation preferred by the firm. Moreover, we show that adding uncertainty does not overturn the countercyclical pattern of wages but is helpful in explaining their dynamic behavior in response to demand shocks as well as their typical stickiness observed at the macrolevel. [ABSTRACT FROM AUTHOR]

This article studies the maximum principle for stochastic optimal control problems of forward‐backward stochastic difference systems (FBSΔSs) where the uncertainty is modeled by a discrete time, finite state process, rather than white noises. Two distinct forms of FBSΔSs are investigated. The first one is described by a partially coupled forward‐backward stochastic difference equation (FBSΔE) and the second one is described by a fully coupled FBSΔE. We deduce the adjoint difference equation by adopting an appropriate representation of the product rule and a proper formulation of the backward stochastic difference equation (BSΔE). Finally, the maximum principle for this optimal control problem with the convex control domain is established. [ABSTRACT FROM AUTHOR]

In this article, we propose a class of discrete‐time approximation schemes for stochastic optimal control problems under the G‐expectation framework. The proposed schemes are constructed recursively based on piecewise constant policy. We prove the convergence of the discrete schemes and determine the convergence rates. Several numerical examples are presented to illustrate the effectiveness of the obtained results. [ABSTRACT FROM AUTHOR]

Summary: In this article, we address the task of setting up an optimal production plan taking into account an uncertain demand. The energy system is represented by a system of hyperbolic partial differential equations and the uncertain demand stream is captured by an Ornstein‐Uhlenbeck process. We determine the optimal inflow depending on the producer's risk preferences. The resulting output is intended to optimally match the stochastic demand for the given risk criteria. We use uncertainty quantification for an adaptation to different levels of risk aversion. More precisely, we use two types of chance constraints to formulate the requirement of demand satisfaction at a prescribed probability level. In a numerical analysis, we analyze the chance constrained optimization problem for the Telegrapher's equation and a real‐world coupled gas‐to‐power network. [ABSTRACT FROM AUTHOR]

In most over‐the‐counter (OTC) markets, a small number of market makers provide liquidity to other market participants. More precisely, for a list of assets, they set prices at which they agree to buy and sell. Market makers face therefore an interesting optimization problem: they need to choose bid and ask prices for making money while mitigating the risk associated with holding inventory in a volatile market. Many market‐making models have been proposed in the academic literature, most of them dealing with single‐asset market making whereas market makers are usually in charge of a long list of assets. The rare models tackling multiasset market making suffer however from the curse of dimensionality when it comes to the numerical approximation of the optimal quotes. The goal of this paper is to propose a dimensionality reduction technique to address multiasset market making by using a factor model. Moreover, we generalize existing market‐making models by the addition of an important feature: the existence of different transaction sizes and the possibility for the market makers in OTC markets to answer different prices to requests with different sizes. [ABSTRACT FROM AUTHOR]

Summary: In this article, we discuss an infinite horizon optimal control of the stochastic system with partial information, where the state is governed by a mean‐field stochastic differential delay equation driven by Teugels martingales associated with Lévy processes and an independent Brownian motion. First, we show the existence and uniqueness theorem for an infinite horizon mean‐field anticipated backward stochastic differential equation driven by Teugels martingales. Then applying different approaches for the underlying system, we establish two classes of stochastic maximum principles, which include two necessary conditions and two sufficient conditions for optimality, under a convex control domain. Moreover, compared with the finite horizon optimal control, we add the transversality conditions to the two kinds of stochastic maximum principles. Finally, using the stochastic maximum principle II, we settle an infinite horizon optimal consumption problem driven by Teugels martingales associated with Gamma processes. [ABSTRACT FROM AUTHOR]

This paper is concerned with a finite‐time nonlinear stochastic optimal control problem with input saturation as a hard constraint on the control input. The stochastic Hamilton‐Jacobi‐Bellman (SHJB) equation associated with this problem is derived, and a concrete solution method is also presented by extending an iteration framework of path integral stochastic optimal control. This is the first result that provides an optimal feedback control rigorously satisfying the saturation constraint as a solution to the SHJB equation. © 2020 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc. [ABSTRACT FROM AUTHOR]

Summary: This paper studies the optimal reinsurance‐investment problems for an insurance company where the claim process follows a Brownian motion with drift. It turns out that there is a region where the probability of drawdown, namely, the probability that the value of the insurer's surplus process reaches some fixed fractional value of its maximum value to date is positive. Then in the complementary region, drawdown can be avoided with certainty. For this reason, we call the former region the "danger‐zone" and "safe‐region" for the latter. In the danger‐zone, we consider the problem of minimizing the probability of drawdown; and in the safe‐region, we turn our attention to the optimization problem of minimizing the expected time to reach a given capital level. Using the technique of stochastic control theory and the corresponding Hamilton‐Jacobi‐Bellman equation, explicit expressions of the optimal reinsurance‐investment strategies and the associated value functions are derived for the two optimization problems. Moreover, we provide several detailed comparisons to investigate the impact of some important parameters on the optimal strategies and illustrate the observation from behavior finance of view. [ABSTRACT FROM AUTHOR]

Summary: In this paper, two spectral methods are presented to solve a stochastic optimal control problem of a partially observable system. These two methods work together to solve such problems. In fact, solving such problems involves two cases: obtaining the control function and simulating the partially observable system. At first, a spectral linear filter is defined as a function of time to obtain an appropriate solution for a partially observable system. This linear filter is equipped with an orthogonal basis and it is made to predict the future behavior of this system. In this method, the goal is to approximate the trend of the partially observable system. The second method is suggested to achieve the optimal control corresponding to each sample path. In this method, the spectral Fourier transform is used. These two methods are used together to solve linear and nonlinear cases. In fact, the innovative contents of this paper are both the spectral linear filter and the suggested spectral optimal control method. [ABSTRACT FROM AUTHOR]

Summary: Stochastic model predictive control hinges on the online solution of a stochastic optimal control problem. This paper presents a computationally efficient solution method for stochastic optimal control for nonlinear systems subject to (time‐varying) stochastic disturbances and (time‐invariant) probabilistic model uncertainty in initial conditions and parameters. To this end, new methods are presented for joint propagation of time‐varying and time‐invariant probabilistic uncertainty and the nonconservative approximation of joint chance constraint (JCC) on the system state. The proposed uncertainty propagation method relies on generalized polynomial chaos and conditional probability rules to obtain tractable expressions for the state mean and covariance matrix. A moment‐based surrogate is presented for JCC approximation to circumvent construction of the full probability distribution of the state or the use of integer variables as required when using the sample average approximation. The proposed solution method for stochastic optimal control is illustrated on a nonlinear semibatch reactor case study in the presence of probabilistic model uncertainty and stochastic disturbances. It is shown that the proposed solution method is significantly superior to a standard random sampling method for stochastic optimal control in terms of computational requirements. Furthermore, the moment‐based surrogate for the JCC is shown to be substantially less conservative than the widely used distributionally robust Cantelli‐Chebyshev inequality for chance constraint approximation. [ABSTRACT FROM AUTHOR]

Optimizing aircraft collision avoidance and performing trajectory optimization are the key problems in an air transportation system. This paper is focused on solving these problems by using a stochastic optimal control approach. The major contribution of this paper is a proposed stochastic optimal control algorithm to dynamically adjust and optimize aircraft trajectory. In addition, this algorithm accounts for random wind dynamics and convective weather areas with changing size. Although the system is modeled by a stochastic differential equation, the optimal feedback control for this equation can be computed as a solution of a partial differential equation, namely, an elliptic Hamilton‐Jacobi‐Bellman equation. In this paper, we solve this equation numerically using a Markov Chain approximation approach, where a comparison of three different iterative methods and two different optimization search methods are presented. Simulations show that the proposed method provides better performance in reducing conflict probability in the system and that it is feasible for real applications. [ABSTRACT FROM AUTHOR]

We consider the controlled stochastic Navier–Stokes equations in a bounded multidimensional domain, where the noise term allows jumps. In order to prove existence and uniqueness of an optimal control w.r.t. a given control problem, we first need to show the existence and uniqueness of a local mild solution of the considered controlled stochastic Navier–Stokes equations. We then discuss the control problem, where the related cost functional includes stopping times dependent on controls. Based on the continuity of the cost functional, we can apply existence and uniqueness results provided in [4], which enables us to show that a unique optimal control exists. [ABSTRACT FROM AUTHOR]

Nonlinear stochastic optimal control theory has played an important role in many fields. In this theory, uncertainties of dynamics have usually been represented by Brownian motion, which is Gaussian white noise. However, there are many stochastic phenomena whose probability density has a long tail, which suggests the necessity to study the effect of non-Gaussianity. This paper employs Lévy processes, which cause outliers with a significantly higher probability than Brownian motion, to describe such uncertainties. In general, the optimal control law is obtained by solving the Hamilton-Jacobi-Bellman equation. This paper shows that the path-integral approach combined with the policy iteration method is efficiently applicable to solve the Hamilton-Jacobi-Bellman equation in the Lévy problem setting. Finally, numerical simulations illustrate the usefulness of this method. [ABSTRACT FROM AUTHOR]

Recent technology breakthroughs towards a fully automated artificial pancreas give rise to the need of new monitoring tools aiming at increasing both reliability and performance of a closed-loop glycemic regulator. Based on error grid analysis, an insightful monitoring tool is proposed to assess if a given closed-loop implementation respects its specification of an optimally performing glycemic regulator under uncertainty. The optimal behavior specification is obtained using linearly solvable Markov decision processes, whereby the Bellman optimality equation is made linear through an exponential transformation that allows obtaining the optimal control policy in an explicit form. The specification for the desired glucose dynamics is learned using Gaussian processes for state transitions in an optimally performing artificial pancreas. By means of the proposed grid, the specification is vis-à-vis compared with glucose sensor readings so that any significant deviation from the expected closed-loop performance under abnormal or faulty scenarios can be detected. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Stochastic optimal control problems in which two-dimensional diffusion processes are controlled until they enter a given set are solved explicitly. When symmetry can be used, exact solutions are obtained. The same arguments are valid for one-dimensional processes. In the general case, it is shown how to obtain good approximate solutions. Various examples are presented. [ABSTRACT FROM AUTHOR]

The classical literature on optimal liquidation, rooted in Almgren-Chriss models, tackles the optimal liquidation problem using a trade-off between market impact and price risk. It answers the general question of optimal scheduling but the very question of the actual way to proceed with liquidation is rarely dealt with. Our model, which incorporates both price risk and nonexecution risk, is an attempt to tackle this question using limit orders. The very general framework we propose to model liquidation with limit orders generalizes existing ones in two ways. We consider a risk-averse agent, whereas the model of Bayraktar and Ludkovski only tackles the case of a risk-neutral one. We consider very general functional forms for the execution process intensity, whereas Guéant, Lehalle and Fernandez-Tapia are restricted to exponential intensity. Eventually, we link the execution cost function of Almgren-Chriss models to the intensity function in our model, providing then a way to see Almgren-Chriss models as a limit of ours. [ABSTRACT FROM AUTHOR]

This paper is concerned with the forward-backward stochastic optimal control problem with Poisson jumps. A necessary condition of optimality in the form of a global maximum principle as well as a sufficient condition of optimality are presented under the assumption that the diffusion and jump coefficients do not contain the control variable, and the control domain need not be convex. The case where there are some state constraints is also discussed. A financial example is discussed to illustrate the application of our result. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society [ABSTRACT FROM AUTHOR]