Search

Abstract We consider linear–quadratic mean field Stackelberg differential games with the adapted open-loop information structure of the leader. There are one leader and N followers, where N is arbitrarily large. The leader holds a dominating position in the game in the sense that the leader first chooses and then announces the optimal strategy to which the N followers respond by playing a Nash game, i.e., choosing their optimal strategies noncooperatively and simultaneously based on the leader's observed strategy. In our setting, the followers are coupled with each other through the mean field term included in their cost functions, and are strongly influenced by the leader's open-loop strategy included in their cost functions and dynamics. From the leader's perspective, he is coupled with the N followers through the mean field term included in his cost function. To circumvent the complexity brought about by the coupling nature among the leader and the followers with large N , which makes the use of the direct approach almost impossible, our approach in this paper is to characterize an approximated stochastic mean field process by solving a local optimal control problem of the followers with leader's control taken as an exogenous stochastic process. We show that for each fixed strategy of the leader, the followers' local optimal decentralized strategies lead to an ϵ -Nash equilibrium. The paper then solves the leader's local optimal control problem, as a nonstandard constrained optimization problem, with constraints being induced by the approximated mean field process determined by Nash followers (which also depend on the leader's control). We show that the local optimal decentralized controllers for the leader and the followers constitute an (ϵ 1 , ϵ 2) -Stackelberg–Nash equilibrium for the original game, where ϵ 1 and ϵ 2 both converge to zero as N → ∞. Numerical examples are provided to illustrate the theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, we consider a class of general linear forward and , backward stochastic difference equations (FBSDEs) which are fully coupled. The necessary and sufficient conditions for the existence of a (unique) solution to FBSDEs are given in terms of a Riccati equation. Two kinds of stochastic LQ optimal control problem are then studied as applications. First, we derive the optimal solution to the classic stochastic LQ problem by applying the solution to the associated FBSDEs. Secondly, we study a new type of LQ problem governed by a forward–backward stochastic system (FBSS). By applying the maximum principle and the solution to FBSDEs, an explicit solution is given in terms of a Riccati equation. Finally, by exploring the asymptotic behavior of the Riccati equation, we derive an equivalent condition for the mean-square stabilizability of FBSS. [ABSTRACT FROM AUTHOR]

Most cyber–physical systems (CPS) encounter a large volume of data which is added to the system gradually in real time and not altogether in advance. In this paper, we provide a theoretical framework that yields optimal control strategies for such CPS at the intersection of control theory and learning. In the proposed framework, we use the actual CPS, i.e., the "true" system that we seek to optimally control online, in parallel with a model of the CPS that is available. We then institute an information state for the system which does not depend on the control strategy. An important consequence of this independence is that for any given choice of a control strategy and a realization of the system's variables until time t , the information states at future times do not depend on the choice of the control strategy at time t but only on the realization of the decision at time t , and thus they are related to the concept of separation between estimation of the state and control. Namely, the future information states are separated from the choice of the current control strategy. Such control strategies are called separated control strategies. Hence, we can derive offline the optimal control strategy of the system with respect to the information state, which might not be precisely known due to model uncertainties or complexity of the system, and then use standard learning approaches to learn the information state online while data are added gradually to the system in real time. We show that after the information state becomes known, the separated control strategy of the CPS model derived offline is optimal for the actual system. We illustrate the proposed framework in a dynamic system consisting of two subsystems with a delayed sharing information structure. [ABSTRACT FROM AUTHOR]

This work deals with a finite-horizon covariance control problem for discrete-time, stochastic linear systems with complete state information subject to input constraints. First, we present the main steps for the transcription of the covariance control problem, which is originally formulated as a stochastic optimal control problem, into a deterministic nonlinear program (NLP) with a convex performance index and with both convex and non-convex constraints. In particular, the convex constraints in this nonlinear program are induced by the input constraints of the stochastic optimal control problem, whereas the non-convex constraints are induced by the requirement that the terminal state covariance be equal to a prescribed positive definite matrix. Subsequently, we associate this nonlinear program, via a simple convex relaxation technique, with a (convex) semi-definite program, which can be solved numerically by means of modern computational tools of convex optimization. Although, in general, the endpoints of a representative sample of closed-loop trajectories generated by the control policy that corresponds to the solution of the relaxed convex program are not expected to follow exactly the goal terminal Gaussian distribution, they are more likely to be concentrated near the mean of this distribution than if they were drawn from the latter, which is a desirable feature in practice. Numerical simulations that illustrate the key ideas of this work are also presented. [ABSTRACT FROM AUTHOR]

Abstract: This article is a survey of the early development of selected areas in nonlinear continuous-time stochastic control. Key developments in optimal control and the dynamic programming principle, existence of optimal controls under complete and partial observations, nonlinear filtering, stochastic stability, the stochastic maximum principle and ergodic control are discussed. Issues concerning wide bandwidth noise for stability, modeling, filtering and ergodic control are dealt with. The focus is on the earlier work, but many important topics are omitted for lack of space. [Copyright &y& Elsevier]

Abstract: A general maximum principle for optimal control problems derived by forward–backward stochastic systems is established, where control domains are non-convex and forward diffusion coefficients explicitly depend on control variables. These optimal control problems have broad applications in mathematical finance and economics such as the recursive mean–variance portfolio choice problems. The maximum principle is applied to study a forward–backward linear-quadratic optimal control problem with a non-convex control domain; an optimal solution is obtained. [Copyright &y& Elsevier]

Abstract: This paper deals with the finite horizon stochastic optimal control problem with the expectation of the -norm as the objective function and jointly Gaussian, although not necessarily independent, additive disturbance process. We develop an approximation strategy that solves the problem in a certain class of nonlinear feedback policies while ensuring satisfaction of hard input constraints. A bound on suboptimality of the proposed strategy in this class of nonlinear feedback policies is given for the special case of . We also develop a recursively feasible receding horizon policy with respect to state chance constraints and/or hard control input constraints in the presence of bounded disturbances. The performance of the proposed policies is examined in two numerical examples. [Copyright &y& Elsevier]

Abstract: In this paper, we consider the stochastic optimal control problem of discrete-time linear systems subject to Markov jumps and multiplicative noises under two criteria. The first one is an unconstrained mean–variance trade-off performance criterion along the time, and the second one is a minimum variance criterion along the time with constraints on the expected output. We present explicit conditions for the existence of an optimal control strategy for the problems, generalizing previous results in the literature. We conclude the paper by presenting a numerical example of a multi-period portfolio selection problem with regime switching in which it is desired to minimize the sum of the variances of the portfolio along the time under the restriction of keeping the expected value of the portfolio greater than some minimum values specified by the investor. [Copyright &y& Elsevier]

Abstract: In this paper, we develop a theoretical framework for linear quadratic regulator design for linear systems with probabilistic uncertainty in the parameters. The framework is built on the generalized polynomial chaos theory. In this framework, the stochastic dynamics is transformed into deterministic dynamics in higher dimensional state space, and the controller is designed in the expanded state space. The proposed design framework results in a family of controllers, parameterized by the associated random variables. The theoretical results are applied to a controller design problem based on stochastic linear, longitudinal F16 model. The performance of the stochastic design shows excellent consistency, in a statistical sense, with the results obtained from Monte-Carlo based designs. [Copyright &y& Elsevier]

Abstract: A new approach for finding an exact analytical solution to the modified Hamilton–Jacobi–Bellman equation is proposed. Together with the recently developed hybrid solution method, the proposed strategy allows to find a solution to a whole class of stochastic optimal control problems with bounded in magnitude control force. [Copyright &y& Elsevier]

Abstract: The standard approach in computer-controlled systems is to sample and control periodically. In certain applications, such as networked control systems or energy-constrained systems, it could be advantageous to instead use event-based control schemes. Aperiodic event-based control of first-order stochastic systems has been investigated in previous work. In any real implementation, however, it is necessary to have a well-defined minimum inter-event time. In this paper, we explore two such sporadic control schemes for first-order linear stochastic systems and compare the achievable performance to both periodic and aperiodic control. The results show that sporadic control can give better performance than periodic control in terms of both reduced process state variance and reduced control action frequency. [Copyright &y& Elsevier]

We tackle the problem of trajectory planning in an environment comprised of a set of obstacles with uncertain time-varying locations. The uncertainties are modeled using widely accepted Gaussian distributions, resulting in a chance-constrained program. Contrary to previous approaches however, we do not assume perfect knowledge of the moments of the distribution, and instead estimate them through finite samples available from either sensors or past data. We derive tight concentration bounds on the error of these estimates to sufficiently tighten the chance-constraint program. As such, we provide provable guarantees on satisfaction of the chance-constraints corresponding to the nominal yet unknown moments. We illustrate our results with two autonomous vehicle trajectory planning case studies. [ABSTRACT FROM AUTHOR]

Abstract: A method is presented for solving the infinite time Hamilton–Jacobi–Bellman (HJB) equation for certain state-constrained stochastic problems. The HJB equation is reformulated as an eigenvalue problem, such that the principal eigenvalue corresponds to the expected cost per unit time, and the corresponding eigenfunction gives the value function (up to an additive constant) for the optimal control policy. The eigenvalue problem is linear and hence there are fast numerical methods available for finding the solution. [Copyright &y& Elsevier]

Abstract: Temporal patterns for advertising include constant spending over time, decreasing spending over time and increasing spending over time. This research shows that all these spending patterns emerge at optimality for the same response function dynamics, due to differences in salvage value assumptions. I use these results to develop a methodology for determining the optimal planning horizon length for each pattern of spending. [Copyright &y& Elsevier]

Abstract: A ‘proportional’ order-up-to policy reacting to ARMA demand is analyzed using stochastic optimal control theory. This policy is compared with a full-state-feedback order-up-to policy. Necessary conditions for an optimum of a weighted sum of the inventory and the ordering variances for both policies are formulated. Based on this a relatively simple expression for the ‘full-state’ policy is derived. The comparison between the two policies demonstrates that the ‘intuitively’ designed proportional policy does not fulfill the objective of controlling both the inventory and ordering variance for all parameter values of the demand model as well as the full-state-feedback policy. The full-state-feedback policy outperforms the proportional policy in several aspects. [Copyright &y& Elsevier]

Probabilistic guarantees of safety and performance are important in constrained dynamical systems with stochastic uncertainty. We consider the stochastic reachability problem, which maximizes the probability that the state remains within time-varying state constraints (i.e., a "target tube"), despite bounded control authority. This problem subsumes the stochastic viability and terminal hitting-time stochastic reach-avoid problems. Of special interest is the stochastic reach set, the set of all initial states from which it is possible to stay in the target tube with a probability above a desired threshold. We provide sufficient conditions under which the stochastic reach set is closed, compact, and convex, and provide an underapproximative interpolation technique for stochastic reach sets. Utilizing convex optimization, we propose a scalable and grid-free algorithm that computes a polytopic underapproximation of the stochastic reach set and synthesizes an open-loop controller. This algorithm is anytime, i.e., it produces a valid output even on early termination. We demonstrate the efficacy and scalability of our approach on several numerical examples, and show that our algorithm outperforms existing software tools for verification of linear systems. [ABSTRACT FROM AUTHOR]

Optimal control is a prominent approach in robotics and movement neuroscience, among other fields of science. Methods for deriving optimal choices of action have been classically devised either in deterministic or stochastic settings. Here, we consider a setting in-between that retains the stochastic aspect of the controlled system but assumes deterministic open-loop control actions. The rationale stems from observations about the neural control of movement which highlighted that relatively stable behaviors can be achieved without feedback circuitry, via open-loop motor commands adequately tuning the mechanical impedance of the neuromusculoskeletal system. Yet, effective methods for deriving optimal open-loop controls for stochastic systems are lacking overall. This work presents a continuous-time approach based on statistical linearization techniques for the efficient computation of optimal open-loop controls for a broad class of stochastic optimal control problems. We first show that non-trivial departure from the optimal solutions of classical deterministic and stochastic approaches may arise for simple synthetic examples, thereby stressing the originality of the framework. We then exemplify its potential relevance to the planning of biological movement by showing that a well-known phenomenon in motor control, referred to as muscle co-contraction, occurs naturally. More generally, this stochastic optimal control framework may be suited to other fields where the design of optimal open-loop actions is relevant. [ABSTRACT FROM AUTHOR]

In this paper, we provide a closed-form solution to an optimal portfolio execution problem with stochastic price impact and stochastic net demand pressure. Specifically, each trade of an investor has temporary and permanent price impacts, both of which are driven by a continuous-time Markov chain; whereas the net demand pressure from other inventors is modelled by an Ornstein—Uhlenbeck process. The investor optimally liquidates his portfolio to maximize his expected revenue netting his cumulative inventory cost over a finite time. Such a problem is first reformulated as an optimal stochastic control problem for a Markov jump linear system. Then, we derive the value function and the optimal feedback execution strategy in terms of the solutions to coupled differential Riccati equations. Under some mild conditions, we prove that the coupled system is well-posed, and establish a verification theorem. Financially, our closed-form solution shows that the investor optimally liquidates his portfolio towards a dynamic benchmark. Moreover, the investor trades aggressively (conservatively) in the state of low (high) price impact. [ABSTRACT FROM AUTHOR]