Search

We consider a general type of non-Markovian impulse control problems under adverse non-linear expectation or, more specifically, the zero-sum game problem where the adversary player decides the probability measure. We show that the upper and lower value functions satisfy a dynamic programming principle (DPP). We first prove the dynamic programming principle (DPP) for a truncated version of the upper value function in a straightforward manner. Relying on a uniform convergence argument then enables us to show the DPP for the general setting. Following this, we use an approximation based on a combination of truncation and discretization to show that the upper and lower value functions coincide, thus establishing that the game has a value and that the DPP holds for the lower value function as well. Finally, we show that the DPP admits a unique solution and give conditions under which a saddle point for the game exists. As an example, we consider a stochastic differential game (SDG) of impulse versus classical control of path-dependent stochastic differential equations (SDEs). [ABSTRACT FROM AUTHOR]

Optimal control problems are ubiquitous in practical engineering applications and social life with the idea of cost or resource conservation. Based on the critic learning scheme, adaptive dynamic programming (ADP) is regarded as a significant avenue to address the optimal control problems by combining the advanced design ideas such as adaptive control, reinforcement learning, and intelligent control. This survey introduces the recent development of ADP and related intelligent critic control with an emphasis on advanced value iteration (VI) schemes for discrete-time nonlinear systems. The theoretical results focus on convergence and stability properties for general VI, stabilizing VI, integrated VI, evolving VI, adjustable VI schemes and so on. Several significant applications are also elaborated in aspects of optimal regulation, optimal tracking, and zero-sum games. We aim to break through the bottleneck problems for VI algorithms in realizing evolving control, accelerating learning speed, and reducing the calculation expense. In addition, the prospects of new theoretical and technical fields for advanced VI schemes are looked ahead. [ABSTRACT FROM AUTHOR]

When the equipment is working, it is very important to avoid the occurrence of malignant accidents by providing a highly reliable safety protection means. In this paper, for multiplayer mixed zero-sum games, an optimal safety tracking control scheme based on adaptive dynamic programming (ADP) is proposed, and a control barrier function (CBF) is introduced into the value function of the system to ensure that the system operates within its safe region. Firstly, through system transformation, the original tracking problem is transformed into a state tracking error problem. Secondly, an augmented Hamilton-Jacobi-Bellman (HJB) equation is derived from the improved augmented error system and the value function. Different from traditional methods, this paper uses a single critic neural network (NN) instead of the actor-critic NN to approximate the Nash equilibrium solution of the system, and introduces a concurrent learning technique that can relax the traditional continuous excitation condition into a simplified condition of recording data. Then, according to the Lyapunov theory, the stability of the system is analyzed in detail. Finally, two simulation examples are used to verify the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

A novel policy gradient (PG) adaptive dynamic programming method is developed to deal with nonlinear discrete-time zero-sum games with unknown dynamics. To facilitate the implementation, a policy iteration algorithm is established to approximate the iterative Q-function, as well as the control and disturbance policies via three neural network (NN) approximators, respectively. Then, the iterative Q-function is exploited to update the control and disturbance policies via PG method. To stabilize the training process and improve the data usage efficiency, the experience replay technique is applied to train the weight vectors of the three NNs by using mini-batch empirical data from replay memory. Furthermore, the convergence in terms of the iterative Q-function is proved. Simulation results of two numerical examples are provided to show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper designs a novel event-based adaptive learning method for solving zero-sum games (ZSGs) of nonlinear systems with incomplete known dynamics. Firstly, a discounted cost is introduced for the system with nonzero equilibrium point to obtain the near-optimal strategy pair. Then, the employment of integral reinforcement learning (IRL) makes it unnecessary to acquire the model of the drift dynamics. To approximate the solution of the Hamilton-Jacobi-Isaacs equations (HJIEs), single-critic network is constructed with the modified tuning law utilizing preprocessed data. For purpose of increasing algorithm efficiency, the event-triggered mechanism (ETM) is introduced which could obviate Zeno behavior. Furthermore, the state and critic weight vector error are proved to be uniform ultimate bounded (UUB) through Lyapunov approach. Finally, the effectiveness of the proposed method is validated by conducting a simulation experiment. [ABSTRACT FROM AUTHOR]

We apply dynamic programming principle to discuss two optimal investment problems by using zero-sum and nonzero-sum stochastic game approaches in a continuous-time Markov regime-switching environment within the frame work of behavioral finance. We represent different states of an economy and, consequently, investors' floating levels of psychological reactions by a D-state Markov chain. The first application is a zero-sum game between an investor and the market, and the second one formulates a nonzero-sum stochastic differential portfolio game as the sensitivity of two investors' terminal gains. We derive regime-switching Hamilton–Jacobi–Bellman–Isaacs equations and obtain explicit optimal portfolio strategies with Feynman–Kac representations of value functions. We illustrate our results in a two-state special case and observe the impact of regime switches by comparative results. [ABSTRACT FROM AUTHOR]