Search

When the equipment is working, it is very important to avoid the occurrence of malignant accidents by providing a highly reliable safety protection means. In this paper, for multiplayer mixed zero-sum games, an optimal safety tracking control scheme based on adaptive dynamic programming (ADP) is proposed, and a control barrier function (CBF) is introduced into the value function of the system to ensure that the system operates within its safe region. Firstly, through system transformation, the original tracking problem is transformed into a state tracking error problem. Secondly, an augmented Hamilton-Jacobi-Bellman (HJB) equation is derived from the improved augmented error system and the value function. Different from traditional methods, this paper uses a single critic neural network (NN) instead of the actor-critic NN to approximate the Nash equilibrium solution of the system, and introduces a concurrent learning technique that can relax the traditional continuous excitation condition into a simplified condition of recording data. Then, according to the Lyapunov theory, the stability of the system is analyzed in detail. Finally, two simulation examples are used to verify the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton–Jacobi–Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor–critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach. [ABSTRACT FROM AUTHOR]

A complex system exhibiting uncertainty as well as randomness may be portrayed by an uncertain random singular difference equation. This paper investigates two‐person nonzero‐sum and zero‐sum games based on uncertain random singular difference equations. First, an approach is proposed to translate the two‐person nonzero‐sum game into an equivalent game for a standard uncertain random dynamic system. The relevant recursive equations are developed to search the Nash equilibrium for the converted game. Solving the related recursive equations yields the solution to such a game. Following that, a Max‐Min Theorem is provided for finding the saddle‐point equilibrium of an uncertain random two‐person zero‐sum game. Finally, a numerical example is offered to demonstrate the validity of the findings. [ABSTRACT FROM AUTHOR]

In this paper, an event-triggered adaptive dynamic programming (ADP) method is proposed to deal with the H ∞ problem with unknown dynamic and constrained input. Firstly, the H ∞ -constrained problem is regarded as the two-player zero-sum game with the nonquadratic value function. Secondly, we develop the event-triggered Hamilton–Jacobi–Isaacs(HJI) equation, and an event-triggered ADP method is proposed to solve the HJI equation, which is equivalent to solving the Nash saddle point of the zero-sum game. An event-based single-critic neural network (NN) is applied to obtain the optimal value function, which reduces the communication resource and computational cost of algorithm implementation. For the event-triggered control, a triggering condition with the level of disturbance attenuation is developed to limit the number of sampling states, and the condition avoids Zeno behavior by proving the existence of events with minimum triggering interval. It is proved theoretically that the closed-loop system is asymptotically stable, and the critic NN weight error is uniformly ultimately boundedness (UUB). The learning performance of the proposed algorithm is verified by two examples. [ABSTRACT FROM AUTHOR]

In this paper, the guidance system with unknown dynamics is modeled as a partially unknown zero-sum differential game system. Then, a periodic event-triggered optimal control algorithm is designed to interception target under a plug-n-play framework. To realize this algorithm, generalized fuzzy hyperbolic models are employed to construct the identifier-critic structure, where the online identifier is used to estimate unknown dynamics, meanwhile, the generalized fuzzy hyperbolic models-based critic network is utilized to approximate the cost function. Note that plug-n-play framework lets both the designed identifier and critic network work simultaneously, in other words, the prior system information is no longer required, which simplifies the network structure. Using the Lyapunov function method, the approximate optimal control strategy and corresponding weight updating laws are derived to guarantee that the closed-loop system and weight approximation errors are uniformly ultimately bounded, where an additional function is added to weight updating laws to release the requirement for admissible initial control. Finally, to compare the intercept effects and the utilization ratio of communication resources of the periodic event-triggered control algorithm and the common adaptive dynamic programming algorithm, the missile interception system is introduced as an example. • The guidance system with unknown dynamics is modeled as a partially unknown zero-sum differential game system. Then, the event-based optimal control strategy is designed to improve the utilization of data transmissions. • A GFHM-based identifier is proposed under a plug-n-play framework to approximate unknown nonlinear functions and eliminate the priori identification process. • This paper proposes the fuzzy ADP approach to design the optimal controller, where the single critic network is adopted to approximate the optimal performance index by using GFHMs. [ABSTRACT FROM AUTHOR]

The game of darts has enjoyed great growth over the past decade with the perception of darts moving from that of a pub game to a game that is regularly scheduled on prime-time television in many countries such as the United Kingdom, Germany, the Netherlands, and Australia, among others. It involves strategic interactions between two players, but to date, the literature has ignored these interactions. In this paper, we formulate and solve the game of darts as a dynamic zero-sum game (ZSG), and to the best of our knowledge, we are the first to do so. We also estimate individual skill models using a novel data set based on darts matches that were played by the top 16 professional players in the world during the 2019 season. Using the fitted skill models and our ZSG problem formulation, we quantify the importance of playing strategically—that is, taking into account the score and strategy of one's opponent—when computing an optimal strategy. For top professionals, we find that playing strategically results in an increase in win probability of just 0.2%–0.6% over a single leg but as much as 2.2% over a best-of-31-legs match. Summary of Contribution: Dynamic zero-sum games (ZSGs) are of considerable interest, as they arise in many applications including sports, the management of communication networks, interdiction games, and heads-up poker—an important topic in modern artificial intelligence. In this study we consider the game of darts, which is growing increasingly popular around the world today. We formulate the game of darts as a ZSG and solve it iteratively by formulating each player's best-response problem as a stochastic shortest-path (SSP) problem. We then solve these SSPs using standard dynamic programming methods. In solving the ZSG, we are able to accurately quantify the importance of top professionals playing strategically. [ABSTRACT FROM AUTHOR]

In order to address zero-sum game problems for discrete-time (DT) nonlinear systems, this article develops a novel event-triggered control (ETC) approach based on the deterministic policy gradient (PG) adaptive dynamic programming (ADP) algorithm. By adopting the input and output data, the proposed ETC method updates the control law and the disturbance law with a gradient descent algorithm. Compared with the conventional PG ADP-based control scheme, the present controller is updated aperiodically to reduce the computational and communication burden. Then, the actor-critic-disturbance framework is adopted to obtain the optimal control law and the worst disturbance law, which guarantee the input-to-state stability of the closed-loop system. Moreover, a novel neural network weight updating law which guarantees the uniform ultimate boundedness of weight estimation errors is provided based on the experience replay technique. Finally, the validity of the present method is verified by simulation of two DT nonlinear systems. [ABSTRACT FROM AUTHOR]

In this paper, we consider the two-player state and control path-dependent stochastic zero-sum differential game. In our problem setup, the state process, which is controlled by the players, is dependent on (current and past) paths of state and control processes of the players. Furthermore, the running cost of the objective functional depends on both state and control paths of the players. We use the notion of non-anticipative strategies to define lower and upper value functionals of the game, where unlike the existing literature, these value functions are dependent on the initial states and control paths of the players. In the first main result of this paper, we prove that the (lower and upper) value functionals satisfy the dynamic programming principle (DPP), for which unlike the existing literature, the Skorohod metric is necessary to maintain the separability of càdlàg (state and control) spaces. We introduce the lower and upper Hamilton–Jacobi–Isaacs (HJI) equations from the DPP, which correspond to the state and control path-dependent nonlinear second-order partial differential equations. In the second main result of this paper, we show that by using the functional Itô calculus, the lower and upper value functionals are viscosity solutions of (lower and upper) state and control path-dependent HJI equations, where the notion of viscosity solutions is defined on a compact κ -Hölder space to use several important estimates and to guarantee the existence of minimum and maximum points between the (lower and upper) value functionals and the test functions. Based on these two main results, we also show that the Isaacs condition and the uniqueness of viscosity solutions imply the existence of the game value. Finally, we prove the uniqueness of classical solutions for the (state path-dependent) HJI equations in the state path-dependent case, where its proof requires establishing an equivalent classical solution structure as well as an appropriate contradiction argument. [ABSTRACT FROM AUTHOR]

We apply dynamic programming principle to discuss two optimal investment problems by using zero-sum and nonzero-sum stochastic game approaches in a continuous-time Markov regime-switching environment within the frame work of behavioral finance. We represent different states of an economy and, consequently, investors' floating levels of psychological reactions by a D-state Markov chain. The first application is a zero-sum game between an investor and the market, and the second one formulates a nonzero-sum stochastic differential portfolio game as the sensitivity of two investors' terminal gains. We derive regime-switching Hamilton–Jacobi–Bellman–Isaacs equations and obtain explicit optimal portfolio strategies with Feynman–Kac representations of value functions. We illustrate our results in a two-state special case and observe the impact of regime switches by comparative results. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of fault tolerant control (FTC) for reentry vehicles (RV) subject to actuator fault and perturbation. In contrast to the majority of papers assuming bounded and exogenous actuator fault effect, the adaptive dynamic programming (ADP) control algorithm has been considered here to tackle with the dependence of closed-loop states induced by the setup of an active FTC strategy. By this way, the FTC problem formulation takes theoretically into account the fact that an actuator fault can destabilize a closed-loop. First, the FTC problem in attitude tracking is transformed into an error-optimal control problem. Next, a cost function based on zero-sum game theory is considered to achieve tracking under the joint influence of state-dependent actuator faults and perturbations. Lyapunov theory has been next used to prove the stability of FTC architecture and the network weight convergence. Benefits of proposed ADP-FTC algorithm is finally highlighted on a numerical benchmark of RV. [ABSTRACT FROM AUTHOR]