Search

In this paper, an optimisation problem for the monotone mean-variance (MMV) criterion is considered from the perspective of the insurance company. The MMV criterion is an amended version of the classical mean-variance (MV) criterion which guarantees the monotonicity of the utility function. With this criterion we study the optimal investment and reinsurance problem which is formulated as a zero-sum game between the insurance company and an imaginary player. We apply the dynamic programming principle to obtain the corresponding Hamilton–Jacobi–Bellman–Isaacs (HJBI) equation. As the main conclusion of this paper, by solving the HJBI equation explicitly, the closed forms of the optimal strategy and the value function are obtained. Moreover, the MMV efficient frontier is also provided. At the end of the paper, a numerical example is presented. [ABSTRACT FROM AUTHOR]

This paper investigates the robust H∞$$ {H}_{\infty } $$ optimal formation control for the position subsystem of quadrotor unmanned aerial vehicles (UAVs) subject to external disturbances and collision constraints. To prevent collision with both members of the formation and external obstacles, a collision avoidance potential function is constructed using relative position and velocity information. The basic bounded control input can ensure the stable flight and collision avoidance of a quadrotor UAV system. Based on the approximate dynamic programming (ADP) framework and two‐player zero‐sum differential game theory, the H∞$$ {H}_{\infty } $$ optimal controller is designed to further enhance the control performance of the system. The optimal value function is approximated by a single layer neural network, which avoids solving complex nonlinear Hamilton‐JacobiIsaac (HJI) equation. The stability of the closed loop system is proved. The effectiveness of the robust H∞$$ {H}_{\infty } $$ optimal formation controller based on learning is validated through MATLAB simulation in two distinct scenarios. [ABSTRACT FROM AUTHOR]

We consider a general type of non-Markovian impulse control problems under adverse non-linear expectation or, more specifically, the zero-sum game problem where the adversary player decides the probability measure. We show that the upper and lower value functions satisfy a dynamic programming principle (DPP). We first prove the dynamic programming principle (DPP) for a truncated version of the upper value function in a straightforward manner. Relying on a uniform convergence argument then enables us to show the DPP for the general setting. Following this, we use an approximation based on a combination of truncation and discretization to show that the upper and lower value functions coincide, thus establishing that the game has a value and that the DPP holds for the lower value function as well. Finally, we show that the DPP admits a unique solution and give conditions under which a saddle point for the game exists. As an example, we consider a stochastic differential game (SDG) of impulse versus classical control of path-dependent stochastic differential equations (SDEs). [ABSTRACT FROM AUTHOR]

In this article, we present an iterative algorithm based on differential dynamic programming (DDP) for finite‐horizon two‐person zero‐sum differential games. The technique of DDP is used to expand the Hamilton–Jacobi–Isaacs (HJI) partial differential equation into higher‐order differential equations. Using value function and saddle point approximations, the DDP expansion is transformed into algebraic matrix equation in integral form. Based on the algebraic matrix equation, a DDP iterative algorithm is developed to learn the solution to the differential games. Strict proof is proposed to guarantee the iterative convergences of the value function and saddle point. The new algorithm is fundamentally different from existing results, in the sense that it overcome the technical obstacle to address the time‐varying behavior of HJI partial differential equation. Simulation examples are given to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Summary: In this article, a novel event‐triggered zero‐sum differential game through value iteration (VI)‐based adaptive fuzzy control is proposed for modular robot manipulators (MRMs) with uncertain disturbance. The subsystem dynamic of MRMs is established by joint torque feedback (JTF) technology. An identifier based on fuzzy logic model (FLM) is established to identify unknown system dynamics. Then, the trajectory tracking control issue of MRMs with uncertain disturbance is transformed into a two‐player zero‐sum differential game. On the basis of event‐triggered mechanism and VI algorithm, the event‐triggered Hamilton–Jacobi–Issacs (HJI) equation is approximately solved through establishment of actor‐critic structure. Different from the traditional adaptive dynamic programming (ADP) algorithm, this scheme is based on adaptive fuzzy optimal control through an actor‐critic structure, which refers to adaptation of both optimal cost function and optimal control policy. Moreover, tracking errors of closed‐loop MRM system are demonstrated to be uniformly ultimately bounded (UUB) under Lyapunov stability theorem. At last, experimental examples and comparisons show the reliability and effectiveness of this method. [ABSTRACT FROM AUTHOR]

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton–Jacobi–Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor–critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive critic control method is designed to solve an optimal H ∞ tracking control problem for continuous nonlinear systems with nonzero equilibrium based on adaptive dynamic programming (ADP). To guarantee the finiteness of a cost function, traditional methods generally assume that the controlled system has a zero equilibrium point, which is not true in practical systems. In order to overcome such obstacle and realize H ∞ optimal tracking control, this paper proposes a novel cost function design with respect to disturbance, tracking error and the derivative of tracking error. Based on the designed cost function, the H ∞ control problem is formulated as two-player zero-sum differential games, and then a policy iteration (PI) algorithm is proposed to solve the corresponding Hamilton–Jacobi–Isaacs (HJI) equation. In order to obtain the online solution to the HJI equation, a single-critic neural network structure based on PI algorithm is established to learn the optimal control policy and the worst-case disturbance law. It is worth mentioning that the proposed adaptive critic control method can simplify the controller design process when the equilibrium of the systems is not zero. Finally, simulations are conducted to evaluate the tracking performance of the proposed control methods. [ABSTRACT FROM AUTHOR]

A novel policy gradient (PG) adaptive dynamic programming method is developed to deal with nonlinear discrete-time zero-sum games with unknown dynamics. To facilitate the implementation, a policy iteration algorithm is established to approximate the iterative Q-function, as well as the control and disturbance policies via three neural network (NN) approximators, respectively. Then, the iterative Q-function is exploited to update the control and disturbance policies via PG method. To stabilize the training process and improve the data usage efficiency, the experience replay technique is applied to train the weight vectors of the three NNs by using mini-batch empirical data from replay memory. Furthermore, the convergence in terms of the iterative Q-function is proved. Simulation results of two numerical examples are provided to show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

A complex system exhibiting uncertainty as well as randomness may be portrayed by an uncertain random singular difference equation. This paper investigates two‐person nonzero‐sum and zero‐sum games based on uncertain random singular difference equations. First, an approach is proposed to translate the two‐person nonzero‐sum game into an equivalent game for a standard uncertain random dynamic system. The relevant recursive equations are developed to search the Nash equilibrium for the converted game. Solving the related recursive equations yields the solution to such a game. Following that, a Max‐Min Theorem is provided for finding the saddle‐point equilibrium of an uncertain random two‐person zero‐sum game. Finally, a numerical example is offered to demonstrate the validity of the findings. [ABSTRACT FROM AUTHOR]

In this paper, the position–force–based approximate optimal control method is developed for reconfigurable robots using zero-sum game strategy. By utilizing the Newton–Euler iteration technique, the robotic system's dynamic model is formulated and the state space equation is derived. According to adaptive dynamic programming (ADP) and neural network algorithm, the trajectory tracking control problem is transformed into a zero-sum game-based optimal control issue. The optimal control policy and worst disturbance policy are obtained by Hamilton–Jacobi–Issacs (HJI) function, respectively. Unlike the conventional learning–based robotic control method, the proposed zero-sum game-based method no need extra sub-controller that can reduce the computational burden. The reconfigurable robot system's tracking error is uniformly ultimately bounded by the Lyapunov theorem. Finally, simulation experiments demonstrate the advantages of the proposed method. [ABSTRACT FROM AUTHOR]

In this article, the H∞$$ {H}_{\infty } $$ safe control problem of continuous‐time affine nonlinear safety‐critical systems is studied based on the barrier function (BF) and adaptive dynamic programming (ADP). We show that the safety constraints in this article occur under the condition that the system initial state is unsafe and have not been adequately addressed in the existing work. First, the H∞$$ {H}_{\infty } $$ control problem is transformed into a zero‐sum game problem, and a new safe Hamilton–Jacobi–Isaacs equation is proposed by combining with the BF, which makes the unsafe behavior be punished in the learning process. In addition, a damping coefficient is introduced into the BF to punish the unsafe behavior more flexibly. Aiming at the requirement that the initial system state must be strictly constrained in the safe set, a new weight updating method based on ADP is proposed, which can reasonably avoid the influence of the BF on neural network (NN) parameters when the initial system state is unsafe. Furthermore, based on the Lyapunov stability theory, it is proved that the system states of safety‐critical systems and the NN parameters are uniformly ultimately bounded under the safety constraints and disturbances effect. Finally, the effectiveness of the proposed method is verified by two simulation examples. [ABSTRACT FROM AUTHOR]

Summary: This article presents a multiplayer zero‐sum game optimal control for modular robot manipulators (MRMs) via adaptive dynamic programming (ADP). The dynamic model of MRMs with interconnected dynamic couplings (IDCs) is formulated. The control law and IDC effect are regarded as players in each module. The optimal tracking control problem of the MRM system can be transformed into a multiplayer zero‐sum game. Base on ADP algorithm, the Hamilton–Jacobi–Issacs (HJI) equation is solved via critic neural network. Then, the optimal control law can be derivated. The closed‐loop robotic system is proved to be asymptotic stable and experiments are conducted to verify the effectiveness of the control method. [ABSTRACT FROM AUTHOR]

The game of darts has enjoyed great growth over the past decade with the perception of darts moving from that of a pub game to a game that is regularly scheduled on prime-time television in many countries such as the United Kingdom, Germany, the Netherlands, and Australia, among others. It involves strategic interactions between two players, but to date, the literature has ignored these interactions. In this paper, we formulate and solve the game of darts as a dynamic zero-sum game (ZSG), and to the best of our knowledge, we are the first to do so. We also estimate individual skill models using a novel data set based on darts matches that were played by the top 16 professional players in the world during the 2019 season. Using the fitted skill models and our ZSG problem formulation, we quantify the importance of playing strategically—that is, taking into account the score and strategy of one's opponent—when computing an optimal strategy. For top professionals, we find that playing strategically results in an increase in win probability of just 0.2%–0.6% over a single leg but as much as 2.2% over a best-of-31-legs match. Summary of Contribution: Dynamic zero-sum games (ZSGs) are of considerable interest, as they arise in many applications including sports, the management of communication networks, interdiction games, and heads-up poker—an important topic in modern artificial intelligence. In this study we consider the game of darts, which is growing increasingly popular around the world today. We formulate the game of darts as a ZSG and solve it iteratively by formulating each player's best-response problem as a stochastic shortest-path (SSP) problem. We then solve these SSPs using standard dynamic programming methods. In solving the ZSG, we are able to accurately quantify the importance of top professionals playing strategically. [ABSTRACT FROM AUTHOR]

In order to address zero-sum game problems for discrete-time (DT) nonlinear systems, this article develops a novel event-triggered control (ETC) approach based on the deterministic policy gradient (PG) adaptive dynamic programming (ADP) algorithm. By adopting the input and output data, the proposed ETC method updates the control law and the disturbance law with a gradient descent algorithm. Compared with the conventional PG ADP-based control scheme, the present controller is updated aperiodically to reduce the computational and communication burden. Then, the actor-critic-disturbance framework is adopted to obtain the optimal control law and the worst disturbance law, which guarantee the input-to-state stability of the closed-loop system. Moreover, a novel neural network weight updating law which guarantees the uniform ultimate boundedness of weight estimation errors is provided based on the experience replay technique. Finally, the validity of the present method is verified by simulation of two DT nonlinear systems. [ABSTRACT FROM AUTHOR]

In this paper, an adjustable Q-learning scheme is developed to solve the discrete-time nonlinear zero-sum game problem, which can accelerate the convergence rate of the iterative Q-function sequence. First, the monotonicity and convergence of the iterative Q-function sequence are analyzed under some conditions. Moreover, by employing neural networks, the model-free tracking control problem can be overcome for zero-sum games. Second, two practical algorithms are designed to guarantee the convergence with accelerated learning. In one algorithm, an adjustable acceleration phase is added to the iteration process of Q-learning, which can be adaptively terminated with convergence guarantee. In another algorithm, a novel acceleration function is developed, which can adjust the relaxation factor to ensure the convergence. Finally, through a simulation example with the practical physical background, the fantastic performance of the developed algorithm is demonstrated with neural networks. [ABSTRACT FROM AUTHOR]

This paper designs a novel adaptive robust optimal sliding mode control (SMC) strategy for constrained input missile system with uncertain input and unknown disturbance. First, a smooth function is employed to map the constrained input. Based on SMC theory, a fast adaptive SMC scheme is designed to reduce the effects of unknown input and approximate error which has two advantages. One is that the proposed scheme is independent of the uncertain information. The other is that the proposed scheme forces the states to converge with exponential rate as they are far away from the sliding mode surface, and the states move with linear rate when they are closed to the sliding mode surface. Then, a zero-sum differential games strategy-based adaptive dynamic programming (ADP) technique is employed to obtain the optimal performance of sliding mode dynamics system. The loop-system and estimation weight error can be proved stable by utilising Lyapunov theory. Finally, the missile autopilot system is used to indicate the effectiveness of the designed approach. [ABSTRACT FROM AUTHOR]

This paper designs a novel event-based adaptive learning method for solving zero-sum games (ZSGs) of nonlinear systems with incomplete known dynamics. Firstly, a discounted cost is introduced for the system with nonzero equilibrium point to obtain the near-optimal strategy pair. Then, the employment of integral reinforcement learning (IRL) makes it unnecessary to acquire the model of the drift dynamics. To approximate the solution of the Hamilton-Jacobi-Isaacs equations (HJIEs), single-critic network is constructed with the modified tuning law utilizing preprocessed data. For purpose of increasing algorithm efficiency, the event-triggered mechanism (ETM) is introduced which could obviate Zeno behavior. Furthermore, the state and critic weight vector error are proved to be uniform ultimate bounded (UUB) through Lyapunov approach. Finally, the effectiveness of the proposed method is validated by conducting a simulation experiment. [ABSTRACT FROM AUTHOR]

We apply dynamic programming principle to discuss two optimal investment problems by using zero-sum and nonzero-sum stochastic game approaches in a continuous-time Markov regime-switching environment within the frame work of behavioral finance. We represent different states of an economy and, consequently, investors' floating levels of psychological reactions by a D-state Markov chain. The first application is a zero-sum game between an investor and the market, and the second one formulates a nonzero-sum stochastic differential portfolio game as the sensitivity of two investors' terminal gains. We derive regime-switching Hamilton–Jacobi–Bellman–Isaacs equations and obtain explicit optimal portfolio strategies with Feynman–Kac representations of value functions. We illustrate our results in a two-state special case and observe the impact of regime switches by comparative results. [ABSTRACT FROM AUTHOR]

In this paper, we consider the two-player state and control path-dependent stochastic zero-sum differential game. In our problem setup, the state process, which is controlled by the players, is dependent on (current and past) paths of state and control processes of the players. Furthermore, the running cost of the objective functional depends on both state and control paths of the players. We use the notion of non-anticipative strategies to define lower and upper value functionals of the game, where unlike the existing literature, these value functions are dependent on the initial states and control paths of the players. In the first main result of this paper, we prove that the (lower and upper) value functionals satisfy the dynamic programming principle (DPP), for which unlike the existing literature, the Skorohod metric is necessary to maintain the separability of càdlàg (state and control) spaces. We introduce the lower and upper Hamilton–Jacobi–Isaacs (HJI) equations from the DPP, which correspond to the state and control path-dependent nonlinear second-order partial differential equations. In the second main result of this paper, we show that by using the functional Itô calculus, the lower and upper value functionals are viscosity solutions of (lower and upper) state and control path-dependent HJI equations, where the notion of viscosity solutions is defined on a compact κ -Hölder space to use several important estimates and to guarantee the existence of minimum and maximum points between the (lower and upper) value functionals and the test functions. Based on these two main results, we also show that the Isaacs condition and the uniqueness of viscosity solutions imply the existence of the game value. Finally, we prove the uniqueness of classical solutions for the (state path-dependent) HJI equations in the state path-dependent case, where its proof requires establishing an equivalent classical solution structure as well as an appropriate contradiction argument. [ABSTRACT FROM AUTHOR]

This paper investigates the fuzzy approximation-based optimal consensus control problem for nonlinear multiagent systems with unknown perturbations. By constructing local error dynamics, the considered optimal consensus problem is reformulated as finding Nash-equilibrium solutions to zero-sum games. Then, by using sliding mode control technology and the concept of hierarchical design, a series of control signals are sequentially designed to regulate the consensus error and minimize the local value function. In addition, an identifier-critic architecture is developed by using generalized fuzzy hyperbolic models, where the identifier is employed to relax the requirement for complete system dynamics information, and the hierarchical sliding mode surface-based critic network is applied to approximate optimal control inputs. Finally, A simulation example is presented to illustrate the validity of the proposed approach. [ABSTRACT FROM AUTHOR]

The paper addresses the problem of fault tolerant control (FTC) for reentry vehicles (RV) subject to actuator fault and perturbation. In contrast to the majority of papers assuming bounded and exogenous actuator fault effect, the adaptive dynamic programming (ADP) control algorithm has been considered here to tackle with the dependence of closed-loop states induced by the setup of an active FTC strategy. By this way, the FTC problem formulation takes theoretically into account the fact that an actuator fault can destabilize a closed-loop. First, the FTC problem in attitude tracking is transformed into an error-optimal control problem. Next, a cost function based on zero-sum game theory is considered to achieve tracking under the joint influence of state-dependent actuator faults and perturbations. Lyapunov theory has been next used to prove the stability of FTC architecture and the network weight convergence. Benefits of proposed ADP-FTC algorithm is finally highlighted on a numerical benchmark of RV. [ABSTRACT FROM AUTHOR]

This paper considers a new class of deterministic finite-time horizon, two-player, zero-sum differential games (DGs) in which the maximizing player is allowed to take continuous and impulse controls whereas the minimizing player is allowed to take impulse control only. We seek to approximate the value function, and to provide a verification theorem for this class of DGs. By means of dynamic programming principle (DPP) in viscosity solution (VS) framework, we first characterize the value function as the unique VS to the related Hamilton–Jacobi-Bellman–Isaacs (HJBI) double-obstacle equation. Next, we prove that an approximate value function exists, that it is the unique solution to an approximate HJBI double-obstacle equation, and converges locally uniformly towards the value function of each player when the time discretization step goes to zero. Moreover, we provide a verification theorem which characterizes a Nash-equilibrium (NE) for the DG control problem considered. Finally, by applying our results, we derive a new continuous-time portfolio optimization model, and we provide related computational algorithms and numerical results. [ABSTRACT FROM AUTHOR]