Search

The well-known H ∞ control design gives robustness to a controller by rejecting perturbations from the external environment, which is difficult to do for completely unknown affine nonlinear systems. Accordingly, the immediate objective of this paper is to develop an on-line real-time synergetic learning algorithm, so that a data-driven H ∞ controller can be received. By converting the H ∞ control problem into a two-player zero-sum game, a model-free Hamilton–Jacobi–Isaacs equation (MF-HJIE) is first derived using off-policy reinforcement learning, followed by a proof of equivalence between the MF-HJIE and the conventional HJIE. Next, by applying the temporal difference to the MF-HJIE, a synergetic evolutionary rule with experience replay is designed to learn the optimal value function, the optimal control, and the worst perturbation, that can be performed on-line and in real-time along the system state trajectory. It is proven that the synergistic learning system constructed by the system plant and the evolutionary rule is uniformly ultimately bounded. Finally, simulation results on an F16 aircraft system and a nonlinear system back up the tractability of the proposed method. • A model-free Hamiltonian–Jacobi–Isaacs equation is derived for H ∞ control problems. • A real-time on-line evolutionary rule for tuning neural network weights is developed. • The system and neural network approximation errors achieve synergetic learning. [ABSTRACT FROM AUTHOR]

In this paper, a novel parallel learning framework is developed to solve zero-sum games for discrete-time nonlinear systems. Briefly, the purpose of this study is to determine a tentative function according to the prior knowledge of the value iteration (VI) algorithm. The learning process of the parallel controllers can be guided by the tentative function. That is to say, the neighborhood of the optimal cost function can be compressed within a small range via two typical exploration policies. Based on the parallel learning framework, a novel dichotomy VI algorithm is established to accelerate the learning speed. It is shown that the parallel controllers will converge to the optimal policy from contrary initial policies. Finally, two typical systems are used to demonstrate the learning performance of the constructed dichotomy VI algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we present two-timescale neurodynamic optimization approaches to distributed minimax optimization. We propose four multilayer recurrent neural networks for solving four different types of generally nonlinear convex–concave minimax problems subject to linear equality and nonlinear inequality constraints. We derive sufficient conditions to guarantee the stability and optimality of the neural networks. We demonstrate the viability and efficiency of the proposed neural networks in two specific paradigms for Nash-equilibrium seeking in a zero-sum game and distributed constrained nonlinear optimization. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive critic control method is designed to solve an optimal H ∞ tracking control problem for continuous nonlinear systems with nonzero equilibrium based on adaptive dynamic programming (ADP). To guarantee the finiteness of a cost function, traditional methods generally assume that the controlled system has a zero equilibrium point, which is not true in practical systems. In order to overcome such obstacle and realize H ∞ optimal tracking control, this paper proposes a novel cost function design with respect to disturbance, tracking error and the derivative of tracking error. Based on the designed cost function, the H ∞ control problem is formulated as two-player zero-sum differential games, and then a policy iteration (PI) algorithm is proposed to solve the corresponding Hamilton–Jacobi–Isaacs (HJI) equation. In order to obtain the online solution to the HJI equation, a single-critic neural network structure based on PI algorithm is established to learn the optimal control policy and the worst-case disturbance law. It is worth mentioning that the proposed adaptive critic control method can simplify the controller design process when the equilibrium of the systems is not zero. Finally, simulations are conducted to evaluate the tracking performance of the proposed control methods. [ABSTRACT FROM AUTHOR]

In this paper, a synergetic learning structure-based neuro-optimal fault tolerant control (SLSNOFTC) method is proposed for unknown nonlinear continuous-time systems with actuator failures. Under the framework of the synergetic learning structure (SLS), the optimal control input and the actuator failure are viewed as two subsystems. Then, the fault tolerant control (FTC) problem can be regarded as a two-player zero-sum differential game according to the game theory. A radial basis function neural network-based identifier, which uses the measured input/output data, is constructed to identify the completely unknown system dynamics. To develop the SLSNOFTC method, the Hamilton–Jacobi–Isaacs equation is solved by an asymptotically stable critic neural network (ASCNN) which is composed of cooperative adaptive tuning laws. Besides, with the help of the Lyapunov stability analysis, the identification error, the weight error of ASCNN, and all signals of closed-loop system are guaranteed to be converged to zero asymptotically, rather than uniformly ultimately bounded. Numerical simulation examples further verify the effectiveness and reliability of the proposed method. [ABSTRACT FROM AUTHOR]

This paper investigates the optimal tracking issue for continuous-time (CT) nonlinear asymmetric constrained zero-sum games (ZSGs) by exploiting the neural critic technique. Initially, an improved algorithm is constructed to tackle the tracking control problem of nonlinear CT multiplayer ZSGs. Also, we give a novel nonquadratic function to settle the asymmetric constraints. One thing worth noting is that the method used in this paper to solve asymmetric constraints eliminates the strict restriction on the control matrix compared to the previous ones. Further, the optimal controls, the worst disturbances, and the tracking Hamilton–Jacobi–Isaacs equation are derived. Next, a single critic neural network is built to estimate the optimal cost function, thus obtaining the approximations of the optimal controls and the worst disturbances. The critic network weight is updated by the normalized steepest descent algorithm. Additionally, based on the Lyapunov method, the stability of the tracking error and the weight estimation error of the critic network is analyzed. In the end, two examples are offered to validate the theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, an adjustable Q-learning scheme is developed to solve the discrete-time nonlinear zero-sum game problem, which can accelerate the convergence rate of the iterative Q-function sequence. First, the monotonicity and convergence of the iterative Q-function sequence are analyzed under some conditions. Moreover, by employing neural networks, the model-free tracking control problem can be overcome for zero-sum games. Second, two practical algorithms are designed to guarantee the convergence with accelerated learning. In one algorithm, an adjustable acceleration phase is added to the iteration process of Q-learning, which can be adaptively terminated with convergence guarantee. In another algorithm, a novel acceleration function is developed, which can adjust the relaxation factor to ensure the convergence. Finally, through a simulation example with the practical physical background, the fantastic performance of the developed algorithm is demonstrated with neural networks. [ABSTRACT FROM AUTHOR]

This paper aims at solving a stochastic two-player zero-sum Nash game problem studied in Singh and Lisser (2019). The main contribution of our paper is that we model this game problem as a dynamical neural network (DNN for short). In this paper, we show that the saddle point of this game problem is the equilibrium point of the DNN model, and we study the globally asymptotically stable of the DNN model. In our numerical experiments, we present the time-continuous feature of the DNN model and compare it with the state-of-the-art convex solvers, i.e., Splitting conic solver (SCS for short) and Cvxopt. Our numerical results show that our DNN method has two advantages in dealing with this game problem. Firstly, the DNN model can converge to a better optimal point. Secondly, the DNN method can solve all problems, even when the problem size is large. [ABSTRACT FROM AUTHOR]