Search

This paper primarily investigates a neural-network-based safe control scheme for solving the optimal control problem of continuous-time (CT) nonlinear systems with asymmetric input constraints under non-zero-sum (NZS) differential game scenarios. Initially, by constructing a novel non-quadratic function, the issue of asymmetric input constraints in the non-zero-sum differential game controllers is addressed. Subsequently, the safe Hamilton-Jacobi-Bellman (HJB) equation is derived from the direct integration of the control barrier function (CBF) into the traditional cost function, ensuring that the system states remain within a safe region. Then, the safe learning control scheme based on single critic neural network (NN) and adaptive dynamic programming (ADP) is proposed to approximate the optimal control strategy, differing from the dual-network update method commonly used in traditional ADP. Based on the constructed neural network weight adjustment rules, the optimal solution to the HJB equation can be derived within the safe learning control framework. Following this, Lyapunov's stability theory demonstrates that the errors in neural network weights and all signals within the closed-loop system are uniformly ultimately bounded (UUB). Finally, the effectiveness of the developed neural-network-based safe learning control method is validated through two simulation results. [ABSTRACT FROM AUTHOR]

When the equipment is working, it is very important to avoid the occurrence of malignant accidents by providing a highly reliable safety protection means. In this paper, for multiplayer mixed zero-sum games, an optimal safety tracking control scheme based on adaptive dynamic programming (ADP) is proposed, and a control barrier function (CBF) is introduced into the value function of the system to ensure that the system operates within its safe region. Firstly, through system transformation, the original tracking problem is transformed into a state tracking error problem. Secondly, an augmented Hamilton-Jacobi-Bellman (HJB) equation is derived from the improved augmented error system and the value function. Different from traditional methods, this paper uses a single critic neural network (NN) instead of the actor-critic NN to approximate the Nash equilibrium solution of the system, and introduces a concurrent learning technique that can relax the traditional continuous excitation condition into a simplified condition of recording data. Then, according to the Lyapunov theory, the stability of the system is analyzed in detail. Finally, two simulation examples are used to verify the effectiveness of the proposed scheme. [ABSTRACT FROM AUTHOR]

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton–Jacobi–Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor–critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach. [ABSTRACT FROM AUTHOR]

In this article, the H∞$$ {H}_{\infty } $$ safe control problem of continuous‐time affine nonlinear safety‐critical systems is studied based on the barrier function (BF) and adaptive dynamic programming (ADP). We show that the safety constraints in this article occur under the condition that the system initial state is unsafe and have not been adequately addressed in the existing work. First, the H∞$$ {H}_{\infty } $$ control problem is transformed into a zero‐sum game problem, and a new safe Hamilton–Jacobi–Isaacs equation is proposed by combining with the BF, which makes the unsafe behavior be punished in the learning process. In addition, a damping coefficient is introduced into the BF to punish the unsafe behavior more flexibly. Aiming at the requirement that the initial system state must be strictly constrained in the safe set, a new weight updating method based on ADP is proposed, which can reasonably avoid the influence of the BF on neural network (NN) parameters when the initial system state is unsafe. Furthermore, based on the Lyapunov stability theory, it is proved that the system states of safety‐critical systems and the NN parameters are uniformly ultimately bounded under the safety constraints and disturbances effect. Finally, the effectiveness of the proposed method is verified by two simulation examples. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the constrained optimal control problem of nonlinear multi-input safety-critical systems with uncertain disturbances and time-varying safety constraints. By utilizing a barrier function transformation, together with a new disturbance-related term and a smooth safety boundary function, a nominal system-dependent multi-input barrier transformation architecture is developed to deal with the time-varying safety constraints and uncertain disturbances. Based on the obtained transformation system, the coupled Hamilton–Jacobi–Bellman (HJB) function is established to obtain the constrained Nash equilibrium solution. In addition, due to the fact that it is difficult to solve the HJB function directly, the single critic neural network (NN) is constructed to approximate the optimal performance index function of different control inputs, respectively. It is proved theoretically that, under the influence of uncertain disturbances and time-varying safety constraints, the system states and neural network parameters can be uniformly ultimately bounded (UUB) by the proposed neural network approximation method. Finally, the effectiveness of the proposed method is verified by two nonlinear simulation examples. [ABSTRACT FROM AUTHOR]

In this paper, a new adaptive critic design is proposed to approximate the online Nash equilibrium solution for the robust trajectory tracking control of non-zero-sum (NZS) games for continuous-time uncertain nonlinear systems. First, the augmented system was constructed by combining the tracking error and the reference trajectory. By modifying the cost function, the robust tracking control problem was transformed into an optimal tracking control problem. Based on adaptive dynamic programming (ADP), a single critic neural network (NN) was applied for each player to solve the coupled Hamilton–Jacobi–Bellman (HJB) equations approximately, and the obtained control laws were regarded as the feedback Nash equilibrium. Two additional terms were introduced in the weight update law of each critic NN, which strengthened the weight update process and eliminated the strict requirements for the initial stability control policy. More importantly, in theory, through the Lyapunov theory, the stability of the closed-loop system was guaranteed, and the robust tracking performance was analyzed. Finally, the effectiveness of the proposed scheme was verified by two examples. [ABSTRACT FROM AUTHOR]

In this paper, a robust trajectory tracking control method with state constraints and uncertain disturbances on the ground of adaptive dynamic programming (ADP) is proposed for nonlinear systems. Firstly, the augmented system consists of the tracking error and the reference trajectory, and the tracking control problems with uncertain disturbances is described as the problem of robust control adjustment. In addition, considering the nominal system of the augmented system, the guaranteed cost tracking control problem is transformed into the optimal control problem by using the discount coefficient in the nominal system. A new safe Hamilton–Jacobi–Bellman (HJB) equation is proposed by combining the cost function with the control barrier function (CBF), so that the behavior of violating the safety regulations for the system states will be punished. In order to solve the new safe HJB equation, a critic neural network (NN) is used to approximate the solution of the safe HJB equation. According to the Lyapunov stability theory, in the case of state constraints and uncertain disturbances, the system states and the parameters of the critic neural network are guaranteed to be uniformly ultimately bounded (UUB). At the end of this paper, the feasibility of the proposed method is verified by a simulation example. [ABSTRACT FROM AUTHOR]

In this paper, the problem of H ∞ control design for affine nonlinear discrete-time systems is addressed by using adaptive dynamic programming (ADP). First, the nonlinear H ∞ control problem is transformed into solving the two-player zero-sum differential game problem of the nonlinear system. Then, the critic, action and disturbance networks are designed by using neural networks to solve online the Hamilton–Jacobi–Isaacs (HJI) equation associating with the two-player zero-sum differential game. When novel weight update laws for the critic, action and disturbance networks are tuned online by using data generated in real-time along the system trajectories, it is shown that the system states, all neural networks weight estimation errors are uniformly ultimately bounded by using Lyapunov techniques. Further, it is shown that the output of the action network approaches the optimal control input with small bounded error and the output of the disturbance network approaches the worst disturbance with small bounded error. At last, simulation results are presented to demonstrate the effectiveness of the new ADP-based method. [ABSTRACT FROM AUTHOR]

In this paper, a new online model-free adaptive dynamic programming algorithm is developed to solve the H ∞ control problem of the continuous-time linear system with completely unknown system dynamics. Solving the game algebraic Riccati equation, commonly used in H ∞ state feedback control design, is often referred to as a two-player differential game where one player tries to minimize the predefined performance index while the other tries to maximize it. Using data generated in real time along the system trajectories, this new method can solve online the game algebraic Riccati equation without requiring the full knowledge of system dynamics. A rigorous proof of convergence of the proposed algorithm is given. Finally, simulation studies on two examples demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we aim to solve the finite-horizon optimal control problem for a class of non-linear discrete-time switched systems using adaptive dynamic programming(ADP) algorithm. A new ε-optimal control scheme based on the iterative ADP algorithm is presented which makes the value function converge iteratively to the greatest lower bound of all value function indices within an error according to ε within finite time. Two neural networks are used as parametric structures to implement the iterative ADP algorithm with ε-error bound, which aim at approximating the value function and the control policy, respectively. And then, the optimal control policy is obtained. Finally, a simulation example is included to illustrate the applicability of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a novel iterative two-stage dual heuristic programming (DHP) is proposed to solve the optimal control problems for a class of discrete-time switched nonlinear systems subject to actuators saturation. First, a novel nonquadratic performance functional is introduced to confront control constraints of the saturating actuator. Then, the iterative two-stage DHP algorithm is developed to solve the Hamilton–Jacobi–Bellman (HJB) equation of the switched system with the saturating actuator. Moreover, the convergence and optimality of the two-stage DHP algorithm are strictly proven. To implement this algorithm efficiently, there are two neural networks used as parametric structure to approximate the costate function and the corresponding control law, respectively. Finally, simulation results are given to verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, a novel theoretic formulation based on adaptive dynamic programming (ADP) is developed to solve online the optimal tracking problem of the continuous-time linear system with unknown dynamics. First, the original system dynamics and the reference trajectory dynamics are transformed into an augmented system. Then, under the same performance index with the original system dynamics, an augmented algebraic Riccati equation is derived. Furthermore, the solutions for the optimal control problem of the augmented system are proven to be equal to the standard solutions for the optimal tracking problem of the original system dynamics. Moreover, a new online algorithm based on the ADP technique is presented to solve the optimal tracking problem of the linear system with unknown system dynamics. Finally, simulation results are given to verify the effectiveness of the theoretic results. [ABSTRACT FROM PUBLISHER]

In this paper, an infinite-horizon optimal tracking control scheme is proposed for a class of nonlinear discrete-time switched systems. First, via system transformation, the optimal tracking problem is converted into designing an optimal regulator for the tracking error dynamics. And then, with convergence analysis in terms of value function and control policy, the iterative adaptive dynamic programming (ADP) algorithm is introduced to obtain the infinite-horizon optimal tracking controller which makes the value function close to its optimal value function. Next, two neural networks are used as parametric structures to implement the ADP algorithm, which aim at approximating the value function and the control policy, respectively. Finally, a simulation example is included to complement the theoretical discussions. [ABSTRACT FROM AUTHOR]