Search

In this paper, an event‐triggered safe control method based on adaptive critic learning (ACL) is proposed for a class of nonlinear safety‐critical systems. First, a safe cost function is constructed by adding a control barrier function (CBF) to the traditional quadratic cost function; the optimization problem with safety constraints that is difficult to deal with by classical ACL methods is solved. Subsequently, the event‐triggered scheme is introduced to reduce the amount of computation. Further, combining the properties of CBF with the ACL‐based event‐triggering mechanism, the event‐triggered safe Hamilton–Jacobi–Bellman (HJB) equation is derived, and a single critic neural network (NN) framework is constructed to approximate the solution of the event‐triggered safe HJB equation. In addition, the concurrent learning method is applied to the NN learning process, so that the persistence of excitation (PE) condition is not required. The weight approximation error of the NN and the states of the system are proven to be uniformly ultimately bounded (UUB) in the safe set with the Lyapunov theory. Finally, the availability of the presented method can be validated through the simulation. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of decentralized safety control (DSC) of constrained interconnected nonlinear safety-critical systems under reinforcement learning strategies, where asymmetric input constraints and security constraints are considered. To begin with, improved performance functions associated with the actuator estimates for each auxiliary subsystem are constructed. Then, the decentralized control problem with security constraints and asymmetric input constraints is transformed into an equivalent decentralized control problem with asymmetric input constraints using the barrier function. This approach ensures that safety-critical systems operate and learn optimal DSC policies within their safe global domains. Then, the optimal control strategy is shown to ensure that the entire system is uniformly ultimately bounded (UUB). In addition, all signals in the closed-loop auxiliary subsystem, based on Lyapunov theory, are uniformly ultimately bounded, and the effectiveness of the designed method is verified by practical simulation. [ABSTRACT FROM AUTHOR]

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton–Jacobi–Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor–critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach. [ABSTRACT FROM AUTHOR]

In this article, the H∞$$ {H}_{\infty } $$ safe control problem of continuous‐time affine nonlinear safety‐critical systems is studied based on the barrier function (BF) and adaptive dynamic programming (ADP). We show that the safety constraints in this article occur under the condition that the system initial state is unsafe and have not been adequately addressed in the existing work. First, the H∞$$ {H}_{\infty } $$ control problem is transformed into a zero‐sum game problem, and a new safe Hamilton–Jacobi–Isaacs equation is proposed by combining with the BF, which makes the unsafe behavior be punished in the learning process. In addition, a damping coefficient is introduced into the BF to punish the unsafe behavior more flexibly. Aiming at the requirement that the initial system state must be strictly constrained in the safe set, a new weight updating method based on ADP is proposed, which can reasonably avoid the influence of the BF on neural network (NN) parameters when the initial system state is unsafe. Furthermore, based on the Lyapunov stability theory, it is proved that the system states of safety‐critical systems and the NN parameters are uniformly ultimately bounded under the safety constraints and disturbances effect. Finally, the effectiveness of the proposed method is verified by two simulation examples. [ABSTRACT FROM AUTHOR]

With the continuous development of Unmanned Aerial Vehicle (UAV) technology, UAVs are widely used in military and civilian fields. Multi-UAV networks are often referred to as flying ad hoc networks (FANET). Dividing multiple UAVs into clusters for management can reduce energy consumption, maximize network lifetime, and enhance network scalability to a certain extent, so UAV clustering is an important direction for UAV network applications. However, UAVs have the characteristics of limited energy resources and high mobility, which bring challenges to UAV cluster communication networking. Therefore, this paper proposes a clustering scheme for UAV clusters based on the binary whale optimization (BWOA) algorithm. First, the optimal number of clusters in the network is calculated based on the network bandwidth and node coverage constraints. Then, the cluster heads are selected based on the optimal number of clusters using the BWOA algorithm, and the clusters are divided based on the distance. Finally, the cluster maintenance strategy is set to achieve efficient maintenance of clusters. The experimental simulation results show that the scheme has better performance in terms of energy consumption and network lifetime compared with the BPSO and K-means-based schemes. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the constrained optimal control problem of nonlinear multi-input safety-critical systems with uncertain disturbances and time-varying safety constraints. By utilizing a barrier function transformation, together with a new disturbance-related term and a smooth safety boundary function, a nominal system-dependent multi-input barrier transformation architecture is developed to deal with the time-varying safety constraints and uncertain disturbances. Based on the obtained transformation system, the coupled Hamilton–Jacobi–Bellman (HJB) function is established to obtain the constrained Nash equilibrium solution. In addition, due to the fact that it is difficult to solve the HJB function directly, the single critic neural network (NN) is constructed to approximate the optimal performance index function of different control inputs, respectively. It is proved theoretically that, under the influence of uncertain disturbances and time-varying safety constraints, the system states and neural network parameters can be uniformly ultimately bounded (UUB) by the proposed neural network approximation method. Finally, the effectiveness of the proposed method is verified by two nonlinear simulation examples. [ABSTRACT FROM AUTHOR]

In this paper, a new adaptive critic design is proposed to approximate the online Nash equilibrium solution for the robust trajectory tracking control of non-zero-sum (NZS) games for continuous-time uncertain nonlinear systems. First, the augmented system was constructed by combining the tracking error and the reference trajectory. By modifying the cost function, the robust tracking control problem was transformed into an optimal tracking control problem. Based on adaptive dynamic programming (ADP), a single critic neural network (NN) was applied for each player to solve the coupled Hamilton–Jacobi–Bellman (HJB) equations approximately, and the obtained control laws were regarded as the feedback Nash equilibrium. Two additional terms were introduced in the weight update law of each critic NN, which strengthened the weight update process and eliminated the strict requirements for the initial stability control policy. More importantly, in theory, through the Lyapunov theory, the stability of the closed-loop system was guaranteed, and the robust tracking performance was analyzed. Finally, the effectiveness of the proposed scheme was verified by two examples. [ABSTRACT FROM AUTHOR]

In this paper, a robust trajectory tracking control method with state constraints and uncertain disturbances on the ground of adaptive dynamic programming (ADP) is proposed for nonlinear systems. Firstly, the augmented system consists of the tracking error and the reference trajectory, and the tracking control problems with uncertain disturbances is described as the problem of robust control adjustment. In addition, considering the nominal system of the augmented system, the guaranteed cost tracking control problem is transformed into the optimal control problem by using the discount coefficient in the nominal system. A new safe Hamilton–Jacobi–Bellman (HJB) equation is proposed by combining the cost function with the control barrier function (CBF), so that the behavior of violating the safety regulations for the system states will be punished. In order to solve the new safe HJB equation, a critic neural network (NN) is used to approximate the solution of the safe HJB equation. According to the Lyapunov stability theory, in the case of state constraints and uncertain disturbances, the system states and the parameters of the critic neural network are guaranteed to be uniformly ultimately bounded (UUB). At the end of this paper, the feasibility of the proposed method is verified by a simulation example. [ABSTRACT FROM AUTHOR]

At present, the intelligent control system of robots is closed, which has the disadvantages of poor fault tolerance, unstable operation and low positioning accuracy. Aiming at these deficiencies, a Petri net model of the intelligent control system for open architecture robots based on PMAC is designed. Starting from the kinematics of robots, the forward and inverse kinematics model of open architecture robots are established according to DH method; then the trajectory planning is performed from Cartesian space linear interpolation algorithm and circular interpolation algorithm respectively, and the basic function of robot path planning is constructed. Finally, a PMAC-based open architecture robot intelligent control system is established. The control system adopts dual-microcomputer hierarchical control mode and modular structure design. Real-time communication between the upper computer and the lower computer can be realized by calling the Pcomm32 dynamic link library; based on the robot's forward and inverse kinematics model and trajectory interpolation algorithm, the modular control software for the robot system is developed. The control software realizes functions such as security check, parameters setting, kinematics analysis, and teaching reproduction. Combined with the principle of hierarchical Petri nets, various modules of open architecture robot control system based on PMAC are modeled. Experiments show that the designed system runs smoothly, has high positioning accuracy, good openness and scalability. [ABSTRACT FROM AUTHOR]

In this paper, the problem of H ∞ control design for affine nonlinear discrete-time systems is addressed by using adaptive dynamic programming (ADP). First, the nonlinear H ∞ control problem is transformed into solving the two-player zero-sum differential game problem of the nonlinear system. Then, the critic, action and disturbance networks are designed by using neural networks to solve online the Hamilton–Jacobi–Isaacs (HJI) equation associating with the two-player zero-sum differential game. When novel weight update laws for the critic, action and disturbance networks are tuned online by using data generated in real-time along the system trajectories, it is shown that the system states, all neural networks weight estimation errors are uniformly ultimately bounded by using Lyapunov techniques. Further, it is shown that the output of the action network approaches the optimal control input with small bounded error and the output of the disturbance network approaches the worst disturbance with small bounded error. At last, simulation results are presented to demonstrate the effectiveness of the new ADP-based method. [ABSTRACT FROM AUTHOR]

In this paper, a new online model-free adaptive dynamic programming algorithm is developed to solve the H ∞ control problem of the continuous-time linear system with completely unknown system dynamics. Solving the game algebraic Riccati equation, commonly used in H ∞ state feedback control design, is often referred to as a two-player differential game where one player tries to minimize the predefined performance index while the other tries to maximize it. Using data generated in real time along the system trajectories, this new method can solve online the game algebraic Riccati equation without requiring the full knowledge of system dynamics. A rigorous proof of convergence of the proposed algorithm is given. Finally, simulation studies on two examples demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The problem of \(H_{\infty }\) state feedback control of affine nonlinear discrete-time systems with unknown dynamics is investigated in this paper. An online adaptive policy learning algorithm (APLA) based on adaptive dynamic programming (ADP) is proposed for learning in real-time the solution to the Hamilton–Jacobi–Isaacs (HJI) equation, which appears in the \(H_{\infty }\) control problem. In the proposed algorithm, three neural networks (NNs) are utilized to find suitable approximations of the optimal value function and the saddle point feedback control and disturbance policies. Novel weight updating laws are given to tune the critic, actor, and disturbance NNs simultaneously by using data generated in real-time along the system trajectories. Considering NN approximation errors, we provide the stability analysis of the proposed algorithm with Lyapunov approach. Moreover, the need of the system input dynamics for the proposed algorithm is relaxed by using a NN identification scheme. Finally, simulation examples show the effectiveness of the proposed algorithm. [ABSTRACT FROM PUBLISHER]

In this paper, we aim to solve the finite-horizon optimal control problem for a class of non-linear discrete-time switched systems using adaptive dynamic programming(ADP) algorithm. A new ε-optimal control scheme based on the iterative ADP algorithm is presented which makes the value function converge iteratively to the greatest lower bound of all value function indices within an error according to ε within finite time. Two neural networks are used as parametric structures to implement the iterative ADP algorithm with ε-error bound, which aim at approximating the value function and the control policy, respectively. And then, the optimal control policy is obtained. Finally, a simulation example is included to illustrate the applicability of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a novel iterative two-stage dual heuristic programming (DHP) is proposed to solve the optimal control problems for a class of discrete-time switched nonlinear systems subject to actuators saturation. First, a novel nonquadratic performance functional is introduced to confront control constraints of the saturating actuator. Then, the iterative two-stage DHP algorithm is developed to solve the Hamilton–Jacobi–Bellman (HJB) equation of the switched system with the saturating actuator. Moreover, the convergence and optimality of the two-stage DHP algorithm are strictly proven. To implement this algorithm efficiently, there are two neural networks used as parametric structure to approximate the costate function and the corresponding control law, respectively. Finally, simulation results are given to verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, a novel theoretic formulation based on adaptive dynamic programming (ADP) is developed to solve online the optimal tracking problem of the continuous-time linear system with unknown dynamics. First, the original system dynamics and the reference trajectory dynamics are transformed into an augmented system. Then, under the same performance index with the original system dynamics, an augmented algebraic Riccati equation is derived. Furthermore, the solutions for the optimal control problem of the augmented system are proven to be equal to the standard solutions for the optimal tracking problem of the original system dynamics. Moreover, a new online algorithm based on the ADP technique is presented to solve the optimal tracking problem of the linear system with unknown system dynamics. Finally, simulation results are given to verify the effectiveness of the theoretic results. [ABSTRACT FROM PUBLISHER]