Search

This paper addresses the problem of decentralized safety control (DSC) of constrained interconnected nonlinear safety-critical systems under reinforcement learning strategies, where asymmetric input constraints and security constraints are considered. To begin with, improved performance functions associated with the actuator estimates for each auxiliary subsystem are constructed. Then, the decentralized control problem with security constraints and asymmetric input constraints is transformed into an equivalent decentralized control problem with asymmetric input constraints using the barrier function. This approach ensures that safety-critical systems operate and learn optimal DSC policies within their safe global domains. Then, the optimal control strategy is shown to ensure that the entire system is uniformly ultimately bounded (UUB). In addition, all signals in the closed-loop auxiliary subsystem, based on Lyapunov theory, are uniformly ultimately bounded, and the effectiveness of the designed method is verified by practical simulation. [ABSTRACT FROM AUTHOR]

In this paper, the safe optimal control method for continuous-time (CT) nonlinear safety-critical systems with asymmetric input constraints and unmatched disturbances based on the adaptive dynamic programming (ADP) is investigated. Initially, a new non-quadratic form function is implemented to effectively handle the asymmetric input constraints. Subsequently, the safe optimal control problem is transformed into a two-player zero-sum game (ZSG) problem to suppress the influence of unmatched disturbances, and a new Hamilton–Jacobi–Isaacs (HJI) equation is introduced by integrating the control barrier function (CBF) with the cost function to penalize unsafe behavior. Moreover, a damping factor is embedded in the CBF to balance safety and optimality. To obtain a safe optimal controller, only one critic neural network (CNN) is utilized to tackle the complex HJI equation, leading to a decreased computational load in contrast to the utilization of the conventional actor–critic network. Then, the system state and the parameters of the CNN are uniformly ultimately bounded (UUB) through the application of the Lyapunov stability method. Lastly, two examples are presented to confirm the efficacy of the presented approach. [ABSTRACT FROM AUTHOR]

In this article, the H∞$$ {H}_{\infty } $$ safe control problem of continuous‐time affine nonlinear safety‐critical systems is studied based on the barrier function (BF) and adaptive dynamic programming (ADP). We show that the safety constraints in this article occur under the condition that the system initial state is unsafe and have not been adequately addressed in the existing work. First, the H∞$$ {H}_{\infty } $$ control problem is transformed into a zero‐sum game problem, and a new safe Hamilton–Jacobi–Isaacs equation is proposed by combining with the BF, which makes the unsafe behavior be punished in the learning process. In addition, a damping coefficient is introduced into the BF to punish the unsafe behavior more flexibly. Aiming at the requirement that the initial system state must be strictly constrained in the safe set, a new weight updating method based on ADP is proposed, which can reasonably avoid the influence of the BF on neural network (NN) parameters when the initial system state is unsafe. Furthermore, based on the Lyapunov stability theory, it is proved that the system states of safety‐critical systems and the NN parameters are uniformly ultimately bounded under the safety constraints and disturbances effect. Finally, the effectiveness of the proposed method is verified by two simulation examples. [ABSTRACT FROM AUTHOR]

In this paper, for the safety–critical systems with unmatched disturbances, a safe optimal robust control method based on neural network is proposed to ensure that the safety–critical system operates within its safe region and learns the optimal control strategy. The cost function which is the goal of the designers is augmented by a control barrier function (CBF) to achieve both safety and optimality. This method does not directly regard security as a constraint on the system state, but influences the cost function through a security penalty mechanism. An additional function is used to approximate the effect of the unmatched disturbances on the safety–critical systems. On the premise of satisfying the security and robustness, the neural network approximation method is used to learn the optimal control strategy. Based on Lyapunov stability theory, it is shown that the neural network-based safe optimal robust controller can guarantee all the signals of the resulting closed-loop systems to be uniformly ultimately bounded. Finally, two simulation examples are given to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the constrained optimal control problem of nonlinear multi-input safety-critical systems with uncertain disturbances and time-varying safety constraints. By utilizing a barrier function transformation, together with a new disturbance-related term and a smooth safety boundary function, a nominal system-dependent multi-input barrier transformation architecture is developed to deal with the time-varying safety constraints and uncertain disturbances. Based on the obtained transformation system, the coupled Hamilton–Jacobi–Bellman (HJB) function is established to obtain the constrained Nash equilibrium solution. In addition, due to the fact that it is difficult to solve the HJB function directly, the single critic neural network (NN) is constructed to approximate the optimal performance index function of different control inputs, respectively. It is proved theoretically that, under the influence of uncertain disturbances and time-varying safety constraints, the system states and neural network parameters can be uniformly ultimately bounded (UUB) by the proposed neural network approximation method. Finally, the effectiveness of the proposed method is verified by two nonlinear simulation examples. [ABSTRACT FROM AUTHOR]

In this article, an integral reinforcement learning-based dynamic event-triggered safety control scheme is proposed to tackle the multiplayer Stackelberg–Nash games (MSNG) problem in continuous-time nonlinear systems with time-varying state constraints. Initially, a new barrier function (BF) is introduced by integrating traditional barrier functions with a novel smooth function to address the challenge of time-varying state constraints. Then, the constrained MSNG system is transformed into an unconstrained system through the application of the state transformation technique, which helps to characterize hierarchical decision problems as MSNG problems with the leader and followers. Meanwhile, the integral reinforcement learning (IRL) technique is also applied to ease the demand for precise system dynamics. Moreover, a new dynamic event-triggered control (DETC) mechanism is designed, resulting in coupled dynamic event-triggered Hamilton–Jacobi (HJ) equations. A single critic neural network (NN) is constructed to learn the optimal control laws for the leader and followers. By employing Lyapunov theory, the integral reinforcement learning-based dynamic event-triggered safety control method ensures that the system state and the critic NN weight errors are uniformly ultimately bounded (UUB). Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this article, an integral reinforcement learning-based dynamic event-triggered safety control scheme is proposed to tackle the multiplayer Stackelberg–Nash games (MSNG) problem in continuous-time nonlinear systems with time-varying state constraints. Initially, a new barrier function (BF) is introduced by integrating traditional barrier functions with a novel smooth function to address the challenge of time-varying state constraints. Then, the constrained MSNG system is transformed into an unconstrained system through the application of the state transformation technique, which helps to characterize hierarchical decision problems as MSNG problems with the leader and followers. Meanwhile, the integral reinforcement learning (IRL) technique is also applied to ease the demand for precise system dynamics. Moreover, a new dynamic event-triggered control (DETC) mechanism is designed, resulting in coupled dynamic event-triggered Hamilton–Jacobi (HJ) equations. A single critic neural network (NN) is constructed to learn the optimal control laws for the leader and followers. By employing Lyapunov theory, the integral reinforcement learning-based dynamic event-triggered safety control method ensures that the system state and the critic NN weight errors are uniformly ultimately bounded (UUB). Finally, two simulation examples are provided to demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Utilizing parallel learning technology and an event-triggering mechanism, we introduce an adaptive security tracking control approach for nonlinear systems with both matched and mismatched uncertainties. The methodology commences with an overview of the constraints imposed by the control barrier function (CBF) on the system state, followed by the selection of a suitable CBF. An augmented system is constructed by amalgamating the system state and tracking error, with the mismatched uncertainty being decomposed to introduce an auxiliary system. Incorporating the CBF into the performance metrics ensures that the system state remains within a secure region. Subsequently, the paper employs an event-triggering mechanism to derive an event-based secure Hamiltonian–Jacobi–Bellman (HJB) equation. An adaptive single-critic network, designed using parallel learning technology, approximates the solution to the event-based secure HJB equation. Lastly, the Lyapunov method is employed to demonstrate the uniform ultimate boundedness (UUB) of both the tracking error and weight error. The efficacy of the proposed method is validated through two nonlinear examples. [ABSTRACT FROM AUTHOR]

In this paper, a robust trajectory tracking control method with state constraints and uncertain disturbances on the ground of adaptive dynamic programming (ADP) is proposed for nonlinear systems. Firstly, the augmented system consists of the tracking error and the reference trajectory, and the tracking control problems with uncertain disturbances is described as the problem of robust control adjustment. In addition, considering the nominal system of the augmented system, the guaranteed cost tracking control problem is transformed into the optimal control problem by using the discount coefficient in the nominal system. A new safe Hamilton–Jacobi–Bellman (HJB) equation is proposed by combining the cost function with the control barrier function (CBF), so that the behavior of violating the safety regulations for the system states will be punished. In order to solve the new safe HJB equation, a critic neural network (NN) is used to approximate the solution of the safe HJB equation. According to the Lyapunov stability theory, in the case of state constraints and uncertain disturbances, the system states and the parameters of the critic neural network are guaranteed to be uniformly ultimately bounded (UUB). At the end of this paper, the feasibility of the proposed method is verified by a simulation example. [ABSTRACT FROM AUTHOR]

In this paper, a new adaptive critic design is proposed to approximate the online Nash equilibrium solution for the robust trajectory tracking control of non-zero-sum (NZS) games for continuous-time uncertain nonlinear systems. First, the augmented system was constructed by combining the tracking error and the reference trajectory. By modifying the cost function, the robust tracking control problem was transformed into an optimal tracking control problem. Based on adaptive dynamic programming (ADP), a single critic neural network (NN) was applied for each player to solve the coupled Hamilton–Jacobi–Bellman (HJB) equations approximately, and the obtained control laws were regarded as the feedback Nash equilibrium. Two additional terms were introduced in the weight update law of each critic NN, which strengthened the weight update process and eliminated the strict requirements for the initial stability control policy. More importantly, in theory, through the Lyapunov theory, the stability of the closed-loop system was guaranteed, and the robust tracking performance was analyzed. Finally, the effectiveness of the proposed scheme was verified by two examples. [ABSTRACT FROM AUTHOR]

At present, the intelligent control system of robots is closed, which has the disadvantages of poor fault tolerance, unstable operation and low positioning accuracy. Aiming at these deficiencies, a Petri net model of the intelligent control system for open architecture robots based on PMAC is designed. Starting from the kinematics of robots, the forward and inverse kinematics model of open architecture robots are established according to DH method; then the trajectory planning is performed from Cartesian space linear interpolation algorithm and circular interpolation algorithm respectively, and the basic function of robot path planning is constructed. Finally, a PMAC-based open architecture robot intelligent control system is established. The control system adopts dual-microcomputer hierarchical control mode and modular structure design. Real-time communication between the upper computer and the lower computer can be realized by calling the Pcomm32 dynamic link library; based on the robot's forward and inverse kinematics model and trajectory interpolation algorithm, the modular control software for the robot system is developed. The control software realizes functions such as security check, parameters setting, kinematics analysis, and teaching reproduction. Combined with the principle of hierarchical Petri nets, various modules of open architecture robot control system based on PMAC are modeled. Experiments show that the designed system runs smoothly, has high positioning accuracy, good openness and scalability. [ABSTRACT FROM AUTHOR]

In this paper, the problem of H ∞ control design for affine nonlinear discrete-time systems is addressed by using adaptive dynamic programming (ADP). First, the nonlinear H ∞ control problem is transformed into solving the two-player zero-sum differential game problem of the nonlinear system. Then, the critic, action and disturbance networks are designed by using neural networks to solve online the Hamilton–Jacobi–Isaacs (HJI) equation associating with the two-player zero-sum differential game. When novel weight update laws for the critic, action and disturbance networks are tuned online by using data generated in real-time along the system trajectories, it is shown that the system states, all neural networks weight estimation errors are uniformly ultimately bounded by using Lyapunov techniques. Further, it is shown that the output of the action network approaches the optimal control input with small bounded error and the output of the disturbance network approaches the worst disturbance with small bounded error. At last, simulation results are presented to demonstrate the effectiveness of the new ADP-based method. [ABSTRACT FROM AUTHOR]

In this paper, we aim to solve the finite-horizon optimal control problem for a class of non-linear discrete-time switched systems using adaptive dynamic programming(ADP) algorithm. A new ε-optimal control scheme based on the iterative ADP algorithm is presented which makes the value function converge iteratively to the greatest lower bound of all value function indices within an error according to ε within finite time. Two neural networks are used as parametric structures to implement the iterative ADP algorithm with ε-error bound, which aim at approximating the value function and the control policy, respectively. And then, the optimal control policy is obtained. Finally, a simulation example is included to illustrate the applicability of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a novel iterative two-stage dual heuristic programming (DHP) is proposed to solve the optimal control problems for a class of discrete-time switched nonlinear systems subject to actuators saturation. First, a novel nonquadratic performance functional is introduced to confront control constraints of the saturating actuator. Then, the iterative two-stage DHP algorithm is developed to solve the Hamilton–Jacobi–Bellman (HJB) equation of the switched system with the saturating actuator. Moreover, the convergence and optimality of the two-stage DHP algorithm are strictly proven. To implement this algorithm efficiently, there are two neural networks used as parametric structure to approximate the costate function and the corresponding control law, respectively. Finally, simulation results are given to verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, a novel theoretic formulation based on adaptive dynamic programming (ADP) is developed to solve online the optimal tracking problem of the continuous-time linear system with unknown dynamics. First, the original system dynamics and the reference trajectory dynamics are transformed into an augmented system. Then, under the same performance index with the original system dynamics, an augmented algebraic Riccati equation is derived. Furthermore, the solutions for the optimal control problem of the augmented system are proven to be equal to the standard solutions for the optimal tracking problem of the original system dynamics. Moreover, a new online algorithm based on the ADP technique is presented to solve the optimal tracking problem of the linear system with unknown system dynamics. Finally, simulation results are given to verify the effectiveness of the theoretic results. [ABSTRACT FROM PUBLISHER]

In this paper, an infinite-horizon optimal tracking control scheme is proposed for a class of nonlinear discrete-time switched systems. First, via system transformation, the optimal tracking problem is converted into designing an optimal regulator for the tracking error dynamics. And then, with convergence analysis in terms of value function and control policy, the iterative adaptive dynamic programming (ADP) algorithm is introduced to obtain the infinite-horizon optimal tracking controller which makes the value function close to its optimal value function. Next, two neural networks are used as parametric structures to implement the ADP algorithm, which aim at approximating the value function and the control policy, respectively. Finally, a simulation example is included to complement the theoretical discussions. [ABSTRACT FROM AUTHOR]

With the continuous development of Unmanned Aerial Vehicle (UAV) technology, UAVs are widely used in military and civilian fields. Multi-UAV networks are often referred to as flying ad hoc networks (FANET). Dividing multiple UAVs into clusters for management can reduce energy consumption, maximize network lifetime, and enhance network scalability to a certain extent, so UAV clustering is an important direction for UAV network applications. However, UAVs have the characteristics of limited energy resources and high mobility, which bring challenges to UAV cluster communication networking. Therefore, this paper proposes a clustering scheme for UAV clusters based on the binary whale optimization (BWOA) algorithm. First, the optimal number of clusters in the network is calculated based on the network bandwidth and node coverage constraints. Then, the cluster heads are selected based on the optimal number of clusters using the BWOA algorithm, and the clusters are divided based on the distance. Finally, the cluster maintenance strategy is set to achieve efficient maintenance of clusters. The experimental simulation results show that the scheme has better performance in terms of energy consumption and network lifetime compared with the BPSO and K-means-based schemes. [ABSTRACT FROM AUTHOR]