Search

In many pilot research and development (RD MMGs are microgrids equipped with advanced controls to address more emerging scenarios in smart grids. For having a stable and reliable MMG, we need to synthesize an optimal, robust, primary frequency controller for the islanded mode of MMG of the future. This task is challenging because of unknown mechanical parameters, occurrence of uncertain disturbances, uncertainty of loads, operating point variations, and the appearance of engine delays, and hence nonminimum phase dynamics. This article presents an innovative primary frequency control for the engine generators regulating the frequency of an islanded MMG in the context of smart grids. The proposed approach is based on an adaptive optimal output-feedback control algorithm using adaptive dynamic programming (ADP). The convergence of algorithms, along with the stability analysis of the closed-loop system, is also shown in this article. Finally, as experimental validation, hardware-in-the-loop (HIL) test results are provided in order to examine the effectiveness of the proposed methodology practically. Note to Practitioners —This article was motivated by the problem of primary frequency controls in modernized microgrids (MMGs) using engine generators, which are still one of the prime sources of regulating frequency in pilot research and development (R&D) microgrid projects. Although MMGs will be integral parts of the smart grid of the future, their primary controls in the islanded mode are not advanced enough and not considering existing theoretical challenges scientifically. Existing approaches to regulate frequency using industrially accepted methods are highly model-based and not optimal. Besides, they are not considering the nonminimum phase dynamics. These dynamics are mainly associated with the engine delays—an inherent issue of mechanical parts—for islanded microgrids. This article suggests a new adaptive optimal output-feedback control approach based on the adaptive dynamic programming (ADP) to the abovementioned problem under consideration. By using the proposed methodology, MMGs can deal with the issues mentioned earlier, which are challenging. The proposed approach is optimally rejecting uncertain disturbances (considering the load uncertainty and operating point variations) and reducing the impacts of nonminimum phase dynamics caused by the engine delay. Based on our currently available hardware-in-the-loop (HIL) device’s capability of modeling power systems’ components in real time, our HIL-based experiments demonstrate that this approach is feasible.

This paper studies the infinite-horizon adaptive optimal control of continuous-time linear periodic (CTLP) systems. A novel value iteration (VI) based off-policy ADP algorithm is proposed for a general class of CTLP systems, so that approximate optimal solutions can be obtained directly from the collected data, without the exact knowledge of system dynamics. Under mild conditions, the proofs on uniform convergence of the proposed algorithm to the optimal solutions are given for both the model-based and model-free cases. The VI-based ADP algorithm is able to find suboptimal controllers without assuming the knowledge of an initial stabilizing controller. Application to the optimal control of a triple inverted pendulum subjected to a periodically varying load demonstrates the feasibility and effectiveness of the proposed method., Comment: 9 pages, 2 figures

This article presents a solution to event-triggered stabilization of a class of nonlinear time-delay systems in the strict-feedback form. Set-valued maps are employed to handle the event-triggered sampling errors, and the closed-loop system is transformed into an interconnection of input-to-state stable subsystems through a set-valued constructive design. The event trigger is designed to satisfy a cyclic-small-gain condition, and the stability of the closed-loop event-triggered system is guaranteed by the cyclic-small-gain theorem for time-delay systems. The proposed design is verified by a practical example of cascade connected reactors.

This article studies the event-triggered control problem for nonholonomic systems in the chained form with disturbances and drift uncertain nonlinearities. Both the cases of state-feedback and output-feedback are investigated. To address the effects of nonholonomic constraints in event-triggered control, a new systematic design integrating a state-scaling technique and set-valued maps are proposed. A crucial strategy is to transform the event-triggered control system into an interconnection of multiple input-to-state stable systems, to which the cyclic-small-gain theorem is applied for event-based controller synthesis. It is shown that the cyclic-small-gain-based design scheme leads to Zeno-free event-triggered controllers. For the output-feedback case, a new nonlinear observer is designed to deal with the sampling errors. Interestingly, the obtained results are new even if the plant model is disturbance-free. Both numerical and experimental results validate the efficiency of the proposed cyclic-small-gain-based event-triggered control methodology.

This paper investigates the problem of the resilient control for cyber-physical systems (CPSs) in the presence of malicious sensor denial-of-service (DoS) attacks, which result in the loss of state information. The concepts of DoS frequency and DoS duration are introduced to describe the DoS attacks. According to the attack situation, that is, whether the attack is successfully implemented or not, the original physical system is rewritten as a switched version. A resilient sliding mode control scheme is designed to guarantee that the physical process is exponentially stable, which is a foundation of the main results. Then, a zero-sum game is employed to establish an effective mixed defense mechanism. Furthermore, a defense-based resilient sliding mode control scheme is proposed and the desired control performance is achieved. Compared with the existing results, the differences mainly lie in two aspects, that is, one where a switched model is obtained, based on which the average dwell-time like approach is utilized to derive the resilient control scheme, and the other where the zero-sum game in employed to make the attacks satisfy the concepts of DoS frequency and DoS duration. Finally, simulation results are given to demonstrate the effectiveness of the proposed resilient control approach.

This paper studies the extremum seeking problem for static maps with the inputs of the maps generated by a nonlinear uncertain system. A new small-gain approach is developed which uses an extremum seeking strategy to generate a reference signal, and employs a control law for reference-tracking of the nonlinear uncertain systems. The notions of input-to-state stability (ISS) and input-to-output stability (IOS) are used to characterize the interconnection between the extremum seeking strategy and the reference-tracking controller, and the nonlinear small-gain theorem is employed to guarantee the stability of the closed-loop extremum seeking system. With the proposed approach, the extremum seeking problem for a complex nonlinear system is solvable as long as one can design a proper reference-tracking controller for the system. Examples are given to show the feasibility of the proposed approach, and a numerical simulation is employed to show the effectiveness of the proposed design.

This paper presents a new tool for feedback control design of nonlinear systems in the presence of non-smooth measurement errors. We introduce a small-gain design approach to robust control of nonl...

Predictive cruise control concerns designing controllers for autonomous vehicles using the broadcasted information from the traffic lights such that the idle time around the intersection can be reduced. This study proposes a novel adaptive optimal control approach based on reinforcement learning to solve the predictive cruise control problem of a platoon of connected and autonomous vehicles. First, the reference velocity is determined for each autonomous vehicle in the platoon. Second, a data-driven adaptive optimal control algorithm is developed to estimate the gains of the desired distributed optimal controllers without the exact knowledge of system dynamics. The obtained controller is able to regulate the headway, velocity, and acceleration of each vehicle in a suboptimal sense. The goal of trip time reduction is achieved without compromising vehicle safety and passenger comfort. Numerical simulations are presented to validate the efficacy of the proposed methodology.

This paper presents a cooperative/shared framework of the driver and his/her semi-autonomous vehicle in order to achieve desired steering performance. In particular, a copilot controller and the driver together operate and control the vehicle. Exploiting the classical small-gain theory, our proposed shared steering controller is developed independent of the unmeasurable internal states of the human driver, and only relies on his/her steering torque. Furthermore, by adopting data-driven adaptive dynamic programming and an iterative learning scheme, the shared steering controller is studied from the measurable data of the driver and the vehicle. Meanwhile, the accurate knowledge of the driver and the vehicle dynamics is unnecessary, which settles the problem of their potential parametric variations/uncertainties in practice. The effectiveness of the proposed method is validated by rigorous analysis and demonstrated by numerical simulations.

This article investigates the detection and isolation of false data injection (FDI) attacks in a smart grid based on the unknown input (UI) interval observer. Recent studies have shown that the FDI attacks can bypass the traditional bad data detection methods by using the vulnerability of state estimation. For this reason, the emergency of FDI attacks brings enormous risk to the security of smart grid. To solve this crucial problem, an UI interval observer-based detection and the isolation scheme against FDI attacks are proposed. We first design the UI interval observers to obtain interval state estimation accurately, based on the constructed physical dynamics grid model. Through the capabilities of the designed UI interval observers, the accurate interval estimation state can be decoupled from unknown disturbances. Based on the characteristics of the interval residuals, a UI interval observer-based global detection algorithm was proposed. Particularly, the interval residual-based detection criteria can address the limitation of the precomputed threshold in traditional bad data detection methods. On this basis, we further consider the detection and isolation of FDI attacks under structure vulnerability. Namely, there exist undetectable FDI attacks in the grid system. Taking the attack undetectability problem into account, a logic judgment matrix-based local detection and isolation algorithm against FDI attacks are developed. Based on the combinations of observable sensor cases, local control centers can further detect and isolate the attack set under structure vulnerability. Finally, the effectiveness of the developed detection and isolation algorithms against FDI attacks is demonstrated on the IEEE 8-bus and IEEE 118-bus smart grid system, respectively.

This paper investigates the consensus problem of general linear discrete-time multiagent systems by using distributed model predictive control (DMPC) with self-triggered mechanism. First, a novel DMPC-based consensus algorithm is proposed, where each agent only needs to obtain its neighbors’ predicted state sequences once at each time step. We prove that the resultant DMPC optimization problem is feasible, and the proposed algorithm guarantees the dynamic consensus of agents. Then, to further reduce the communication cost and the energy consumption of control updates, a self-triggered DMPC-based consensus algorithm is proposed with the control input and the triggering interval jointly optimized. Numerical examples including the benchmark problem with platooning vehicles are provided to verify the effectiveness and advantages of the proposed algorithms.

In this paper, we introduce a novel reinforcement learning &#x0028 RL &#x0029 scheme for linear continuous-time dynamical systems. Different from traditional batch learning algorithms, an incremental learning approach is developed, which provides a more efficient way to tackle the on-line learning problem in real-world applications. We provide concrete convergence and robust analysis on this incremental-learning algorithm. An extension to solving robust optimal control problems is also given. Two simulation examples are also given to illustrate the effectiveness of our theoretical result.

This brief proposes a novel solution to problems related to the measurement feedback adaptive optimal output regulation of discrete-time linear systems with input time-delay. Based on reinforcement learning and adaptive dynamic programming, an approximate optimal control policy is obtained via recursive numerical algorithms using online information. Convergence proofs for the proposed algorithms are given. Notably, the exact knowledge of the plant and the exosystem is not needed. The learned control policy is only a function of retrospective input and measurement output data. Theoretical analysis and an application to a grid-connected inverter show that the proposed methodologies serve as effective tools for solving adaptive and optimal output regulation problems.

We provide new reduced order observers for continuous-time nonlinear systems, first in the case where there are continuous output measurements and next in the case where there are only discrete output measurements. When continuous measurements are available, we provide observers that converge in finite time. When only discrete measurements are available, we provide observers that do not converge in finite time, but which do converge asymptotically with a rate of convergence that is proportional to the negative of the logarithm of the size of the sampling interval. We illustrate our results in a pendulum example.

This article studies the adaptive optimal control problem for continuous-time nonlinear systems described by differential equations. A key strategy is to exploit the value iteration (VI) method proposed initially by Bellman in 1957 as a fundamental tool to solve dynamic programming problems. However, previous VI methods are all exclusively devoted to the Markov decision processes and discrete-time dynamical systems. In this article, we aim to fill up the gap by developing a new continuous-time VI method that will be applied to address the adaptive or nonadaptive optimal control problems for continuous-time systems described by differential equations. Like the traditional VI, the continuous-time VI algorithm retains the nice feature that there is no need to assume the knowledge of an initial admissible control policy. As a direct application of the proposed VI method, a new class of adaptive optimal controllers is obtained for nonlinear systems with totally unknown dynamics. A learning-based control algorithm is proposed to show how to learn robust optimal controllers directly from real-time data. Finally, two examples are given to illustrate the efficacy of the proposed methodology.

International audience; We provide novel reduced-order observer designs for continuous-time nonlinear systems with measurement error. Our first result applies to systems with continuous output measurements, and provides observers that converge in a fixed finite time that is independent of the initial state when the measurement error is zero. Our second result applies under discrete measurements, and provides observers that converge asymptotically with a rate of convergence that is proportional to the negative of the logarithm of the size of a sampling interval. Our observers satisfy an enhanced input-to-state stability property with respect to the measurement error, in which an overshoot term only depends on a recent history of the measurement error. We illustrate our observers using a model of a single-link robotic manipulator coupled to a DC motor with a nonrigid joint, and in a pendulum example.

The problem of global finite-time stabilization of lower-triangular nonlinear systems is investigated in this paper. Unlike previous papers that either used the full-state information or assumed that the system nonlinearities satisfy some restrictive conditions, in this paper a novel solution to the global output-feedback finite-time stabilization problem for a larger class of nonlinear systems is proposed under a mild condition. A new analytic method is also developed to prove the global finite-time stability of the closed-loop systems.

In this paper, nonlinear control techniques are exploited to balance an unmanned bicycle with enlarged stability domain. We consider two cases. For the first case when the autonomous bicycle is balanced by the flywheel, the steering angle is set to zero, and the torque of the flywheel is used as the control input. The controller is designed based on the Interconnection and Damping Assignment Passivity Based Control (IDA-PBC) method. For the second case when the bicycle is balanced by the handlebar, the bicycle’s velocity is high, and the flywheel is turned off. The angular velocity of the handlebar is used as the control input and the balance controller is designed based on feedback linearization. In these cases, the global stability of the closed-loop unmanned bicycle is theoretically proved based on Lyapunov theory. The experiments are conducted to validate the efficacy of the proposed nonlinear balance controllers.

In this article, we develop a learning-based secure control framework for cyber-physical systems in the presence of sensor and actuator attacks. Specifically, we use a bank of observer-based estimators to detect the attacks while introducing a threat-detection level function. Under nominal conditions, the system operates with a nominal-feedback controller with the developed attack monitoring process checking the reliance of the measurements. If there exists an attacker injecting attack signals to a subset of the sensors and/or actuators, then the attack mitigation process is triggered and a two-player, zero-sum differential game is formulated with the defender being the minimizer and the attacker being the maximizer. Next, we solve the underlying joint state estimation and attack mitigation problem and learn the secure control policy using a reinforcement-learning-based algorithm. Finally, two illustrative numerical examples are provided to show the efficacy of the proposed framework.

This paper investigates a data-driven adaptive optimal control approach for antimony flotation process in presence of input time-delay and disturbance. The integration frame of adaptive dynamic programming (ADP) and value iteration (VI) is applied to the optimal controller design without requirement of system dynamics. Fundamentally different from the existing reagents control methods, the input time-delay and disturbance are simultaneously considered in the VI-based ADP control scheme. Specifically, the disturbance is compensated directly by adding an inner model as the feedforward component to the control action and the optimal feedback gain is computed by iteratively solving Riccati equation. By exploiting industrial collected data, the numerical simulation proves that the proposed data-driven methodology can enable the concentrate and tailing grade to keep tracking the target trajectories with a minimum reagents consumption.

In this paper, the formation problem of unknown multi-quadrotor systems with underactuation and nonlinearities is addressed. A formation controller including a position controller and an attitude controller is designed. The designed formation controller is based on hierarchical scheme and reinforcement learning method is used to learn the control weights of the formation controller. A simulation of formation of multiple quadrotor systems shows the effectiveness of the proposed controller.

This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discrete-time systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing infinite-horizon PI methods are discussed. Then, both data-driven off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.

Considering the presence of time-delay in actuators and nonvanishing disturbance, a data-driven control technique is developed based on adaptive dynamic programming (ADP) to solve the reagents control problem for flotation processes. First, the reagents control problem is converted into an optimal regulator control problem. Second, a policy iteration (PI) algorithm has been adopted to find desirable adaptive suboptimal controllers. Unlike conventional controllers (PID, MPC) design, the proposed method can achieve the desired control performance by only employing online production data without the complete knowledge of the underlying flotation process dynamics. Specifically, the flotation indexes (tailing grade and concentrate grade) are forced to track the desired values with disturbance rejection and keep reagents consumption to a minimum. The convergence and stability of the proposed data-driven optimal control method are given, and the efficacy is validated in the simulation environment with industrial data.

Polarimetric synthetic aperture radar (PolSAR) data are sequentially acquired and usually large scale. Fast and accurate classification is particularly important for their applications. By introducing online learning, the PolSAR system can learn a classification model incrementally from a stream of instances, which is of high efficiency for newly arrived samples processing, strong adaptability for a dynamically changing environment, and excellent scalability for rapidly increasing data. In this paper, we propose an Online Multi-view Passive-Aggressive learning algorithm, named OMPA, for PolSAR data real-time classification. The polarimetric, color, and texture features are extracted to characterize PolSAR data, and each type of features corresponds to one view. In order to exploit the consistency and complementary property of these views, we give a new optimization model that ensembles the classifiers of multiple distinct views and enforces the agreement between each predictor and the combined predictor. The corresponding algorithms for both binary and multiclass classification tasks are derived, and the update steps have analytical solutions. In addition, we rigorously derive a bound on the number of prediction mistakes of the method. The proposed OMPA algorithm is evaluated on two real PolSAR datasets for built-up areas extraction and land cover classification, respectively. Experimental results demonstrate that OMPA consistently maintains a smaller mistake rate with low time cost and achieves about 1% and 2% accuracy improvements on the datasets, respectively, compared with the best results of the previously known online single-view and multiview learning methods.

The finite-time stabilization of a general class of lower-triangular nonlinear systems using output feedback is addressed in this paper. The system nonlinearities are only required to be Holder continuous with output-dependent Holder coefficients. A new constructive output-feedback, finite-time controller design is proposed based on a finite-time observer. Rigorous finite-time stability analysis is given for the closed-loop observer–controller system. A practical example of controlling a robotic model is adopted to illustrate the proposed method.

This note proposes a novel data-driven solution to the cooperative adaptive optimal control problem of leader-follower multiagent systems under switching network topology. The dynamics of all the followers are unknown, and the leader is modeled by a perturbed exosystem. Through the combination of adaptive dynamic programming and internal model principle, an approximate optimal controller is iteratively learned online using real-time input-state data. Rigorous stability analysis shows that the system in closed-loop with the developed control policy is leader-to-formation stable, with guaranteed robustness to unmeasurable leader disturbance. Numerical results illustrate the effectiveness of the proposed data-driven algorithm.

This paper employs the recently developed nonlinear control tools to solve the engineering problem of stable current control of fused magnesium furnaces (FMFs). By fully taking into account the nonlinearity, uncertainty, and couplings in FMFs, this paper first develops a dynamic model for controller design. A mild assumption for controller design is verified by using experimental data. Based on the dynamic model, this paper proposes a class of stable control rules for the disturbance-free model. Then, the result is extended to handle both measurement errors and external disturbances. With the proposed control algorithm, it is proved that the closed-loop system is robustly stable, and the three-phase currents ultimately converge to neighborhoods of the setpoints. If the system is disturbance-free, then asymptotic stability can be guaranteed. It is shown that the proposed control algorithm can be readily implemented as rule-based control. The notion of input-to-state stability and the nonlinear small-gain theorem are employed as tools for the designs. An industrial application is demonstrated to show the validity of the proposed design in solving practical engineering problems.

This paper provides a novel non-model-based, data-driven stochastic H ∞ control design for linear continuous-time stochastic interconnected systems with unknown dynamics. Our contributions are three-fold. First, we develop a tool to show how to assign an arbitrarily small input-to-output stochastic L 2 gain of the closed-loop system, by combining the gain assignment technique with the zero-sum dynamic game-based H ∞ control design. Second, robustness to dynamic uncertainties is tackled using the small-gain theory. Third, we develop a non-model-based stochastic robust adaptive dynamic programming (RADP) algorithm for adaptive optimal controller design. In sharp contrast to the existing methods, the obtained algorithm is based on value iteration (VI), and the knowledge of an initial stabilizing control policy is no longer needed. An example of a power electronic system is adopted to illustrate the obtained results.

This paper develops a large-scale small-gain result for dynamic networks composed of infinite-dimensional subsystems. It is assumed that the subsystems are input-to-output stable (IOS) and unboundedness observable (UO), and the large-scale infinite-dimensional system can be proved to be IOS and UO if the proposed small-gain condition is satisfied.

In this paper, we propose a shared steering control strategy for semi-autonomous vehicles by taking into account the interaction between the driver and his/her vehicle. The copilot controller cooperates with the driver and help maintain the vehicle in the central position of a lane. Taking advantage of the small-gain theory, the design procedure does not rely on the perfect knowledge of the model and states for the driver. An adaptive dynamic programming method is introduced to develop a shared controller in real-time using online measurable input-output data from vehicle sensors. The efficacy of the proposed shared steering controller is demonstrated by computer-based simulations.

This paper provides a personal account of the small-gain theory as a tool for stability analysis, control synthesis, and robustness analysis for interconnected uncertain systems. A milestone in modern control theory is the development of a transformative stability criterion known as the classical small-gain theorem proposed by George Zames in 1966, that surpasses Lyapunov theory in that there is no need to construct Lyapunov functions for the finite-gain stability of feedback systems. Under the small-gain framework, a feedback system composed of two finite-gain stable subsystems remains finite-gain stable if the loop gain is less than one. Despite its apparent simplicity at first sight, Zames’s small-gain theorem plays a crucial role in the development of linear robust control theory. Borrowing techniques in modern nonlinear control, especially Sontag’s notion of input-to-state stability (ISS), the first generalized, nonlinear ISS small-gain theorem proposed by one of the authors in 1994 overcomes the two shortcomings of Zames’s small-gain theorem. First, the use of nonlinear gains allows to consider strongly nonlinear, interconnected systems. Second, the role of initial conditions is made explicit so that both internal Lyapunov stability and external input-output stability can be studied in a unified framework. In this survey paper, we first review early developments in the nonlinear small-gain theory for interconnected systems of various types such as continuous-time systems, discrete-time systems, hybrid systems and time-delay systems, along with applications in robust nonlinear control. Then, we describe how to obtain a network small-gain theory for large-scale dynamical networks that are comprised of more than two interacting nonlinear systems. Constructive methods for the generation of Lyapunov functions for the total network are presented as well. Finally, this paper discusses how the network/nonlinear small-gain theory can be applied to obtain innovative solutions to quantized and event-based nonlinear control problems, that are important for the development of a complete theory of controlling cyber-physical systems subject to communications and computation constraints.

This paper studies the event-triggered control problem for systems subject to both dynamic uncertainties and external disturbances. To overcome infinitely fast sampling caused by the disturbance, a new event-triggering mechanism is proposed, which uses not only the measurable system state but also an estimate of the unmeasurable state and the external disturbance. The inter-sampling intervals can be proved to be lower bounded by a positive constant, which is independent on the magnitudes of the unmeasurable state and the external disturbance. With the proposed design, the closed-loop event-triggered system can be input-to-state stabilized with the external disturbance as the input. Refined tools of input-to-state stability (ISS) and the small-gain theorem are employed in solving the problem.

In this paper, a data-driven non-model-based approach is proposed for the adaptive optimal control of a class of connected vehicles that is composed of $n$ human-driven vehicles only transmitting motional data and an autonomous vehicle in the tail receiving the broadcasted data from preceding vehicles by wireless vehicle-to-vehicle (V2V) communication devices. Considering the cases of range-limited V2V communication and input saturation, several optimal control problems are formulated to minimize the errors of distance and velocity and to optimize the fuel usage. By employing an adaptive dynamic programming technique, the optimal controllers are obtained without relying on the knowledge of system dynamics. The effectiveness of the proposed approaches is demonstrated via the online learning control of the connected vehicles in Paramics' traffic microsimulation.

This paper studies an event-triggered control problem of nonlinear discrete-time systems in the presence of external disturbances. In particular, we focus on the practically interesting situation where the controller update at step k is determined by the feedback information x ( k − 1 ) measured at step k − 1 . The presence of external disturbance makes it hard to predict the magnitude of x ( k ) based on x ( k − 1 ) and then to design a desired event-triggering data transmission mechanism. In this paper, refined tools of input-to-state stability (ISS) and the nonlinear small-gain theorem are developed to estimate the influence of external disturbances, before an ISS-induced design is proposed to solve the problem. The proposed solution leads to a new event-triggering data transmission mechanism, which drastically differs from the existing event-triggered control results.

In this paper, a model-free solution to the H ∞ control of linear discrete-time systems is presented. The proposed approach employs off-policy reinforcement learning (RL) to solve the game algebraic Riccati equation online using measured data along the system trajectories. Like existing model-free RL algorithms, no knowledge of the system dynamics is required. However, the proposed method has two main advantages. First, the disturbance input does not need to be adjusted in a specific manner. This makes it more practical as the disturbance cannot be specified in most real-world applications. Second, there is no bias as a result of adding a probing noise to the control input to maintain persistence of excitation (PE) condition. Consequently, the convergence of the proposed algorithm is not affected by probing noise. An example of the H ∞ control for an F-16 aircraft is given. It is seen that the convergence of the new off-policy RL algorithm is insensitive to probing noise.

This technical note investigates the robust global stabilization problem for nonlinear systems subject to dynamic and stochastic disturbance, from the perspective of stochastic input-to-state stability. A new stochastic gain assignment method is developed. This method can be combined with the nonlinear small-gain theorem to design partial-state feedback controllers for stochastic nonlinear systems. A numerical example is given to illustrate the effectiveness of the proposed method.

The primary contribution of the paper is to show that the Lyapunov–Krasovskii conditions available in the literature, for checking the input-to-state stability property of systems described by neutral functional differential equations in Hale’s form, are also necessary. It is shown that the Lyapunov–Krasovskii conditions available in the literature are equivalent to the input-to-state stability property, for an enlarged class of neutral systems. Finally, a novel necessary and sufficient condition for the input-to-state stability property is provided.

Sensorimotor tasks that humans perform are often affected by different sources of uncertainty. Nevertheless, the central nervous system (CNS) can gracefully coordinate our movements. Most learning frameworks rely on the internal model principle, which requires a precise internal representation in the CNS to predict the outcomes of our motor commands. However, learning a perfect internal model in a complex environment over a short period of time is a nontrivial problem. Indeed, achieving proficient motor skills may require years of training for some difficult tasks. Internal models alone may not be adequate to explain the motor adaptation behavior during the early phase of learning. Recent studies investigating the active regulation of motor variability, the presence of suboptimal inference, and model-free learning have challenged some of the traditional viewpoints on the sensorimotor learning mechanism. As a result, it may be necessary to develop a computational framework that can account for these new phenomena. Here, we develop a novel theory of motor learning, based on model-free adaptive optimal control, which can bypass some of the difficulties in existing theories. This new theory is based on our recently developed adaptive dynamic programming (ADP) and robust ADP (RADP) methods and is especially useful for accounting for motor learning behavior when an internal model is inaccurate or unavailable. Our preliminary computational results are in line with experimental observations reported in the literature and can account for some phenomena that are inexplicable using existing models.

This paper studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulator (LQR) problem. It is shown that Kleinman's policy iteration algorithm is small-disturbance input-to-state stable, a property that is stronger than Sontag's local input-to-state stability but weaker than global input-to-state stability. More precisely, whenever the error in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded and enter a small neighborhood of the optimal solution of the LQR problem. Based on this result, an off-policy data-driven policy iteration algorithm for the LQR problem is shown to be robust when the system dynamics are subjected to small additive unknown bounded disturbances. The theoretical results are validated by a numerical example.

In this paper, we develop a learning-based secure control framework for cyber-physical systems in the presence of sensor attacks. Specifically, we use several observer-based estimators to detect the attacks while also introducing a threat detection level function. We then solve the underlying joint state estimation and attack mitigation problems by using a reinforcement learning algorithm. Finally, an illustrative numerical example is provided to show the efficacy of the proposed framework.

This paper studies the feedback control problem of nonlinear systems in strict-feedback form with the coordinated design of event-trigger, dynamic quantizer and nonlinear control law. Through a novel recursive control design based on set-valued maps, the closed-loop event-triggered quantized system is transformed into an interconnection of input-to-state stable (ISS) subsystems. It is shown by the cyclic-small-gain theorem that the closed-loop system also is ISS. By appropriately choosing the parameters of the event-triggered quantized controller, the state of the closed-loop system can be steered to within an arbitrarily small neighborhood of the origin, and infinitely fast sampling and in particular the Zeno phenomenon are avoided.

This paper studies the problem of distributed containment control for multi-agent systems modeled by double integrators. A new class of rotation-free distributed containment controllers with successive saturated loops is proposed for the challenging situation where the velocity and acceleration of each agent are required to stay within their desired ranges. It is proved that the objective of containment control is achievable if the parameters of the controllers satisfy a mild condition. Moreover, the desired ranges of the accelerations and the velocities are allowed to be very small, and the interconnection topology is allowed to be directed and switched.

Lateral control design is one of the fundamental components for self-driving cars. In this paper, we propose a learning-based control strategy that enables a mobile car equipped with a camera to perfectly perform lane keeping in a road on the ground. Using the method of adaptive dynamic programming, the proposed control algorithm exploits the structural knowledge of the car kinematics as well as the collected data (images) about the lane information. An adaptive optimal lateral controller is obtained through a data-driven learning algorithm. The effectiveness of the proposed method is demonstrated by theoretical stability proofs and experimental evaluations.

In this paper, a reinforcement learning based control law is proposed to solve the attitude synchronization problem of the leader-following multi-quadrotor systems. The overall system is composed of a team of quadrotors, modeled with highly nonlinear and coupled dynamics. An optimal control solution is obtained by solving an augmented Hamilton-Jacobi-Bellman equation. A reinforcement learning approach is used to learn the optimal control law. Simulation results are provided to verify the effectiveness of the proposed controller.

This paper proposes an event-triggered adaptive optimal output-feedback control method for continuous-time linear systems. With the proposed design, the unmeasurable states are reconstructed by using the measurable input and output data, and the discrete-time algebraic Riccati equation is iteratively solved through adaptive dynamic programming. Then, an event-based feedback strategy is introduced to the closed-loop system to reduce the communication between the controller and the plant. Convergence to the optimal solution and the closed-loop stability analysis are carried out by using Lyapunov method. Two numerical examples are employed to verify the effectiveness of the design.

To improve the observation accuracy and shorten the transient time simultaneously, in this paper, we propose a new nonlinear extended state observer (ESO) by using a switching strategy. The convergence of the proposed ESO is proved with explicit error estimation. The effectiveness of the proposed switched nonlinear ESO is demonstrated through numerical simulations and comparison with existing ESOs. The numerical results show that the proposed new ESO is more accurate and the transient time is shorter than the previous ESOs with the same tuning parameter.

This paper presents several regularization schemes for data-driven optimization problems. Data-driven optimization is a challenging task because the model or the distribution describing the inherent stochastic disturbance is unknown but instead only a set of data sampled from it can be available. Most existing literature tackles the data-driven optimization problem by using the approach of distributionally robust optimization which is however an infinite-dimensional optimization problem and thus is required to be transformed into a tractable finite-dimensional optimization problem. As a different line, the regularization schemes presented in this paper contribute to solving the data-driven minimization problem by approximating the unknown expectation with some data-dependent surrogates in a high confidence and by minimizing the surrogates in a tractable way. The tradeoff between the confidence level and the optimization error is analyzed.

This paper studies the measurement feedback control problem of nonlinear systems in the cascaded form with dynamic uncertainties. It is assumed that the dynamic uncertainties are integral input-to-state stable (iISS). A new construction design with set-valued virtual control laws is proposed to deal with the measurement disturbance, and the closed-loop system is transformed into an interconnected system with the subsystem being ISS or iISS. The closed-loop system is proved to be iISS with respect to the measurement disturbance by constructing an iISS Lyapunov function.

This paper focuses on the detection of false data inject attacks (FDIAs) in smart grids. An interval observer-based detection scheme against the FDIAs is proposed, by considering the stealthy characteristics of FDIAs. Based on the constructed physical dynamics grid model, we design interval observer to estimate the interval state of physical dynamics accurately. To address the limitation of precomputed threshold, an interval residual-based detection standard is proposed. The residual evaluation functions and detection threshold in traditional attack detection methodologies are replaced by the interval residuals which can be regarded as the time-varying detection threshold. Finally, the effectiveness of the developed detection method is demonstrated by using a detailed study on the IEEE 37-bus smart grid system.