Search

In this paper, a new framework for the resilient control of continuous-time linear systems under denial-of-service (DoS) attacks and system uncertainty is presented. Integrating techniques from reinforcement learning and output regulation theory, it is shown that resilient optimal controllers can be learned directly from real-time state and input data collected from the systems subjected to attacks. Sufficient conditions are given under which the closed-loop system remains stable given any upper bound of DoS attack duration. Simulation results are used to demonstrate the efficacy of the proposed learning-based framework for resilient control under DoS attacks and model uncertainty.

In this paper, we have proposed a resilient reinforcement learning method for discrete-time linear systems with unknown parameters, under denial-of-service (DoS) attacks. The proposed method is based on policy iteration that learns the optimal controller from input-state data amidst DoS attacks. We achieve an upper bound for the DoS duration to ensure closed-loop stability. The resilience of the closed-loop system, when subjected to DoS attacks with the learned controller and an internal model, has been thoroughly examined. The effectiveness of the proposed methodology is demonstrated on an inverted pendulum on a cart.

This paper studies the safety-critical control problem for Euler-Lagrange (EL) systems subject to multiple ball obstacles and velocity constraints in accordance with affordable velocity ranges. A key strategy is to exploit the underlying inner-outer-loop structure for the design of a new cascade controller for the class of EL systems. In particular, the outer-loop controller is developed based on quadratic programming (QP) to avoid ball obstacles and generate velocity reference signals fulfilling the velocity limitation. Taking full advantage of the conservation-of-energy property, a nonlinear velocity-tracking controller is designed to form the inner loop. One major difficulty is caused by the possible non-Lipschitz continuity of the standard QP algorithm when there are multiple constraints. To solve this problem, we propose a refined QP algorithm with the feasible set reshaped by an appropriately chosen positive basis such that the feasibility is retained while the resulting outer-loop controller is locally Lipschitz. It is proved that the constraint-satisfaction problem is solvable as long as the ball obstacles satisfy a mild distance condition. The proposed design is validated by numerical simulation and an experiment based on a $2$-link planar manipulator.

This paper proposes a constructive approach to safety control of nonlinear cascade systems subject to multiple state constraints. New design ingredients include a unified characterization of safety and stability for systematic designs of safety controllers, and a novel technique of reshaping the feasible sets of quadratically constrained quadratic programming induced from safety control. The proposed method guarantees Lipschitz continuity of virtual control laws, enabling a stepwise constructive design. A refined nonlinear small-gain synthesis is employed to address the nonlinear uncertain interconnections between the resulting subsystems corresponding to different virtual control laws, and to guarantee the achievement of the safety control objective. When the safety constraints are removed, the proposed approach coincides with the standard constructive nonlinear control. The proposed safety-control algorithm is experimentally validated in a testbed involving a vertical takeoff and landing (VTOL) vehicle taking off in narrow spaces.

This paper presents a new systematic framework for nonlinear singularly perturbed systems in which state-dependent perturbation functions are used instead of constant perturbation coefficients. Under this framework, general results are obtained for the global robust stability and input-to-state stability of nonlinear singularly perturbed systems. Interestingly, the proposed methodology provides innovative solutions beyond traditional singular perturbation theory for emerging control problems arising from nonlinear integral control, feedback optimization, and formation-based source seeking.

This paper studies the effect of perturbations on the gradient flow of a general nonlinear programming problem, where the perturbation may arise from inaccurate gradient estimation in the setting of data-driven optimization. Under suitable conditions on the objective function, the perturbed gradient flow is shown to be small-disturbance input-to-state stable (ISS), which implies that, in the presence of a small-enough perturbation, the trajectories of the perturbed gradient flow must eventually enter a small neighborhood of the optimum. This work was motivated by the question of robustness of direct methods for the linear quadratic regulator problem, and specifically the analysis of the effect of perturbations caused by gradient estimation or round-off errors in policy optimization. We show small-disturbance ISS for three of the most common optimization algorithms: standard gradient flow, natural gradient flow, and Newton gradient flow., Comment: 20 pages

This paper proposes a novel robust reinforcement learning framework for discrete-time linear systems with model mismatch that may arise from the sim-to-real gap. A key strategy is to invoke advanced techniques from control theory. Using the formulation of the classical risk-sensitive linear quadratic Gaussian control, a dual-loop policy optimization algorithm is proposed to generate a robust optimal controller. The dual-loop policy optimization algorithm is shown to be globally and uniformly convergent, and robust against disturbances during the learning process. This robustness property is called small-disturbance input-to-state stability and guarantees that the proposed policy optimization algorithm converges to a small neighborhood of the optimal controller as long as the disturbance at each learning step is relatively small. In addition, when the system dynamics is unknown, a novel model-free off-policy policy optimization algorithm is proposed. Finally, numerical examples are provided to illustrate the proposed algorithm., Comment: 27 Pages, 13 Figures

This paper studies the control problem for safety-critical multi-agent systems based on quadratic programming (QP). Each controlled agent is modeled as a cascade connection of an integrator and an uncertain nonlinear actuation system. In particular, the integrator represents the position-velocity relation, and the actuation system describes the dynamic response of the actual velocity to the velocity reference signal. The notion of input-to-output stability (IOS) is employed to characterize the essential velocity-tracking capability of the actuation system. The uncertain actuation dynamics may cause infeasibility or discontinuous solutions of QP algorithms for collision avoidance. Also, the interaction between the controlled integrator and the uncertain actuation dynamics may lead to significant robustness issues. By using nonlinear control methods and numerical optimization methods, this paper first contributes a new feasible-set reshaping technique and a refined QP algorithm for feasibility, robustness, and local Lipschitz continuity. Then, we present a nonlinear small-gain analysis to handle the inherent interaction for guaranteed safety of the closed-loop multi-agent system. The proposed methods are illustrated by numerical simulations and a physical experiment.

This paper investigates recursive feasibility, recursive robust stability and near-optimality properties of policy iteration (PI). For this purpose, we consider deterministic nonlinear discrete-time systems whose inputs are generated by PI for undiscounted cost functions. We first assume that PI is recursively feasible, in the sense that the optimization problems solved at each iteration admit a solution. In this case, we provide novel conditions to establish recursive robust stability properties for a general attractor, meaning that the policies generated at each iteration ensure a robust \KL-stability property with respect to a general state measure. We then derive novel explicit bounds on the mismatch between the (suboptimal) value function returned by PI at each iteration and the optimal one. Afterwards, motivated by a counter-example that shows that PI may fail to be recursively feasible, we modify PI so that recursive feasibility is guaranteed a priori under mild conditions. This modified algorithm, called PI+, is shown to preserve the recursive robust stability when the attractor is compact. Additionally, PI+ enjoys the same near-optimality properties as its PI counterpart under the same assumptions. Therefore, PI+ is an attractive tool for generating near-optimal stabilizing control of deterministic discrete-time nonlinear systems., Comment: Submitted for review

This paper studies the adaptive optimal control problem for a class of linear time-delay systems described by delay differential equations (DDEs). A crucial strategy is to take advantage of recent developments in reinforcement learning and adaptive dynamic programming and develop novel methods to learn adaptive optimal controllers from finite samples of input and state data. In this paper, the data-driven policy iteration (PI) is proposed to solve the infinite-dimensional algebraic Riccati equation (ARE) iteratively in the absence of exact model knowledge. Interestingly, the proposed recursive PI algorithm is new in the present context of continuous-time time-delay systems, even when the model knowledge is assumed known. The efficacy of the proposed learning-based control methods is validated by means of practical applications arising from metal cutting and autonomous driving., Comment: 12 pages, 8 figures