Search

This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example., This paper has been accepted to 2022 American Control Conference (ACC). 6 pages

This paper is concerned with developing mean-field game models for the evolution of epidemics. Specifically, an agent's decision -- to be socially active in the midst of an epidemic -- is modeled as a mean-field game with health-related costs and activity-related rewards. By considering the fully and partially observed versions of this problem, the role of information in guiding an agent's rational decision is highlighted. The main contributions of the paper are to derive the equations for the mean-field game in both fully and partially observed settings of the problem, to present a complete analysis of the fully observed case, and to present some analytical results for the partially observed case.

This paper proposes a Robust Safe Control Architecture (RSCA) for safe-decision making. The system to be controlled is a vehicle in the presence of bounded disturbances. The RSCA consists of two parts: a Supervisor MPC and a Controller MPC. Both the Supervisor and the Controller are tube MPCs (TMPCs). The Supervisor MPC provides a safety certificate for an operating controller and a backup control input in every step. After an unsafe action by the operating controller is predicted, the Controller MPC takes over the system. In this paper, a method for the computation of a terminal set is proposed, which is robust against changes in road curvature and forces the vehicle to reach a safe reference. Moreover, two important proofs are provided in this paper. First, it is shown that the backup control input is safe to be applied to the system to lead the vehicle to a safe state. Next, the recursive feasibility of the RSCA is proven. By simulating some obstacle avoidance scenarios, the effectiveness of the proposed RSCA is confirmed.

There is ample evidence in the automotive cybersecurity literature that the car brake ECUs can be maliciously reprogrammed. Motivated by such threat, this paper investigates the capabilities of an adversary who can directly control the frictional brake actuators and would like to induce wheel lockup conditions leading to catastrophic road injuries. This paper demonstrates that the adversary despite having a limited knowledge of the tire-road interaction characteristics has the capability of driving the states of the vehicle traction dynamics to a vicinity of the lockup manifold in a finite time by means of a properly designed attack policy for the frictional brakes. This attack policy relies on employing a predefined-time controller and a nonlinear disturbance observer acting on the wheel slip error dynamics. Simulations under various road conditions demonstrate the effectiveness of the proposed attack policy., Comment: Submitted to AutoSec'22@NDSS

This paper presents a method to certify the computational complexity of a standard Branch and Bound method for solving Mixed-Integer Quadratic Programming (MIQP) problems defined as instances of a multi-parametric MIQP. Beyond previous work, not only the size of the binary search tree is considered, but also the exact complexity of solving the relaxations in the nodes by using recent result from exact complexity certification of active-set QP methods. With the algorithm proposed in this paper, a total worst-case number of QP iterations to be performed in order to solve the MIQP problem can be determined as a function of the parameter in the problem. An important application of the proposed method is Model Predictive Control for hybrid systems, that can be formulated as an MIQP that has to be solved in real-time. The usefulness of the proposed method is successfully illustrated in numerical examples., Comment: Paper accepted for presentation at, and publication in the proceedings of, the 2022 American Control Conference

Transmission power systems usually consist of interconnected sub-grids that are operated relatively independently. When a failure happens, it is desirable to localize its impact within the sub-grid where the failure occurs. This paper introduces three interface networks to connect sub-grids, achieving better failure localization while maintaining robust network connectivity. The proposed interface networks are validated with numerical experiments on the IEEE 118-bus test network under both DC and AC power flow models., Accepted to the 2022 American Control Conference (ACC 2022)

In this paper, we investigate a decentralized stochastic control problem with two agents, where a part of the memory of the second agent is also available to the first agent at each instance of time. We derive a structural form for optimal control strategies which allows us to restrict their domain to a set which does not grow in size with time. We also present a dynamic programming (DP) decomposition which can utilize our results to derive optimal strategies for arbitrarily long time horizons. Since obtaining optimal control strategies by solving this DP decomposition is computationally intensive, we present potential resolutions in the form of simplified strategies by imposing additional conditions on our model, and an approximation technique which can be used to implement our results with a bounded loss of optimality.

In this paper, we propose a concept of approximate bisimulation relation for feedforward neural networks. In the framework of approximate bisimulation relation, a novel neural network merging method is developed to compute the approximate bisimulation error between two neural networks based on reachability analysis of neural networks. The developed method is able to quantitatively measure the distance between the outputs of two neural networks with the same inputs. Then, we apply the approximate bisimulation relation results to perform neural networks model reduction and compute the compression precision, i.e., assured neural networks compression. At last, using the assured neural network compression, we accelerate the verification processes of ACAS Xu neural networks to illustrate the effectiveness and advantages of our proposed approximate bisimulation approach.

This paper presents the use of discrete Simultaneous Perturbation Stochastic Approximation (DSPSA) to optimize dynamical models meaningful for personalized interventions in behavioral medicine, with emphasis on physical activity. DSPSA is used to determine an optimal set of model features and parameter values which would otherwise be chosen either through exhaustive search or be specified a priori. The modeling technique examined in this study is Model-on-Demand (MoD) estimation, which synergistically manages local and global modeling, and represents an appealing alternative to traditional approaches such as ARX estimation. The combination of DSPSA and MoD in behavioral medicine can provide individualized models for participant-specific interventions. MoD estimation, enhanced with a DSPSA search, can be formulated to provide not only better explanatory information about a participant’s physical behavior but also predictive power, providing greater insight into environmental and mental states that may be most conducive for participants to benefit from the actions of the intervention. A case study from data collected from a representative participant of the Just Walk intervention is presented in support of these conclusions.

Distributed system applications rely on a fine-grain common sense of time. Existing systems maintain the common sense of time by keeping each independent machine as close as possible to wall-clock time through a combination of software protocols like NTP and GPS signals and/or precision references like atomic clocks. This approach is expensive and has tolerance limitations that require protocols to deal with asynchrony and its performance consequences. Moreover, at data-center scale it is impractical to distribute a physical clock as is done on a chip or printed circuit board. In this paper we introduce a distributed system design that removes the need for physical clock distribution or mechanisms for maintaining close alignment to wall-clock time, and instead provides applications with a perfectly synchronized logical clock. We discuss the abstract frame model (AFM), a mathematical model that underpins the system synchronization. The model is based on the rate of communication between nodes in a topology without requiring a global clock. We show that there are families of controllers that satisfy the properties required for existence and uniqueness of solutions to the AFM, and give examples., 8 pages, 2 figures

This paper proposes a novel active Simultaneous Localization and Mapping (SLAM) method with continuous trajectory optimization over a stochastic robot dynamics model. The problem is formalized as a stochastic optimal control over the continuous robot kinematic model to minimize a cost function that involves the covariance matrix of the landmark states. We tackle the problem by separately obtaining an open-loop control sequence subject to deterministic dynamics by iterative Covariance Regulation (iCR) and a closed-loop feedback control under stochastic robot and covariance dynamics by Linear Quadratic Regulator (LQR). The proposed optimization method captures the coupling between localization and mapping in predicting uncertainty evolution and synthesizes highly informative sensing trajectories. We demonstrate its performance in active landmark-based SLAM using relative-position measurements with a limited field of view., Comment: 8 pages, 4 figures, submitted to American Control Conference 2022

In this paper, we present a new trajectory optimization algorithm for stochastic linear systems which combines Model Predictive Path Integral (MPPI) control with Constrained Covariance Steering (CSS) to achieve high performance with safety guarantees (robustness). Although MPPI can be used to solve complex nonlinear trajectory optimization problems, it may not always handle constraints effectively and its performance may degrade in the presence of unmodeled disturbances. By contrast, CCS can handle probabilistic state and / or input constraints (e.g., chance constraints) and also steer the state covariance of the system to a desired positive definite matrix (control of uncertainty) which both imply that CCS can provide robustness against stochastic disturbances. CCS, however, suffers from scalability issues and cannot handle complex cost functions in general. We argue that the combination of the two methods yields a class of trajectory optimization algorithms that can achieve high performance (a feature of MPPI) while ensuring safety with high probability (a feature of CCS). The efficacy of our algorithm is demonstrated in an obstacle avoidance problem and a circular track path generation problem.

Multi-vehicle collision avoidance is a highly crucial problem due to the soaring interests of introducing autonomous vehicles into the real world in recent years. The safety of these vehicles while they complete their objectives is of paramount importance. Hamilton-Jacobi (HJ) reachability is a promising tool for guaranteeing safety for low-dimensional systems. However, due to its exponential complexity in computation time, no reachability-based methods have been able to guarantee safety for more than three vehicles successfully in unstructured scenarios. For systems with four or more vehicles,we can only empirically validate their safety performance.While reachability-based safety methods enjoy a flexible least-restrictive control strategy, it is challenging to reason about long-horizon trajectories online because safety at any given state is determined by looking up its safety value in a pre-computed table that does not exhibit favorable properties that continuous functions have. This motivates the problem of improving the safety performance of unstructured multi-vehicle systems when safety cannot be guaranteed given any least-restrictive safety-aware collision avoidance algorithm while avoiding online trajectory optimization. In this paper, we propose a novel approach using supervised learning to enhance the safety of vehicles by proposing new initial states in very close neighborhood of the original initial states of vehicles. Our experiments demonstrate the effectiveness of our proposed approach and show that vehicles are able to get to their goals with better safety performance with our approach compared to a baseline approach in wide-ranging scenarios.

This paper introduces a new invariant extended Kalman filter design that produces real-time state estimates and rapid error convergence for the estimation of the human body movement even in the presence of sensor misalignment and initial state estimation errors. The filter fuses the data returned by an inertial measurement unit (IMU) attached to the body (e.g., pelvis or chest) and a virtual measurement of zero stance-foot velocity (i.e., leg odometry). The key novelty of the proposed filter lies in that its process model meets the group affine property while the filter explicitly addresses the IMU placement error by formulating its stochastic process model as Brownian motions and incorporating the error in the leg odometry. Although the measurement model is imperfect (i.e., it does not possess an invariant observation form) and thus its linearization relies on the state estimate, experimental results demonstrate fast convergence of the proposed filter (within 0.2 seconds) during squatting motions even under significant IMU placement inaccuracy and initial estimation errors., Comment: 7 pages, 6 figures, submitted to American Control Conference (ACC)

Cooperative control of Connected and Autonomous Vehicles (CAVs) promises great benefits for mixed traffic. Most existing research focuses on model-based control strategies, assuming that car-following dynamics of human-driven vehicles are explicitly known. In this paper, instead of relying on a parametric car-following model, we introduce a data-driven predictive control strategy to achieve safe and optimal control for CAVs in mixed traffic. We first present a linearized dynamical model for mixed traffic systems, and investigate its controllability and observability. Based on these control-theoretic properties, we then propose a novel DeeP-LCC (Data-EnablEd Predictive Leading Cruise Control) strategy for CAVs based on measurable driving data to smooth mixed traffic. Our method is implemented in a receding horizon manner, in which input/output constraints are incorporated to achieve collision-free guarantees. Nonlinear traffic simulations reveal its saving of up to 24.96% fuel consumption during a braking scenario of Extra-Urban Driving Cycle while ensuring safety., 7 figures, 3 figures

In the context of constraint-driven control of multi-robot systems, in this paper, we propose an optimization-based framework that is able to ensure resilience and energy-awareness of teams of robots. The approach is based on a novel, frame-theoretic, measure of resilience which allows us to analyze and enforce resilient behaviors of multi-robot systems. The properties of resilience and energy-awareness are encoded as constraints of a convex optimization program which is used to synthesize the robot control inputs. This allows for the combination of such properties with the execution of coordinated tasks to achieve resilient and energy-aware robot operations. The effectiveness of the proposed method is illustrated in a simulated scenario where a team of robots is deployed to execute two tasks subject to energy and resilience constraints.

The wind preview provided by a nacelle-based lidar system allows the wind turbine controller to react to the wind disturbance prior to its impact on the turbine. This technology, commonly referred to as lidar-assisted wind turbine control, has been shown to be beneficial in reducing wind turbine structural loads. The wind preview quality defines how the lidar estimated disturbance is correlated with the actual one. In practice, the preview quality can vary following the change in atmospheric conditions and lidar operating states. When assessing the benefits of lidar-assisted control, previous studies mainly focused on the freestream turbulence where the turbine wake has not been included. In reality, wind turbines sometimes operate within the wake caused by upstream situated turbines, which happens more often in a narrowly spaced wind farm. Based on existing literature, the wake turbulence has three main phenomena compared with the freestream turbulence, i.e. (1) the reduced wind speed region (wake deficit), (2) the meandering (wake deficit moves in the lateral and vertical directions), and (3) the smaller-scale added turbulence caused by the interaction between rotor and the flow. The extent to which these phenomena affect the quality of lidar wind preview still needs to be investigated. In this paper, we use the dynamic wake meandering model, which covers the three wake characteristics mentioned above, and analyze its impact on lidar wind preview qualities. The most representative turbine layout where two turbines lie in a row will be considered. Frequency-domain analysis will be carried out to assess the measurement coherence of the lidar and the results will be compared to the freestream case.

Machine learning techniques, including Gaussian processes (GPs), are expected to play a significant role in meeting speed, accuracy, and functionality requirements in future data-intensive mechatronic systems. This paper aims to reveal the potential of GPs for motion control applications. Successful applications of GPs for feedforward and learning control, including the identification and learning for noncausal feedforward, position-dependent snap feedforward, nonlinear feedforward, and GP-based spatial repetitive control, are outlined. Experimental results on various systems, including a desktop printer, wirebonder, and substrate carrier, confirmed that data-based learning using GPs can significantly improve the accuracy of mechatronic systems.

Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems. However, its deployment in real-world power systems has been hindered by a lack of formal stability and safety guarantees. In this paper, we propose a stability constrained reinforcement learning method for real-time voltage control in distribution grids and we prove that the proposed approach provides a formal voltage stability guarantee. The key idea underlying our approach is an explicitly constructed Lyapunov function that certifies stability. We demonstrate the effectiveness of the approach in case studies, where the proposed method can reduce the transient control cost by more than 30\% and shorten the response time by a third compared to a widely used linear policy, while always achieving voltage stability. In contrast, standard RL methods often fail to achieve voltage stability.

The growing need for improved battery fast charging algorithms and management systems is pushing forward the development of high-fidelity electrochemical models of cells. Critical to the accuracy of these models is their parameterisation, however this challenge remains unresolved, both in terms of theoretical analysis and practical implementation. This paper develops a framework to analyse from impedance measurements the identifiability of electrolyte dynamics—a subcomponent of a general Li-ion model that is key to enabling accurate fast charging simulations. By assuming that the electrolyte volume fractions in the electrode and separator regions are equal, an analytic expression for the impedance function of the electrolyte dynamics is obtained, and this can be tested for structural identifiability. It is shown that the only parameters of the electrolyte model that may be identified are the diffusion time scale and a geometric coupling parameter. Simulations highlight the identifiability issues of electrolyte dynamics (relating to symmetric cells) and explain how the electrolyte parameters might be identified.

Path integral policy improvement (PI2) is a data-driven method for solving stochastic optimal control problems. Both feedforward and feedback controls are calculated based on a sample of noisy open-loop trajectories of the system and their costs, which can be obtained in a highly parallelizable manner. The control strategy offers theoretical performance guarantees related to the expected cost achieved by the resulting closed-loop system. This paper extends the single-agent case to a multi-agent setting, where such theoretical guarantees have not been attained previously. We provide both a decentralized and a leader-follower scheme for distributing the feedback calculations under different communication constraints. The theoretical results are verified numerically through simulations. QC 20221111Part of proceedings: ISBN 978-1-6654-5196-3

District cooling energy plants (DCEPs) consisting of chillers, cooling towers, and thermal energy storage (TES) systems consume a considerable amount of electricity. Optimizing the scheduling of the TES and chillers to take advantage of time-varying electricity price is a challenging optimal control problem. The classical method, model predictive control (MPC), requires solving a high dimensional mixed-integer nonlinear program (MINLP) because of the on/off actuation of the chillers and charging/discharging of TES, which are computationally challenging. RL is an attractive alternative to MPC: the real time control computation is a low-dimensional optimization problem that can be easily solved. However, the performance of an RL controller depends on many design choices. In this paper, we propose a Q-learning based reinforcement learning (RL) controller for this problem. Numerical simulation results show that the proposed RL controller is able to reduce energy cost over a rule-based baseline controller by approximately 8%, comparable to savings reported in the literature with MPC for similar DCEPs. We describe the design choices in the RL controller, including basis functions, reward function shaping, and learning algorithm parameters. Compared to existing work on RL for DCEPs, the proposed controller is designed for continuous state and actions spaces., Comment: 10 pages, extended ACC2022 version

We investigate the problem of designing optimal stealthy poisoning attacks on the control channel of Markov decision processes (MDPs). This research is motivated by the recent interest of the research community for adversarial and poisoning attacks applied to MDPs, and reinforcement learning (RL) methods. The policies resulting from these methods have been shown to be vulnerable to attacks perturbing the observations of the decision-maker. In such an attack, drawing inspiration from adversarial examples used in supervised learning, the amplitude of the adversarial perturbation is limited according to some norm, with the hope that this constraint will make the attack imperceptible. However, such constraints do not grant any level of undetectability and do not take into account the dynamic nature of the underlying Markov process. In this paper, we propose a new attack formulation, based on information-theoretical quantities, that considers the objective of minimizing the detectability of the attack as well as the performance of the controlled process. We analyze the trade-off between the efficiency of the attack and its detectability. We conclude with examples and numerical simulations illustrating this trade-off.

The ever-increasing power capacities of wind turbines promote the use of tall and slender turbine towers. This poses a challenge from a fatigue loading perspective by the relocation of the first and lightly-damped tower side-side natural frequency into the turbine operating regime, promoting its excitation during nominal operation. The excitation of this resonance can be aggravated by periodic loading in the presence of rotor mass and/or aerodynamic imbalance. Earlier work already presented a method to prevent the side-side excitation using a combination of model demodulation and quasilinear parameter varying model predictive control techniques. However, the method does not incorporate features for active control for side-side load mitigations. Because the information of the beforementioned periodic side-side loading is unknown and unmeasurable in practical scenarios, this paper presents a Kalman filtering method for its estimation in a demodulated fashion. The Kalman filter employs an extended demodulated wind turbine model augmented with random walk models of the periodic load. The simulation result demonstrates the effectiveness of the proposed method in estimating the periodic load components along with unmeasurable tower states in their demodulated form. These estimates pose an opportunity for use in future advanced controller designs for active load reductions.

This paper presents a parameterization of nonlinear controllers for uncertain systems building on a recently developed neural network architecture, called the recurrent equilibrium network (REN), and a nonlinear version of the Youla parameterization. The proposed framework has "built-in" guarantees of stability, i.e., all policies in the search space result in a contracting (globally exponentially stable) closed-loop system. Thus, it requires very mild assumptions on the choice of cost function and the stability property can be generalized to unseen data. Another useful feature of this approach is that policies are parameterized directly without any constraints, which simplifies learning by a broad range of policy-learning methods based on unconstrained optimization (e.g. stochastic gradient descent). We illustrate the proposed approach with a variety of simulation examples., Comment: submitted to ACC2022

Increasingly stringent throughput requirements in the industry necessitate the need for lightweight design of high-precision motion systems to allow for high accelerations, while still achieving accurate positioning of the moving-body. The presence of position dependent dynamics in such motion systems severely limits achievable position tracking performance using conventional sequential loop closing (SLC) control design strategies. This paper presents a novel extension of the conventional SLC design framework towards linear-parameter-varying systems, which allows to circumvent limitations that are introduced by position dependent effects in high-precision motion systems. Advantages of the proposed control design approach are demonstrated in simulation using a high-fidelity model of a moving-magnet planar actuator system, which exhibits position dependency in both actuation and sensing., 6 pages

This paper studies the impact of imperfect information in online control with adversarial disturbances. In particular, we consider both delayed state feedback and inexact predictions of future disturbances. We introduce a greedy, myopic policy that yields a constant competitive ratio against the offline optimal policy. We also analyze the fundamental limits of online control with limited information by showing that our competitive ratio bounds for the greedy, myopic policy in the adversarial setting match (up to lower-order terms) lower bounds in the stochastic setting.

In this paper, a novel electrochemical model for LiFePO$_4$ battery cells that accounts for the positive particle lithium intercalation and deintercalation dynamics is proposed. Starting from the enhanced single particle model, mass transport and balance equations along with suitable boundary conditions are introduced to model the phase transformation phenomena during lithiation and delithiation in the positive electrode material. The lithium-poor and lithium-rich phases are modeled using the core-shell principle, where a core composition is encapsulated with a shell composition. The coupled partial differential equations describing the phase transformation are discretized using the finite difference method, from which a system of ordinary differential equations written in state-space representation is obtained. Finally, model parameter identification is performed using experimental data from a 49Ah LFP pouch cell.

In this paper, we extend a framework that we developed earlier for coordination of connected and automated vehicles (CAVs) at a signal-free intersection to incorporate uncertainty. Using the possibly noisy observations of actual time trajectories and leveraging Gaussian process regression, we learn the bounded confidence intervals for deviations from the nominal trajectories of CAVs online. Incorporating these confidence intervals, we reformulate the trajectory planning as a robust coordination problem, the solution of which guarantees that constraints in the system are satisfied in the presence of bounded deviations from the nominal trajectories. We demonstrate the effectiveness of our extended framework through a numerical simulation., Comment: 8 pages, 6 figures

This paper presents a variant of the Traveling Salesman Problem (TSP) with nonholonomic constraints and dynamic obstacles, with optimal control applications in the mining industry. The problem is discretized and an approach for solving the discretized problem to optimality is proposed. The proposed approach solves the three subproblems (waypoint ordering, heading at each waypoint and motion planning between waypoints) simultaneously using two nested graph-search planners. The higher-level planner solves the waypoint ordering and heading subproblems while making calls to the lower-level planner that solves the motion planning subproblem using a lattice-based motion planner. For the higher-level motion planner A* search is used and two different heuristics, a minimal spanning tree heuristic and a nearest insertion heuristic, are proposed and optimality bounds are proven. The proposed planner is evaluated on numerical examples and compared to Dijkstras algorithm. Furthermore, the performance and observed suboptimality are investigated when the minimal spanning tree heuristic cost is inflated. Funding Agencies|Wallenberg AI, Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation

This paper presents the results of a model predictive controller (MPC) development for diesel engine air-path regulation. The control objective is to track the intake manifold pressure and exhaust gas recirculation (EGR) rate targets by manipulating the EGR valve and variable geometry turbine (VGT) while satisfying state and control constraints. The MPC controller is designed and verified using a high-fidelity engine model in GT-Power. The controller exploits a low-order rate-based linear parameter-varying (LPV) model for prediction which is identified from transient response data generated by the GT-Power model. It is shown that transient engine thermal dynamics influence the airpath dynamics, specifically the intake manifold pressure response, however, MPC demonstrates robustness against inaccuracies in modeling these thermal dynamics. In particular, we show that MPC can be successfully implemented using a rate-based prediction model with two inputs (EGR and VGT positions) identified from data with steady-state wall temperature dynamics, however, closed-loop performance can be improved if a prediction model (i) is identified from data with transient thermal dynamics, and (ii) has the fuel injection rate as extra model input. Further, the MPC calibration process across the engine operating range to achieve improved performance is addressed. As the MPC calibration is shown to be sensitive to the operating conditions, a fast calibration process is proposed., Comment: 2022 American Control Conference (ACC), June 8-10, 2022, Atlanta, GA, USA

In this paper, we propose an accelerated version of Simultaneous Perturbation Stochastic Approximation (Accelerated SPSA). This algorithm belongs to the class of methods used in derivative-free optimization and has proven efficacy in the problems including significant non-statistical uncertainties. We focus on analysis of Accelerated SPSA in a non-stationary setting and consider the presence of unknown-but-bounded disturbances. Research on these problems covers many directions. However, in large-scale systems, efficiency still remains a concern. It gave rise to the research where acceleration represents an objective in the algorithm’s design. This problem motivated us to extend our previous research on SPSA in the direction of acceleration. We show that the proposed new accelerated version converges faster than the initial one. The validation of the algorithm is preformed in a target tracking problem.

This paper presents a framework for designing provably safe feedback controllers for sampled-data control affine systems with measurement and actuation uncertainties. Based on the interval Taylor model of nonlinear functions, a sampled-data control barrier function (CBF) condition is proposed which ensures the forward invariance of a safe set for sampled-data systems. Reachable set overapproximation and Lasserre's hierarchy of polynomial optimization are used for finding the margin term in the sampled-data CBF condition. Sufficient conditions for a safe controller in the presence of measurement and actuation uncertainties are proposed, for CBFs with relative degree 1 and higher relative degree individually. The effectiveness of the proposed method is illustrated by two numerical examples and an experimental example that implements the proposed controller on the Crazyflie quadcopter in real-time., 8 pages, 4 figures

In this paper, we study robustness of input metering policies in dynamic flow networks in the presence of transient disturbances and attacks. We consider a compartmental model for dynamic flow networks with a First-In-First-Out (FIFO) routing rule as found in, e.g., transportation networks. We model the effect of the transient disturbance as an abrupt change to the state of the network and use the notion of the region of attraction to measure the resilience of the network to these changes. For constant and periodic input metering, we introduce the notion of monotone-invariant points to establish inner-estimates for the regions of attraction of free-flow equilibrium points and free-flow periodic orbits using monotone systems theory. These results are applicable to, e.g., networks with cycles, which have not been considered in prior literature on dynamic flow networks with FIFO routing. Finally, we propose two approaches for finding suitable monotone-invariant points in the flow networks with FIFO rules.

This paper develops a novel physics-based approach for fault-resilient multi-quadcopter coordination in the presence of abrupt quadcopter failure. Our approach consists of two main layers: (i) high-level physics-based guidance to safely plan the desired recovery trajectory for every healthy quadcopter and (ii) low-level trajectory control design by choosing an admissible control for every healthy quadcopter to safely recover from the anomalous situation, arisen from quadcopter failure, as quickly as possible. For the high-level trajectory planning, first, we consider healthy quadcopters as particles of an irrotational fluid flow sliding along streamline paths wrapping failed quadcopters in the shared motion space. We then obtain the desired recovery trajectories by maximizing the sliding speeds along the streamline paths such that the rotor angular speeds of healthy quadcopters do not exceed certain upper bounds at all times during the safety recovery. In the low level, a feedback linearization control is designed for every healthy quadcopter such that quadcopter rotor angular speeds remain bounded and satisfy the corresponding safety constraints. Simulation results are given to illustrate the efficacy of the proposed method.

In this paper, we examine an important problem of learning neural networks that certifiably meet certain specifications on input-output behaviors. Our strategy is to find an inner approximation of the set of admissible policy parameters, which is convex in a transformed space. To this end, we address the key technical challenge of convexifying the verification condition for neural networks, which is derived by abstracting the nonlinear specifications and activation functions with quadratic constraints. In particular, we propose a reparametrization scheme of the original neural network based on loop transformation, which leads to a convex condition that can be enforced during learning. This theoretical construction is validated in an experiment that specifies reachable sets for different regions of inputs.

Motion planning methods for autonomous systems based on nonlinear programming offer great flexibility in incorporating various dynamics, objectives, and constraints. One limitation of such tools is the difficulty of efficiently representing obstacle avoidance conditions for non-trivial shapes. For example, it is possible to define collision avoidance constraints suitable for nonlinear programming solvers in the canonical setting of a circular robot navigating around M convex polytopes over N time steps. However, it requires introducing (2+L)MN additional constraints and LMN additional variables, with L being the number of halfplanes per polytope, leading to larger nonlinear programs with slower and less reliable solving time. In this paper, we overcome this issue by building closed-form representations of the collision avoidance conditions by outer-approximating the Minkowski sum conditions for collision. Our solution requires only MN constraints (and no additional variables), leading to a smaller nonlinear program. On motion planning problems for an autonomous car and quadcopter in cluttered environments, we achieve speedups of 4.8x and 8.7x respectively with significantly less variance in solve times and negligible impact on performance arising from the use of outer approximations., Comment: 8 pages, 6 figures. Accepted for publication at the 2022 American Control Conference

Volt-VAR and Volt-Watt control functions are mechanisms that are included in distributed energy resource (DER) power electronic inverters to mitigate excessively high or low voltages in distribution systems. In the event that a subset of DER have had their Volt-VAR and Volt-Watt settings compromised as part of a cyber-attack, we propose a mechanism to control the remaining set of non-compromised DER to ameliorate large oscillations in system voltages and large voltage imbalances in real time. To do so, we construct control policies for individual non-compromised DER, directly searching the policy space using an Adam-based augmented random search (ARS). In this paper we show that, compared to previous efforts aimed at training policies for DER cybersecurity using deep reinforcement learning (DRL), the proposed approach is able to learn optimal (and sometimes linear) policies an order of magnitude faster than conventional DRL techniques (e.g., Proximal Policy Optimization).

In this paper, we analyze the optimization landscape of gradient descent methods for static output feedback (SOF) control of discrete-time linear time-invariant systems with quadratic cost. The SOF setting can be quite common, for example, when there are unmodeled hidden states in the underlying process. We first establish several important properties of the SOF cost function, including coercivity, L-smoothness, and M-Lipschitz continuous Hessian. We then utilize these properties to show that the gradient descent is able to converge to a stationary point at a dimension-free rate. Furthermore, we prove that under some mild conditions, gradient descent converges linearly to a local minimum if the starting point is close to one. These results not only characterize the performance of gradient descent in optimizing the SOF problem, but also shed light on the efficiency of general policy gradient methods in reinforcement learning.