Search

Validating and controlling safety-critical systems in uncertain environments necessitates probabilistic reachable sets of future state evolutions. The existing methods of computing probabilistic reachable sets normally assume that the uncertainties are independent of the state. However, this assumption falls short in many real-world applications, where uncertainties are state-dependent, referred to as contextual uncertainties. This paper formulates the problem of computing probabilistic reachable sets of stochastic nonlinear states with contextual uncertainties by seeking minimum-volume polynomial sublevel sets with contextual chance constraints. The formulated problem cannot be solved by the existing sample-based approximation method since the existing methods do not consider the conditional probability densities. To address this, we propose a consistent sample approximation of the original problem by leveraging the conditional density estimation and resampling. The obtained approximate problem is a tractable optimization problem. Additionally, we prove the almost uniform convergence of the proposed sample-based approximation, showing that it gives the optimal solution almost consistently with the original ones. Through a numerical example, we evaluate the effectiveness of the proposed method against existing approaches, highlighting its capability to significantly reduce the bias inherent in sample-based approximation without considering a conditional probability density.

Safety is an indispensable requirement for applying reinforcement learning (RL) to real problems. Although there has been a surge of safe RL algorithms proposed in recent years, most existing work typically 1) relies on receiving numeric safety feedback; 2) does not guarantee safety during the learning process; 3) limits the problem to a priori known, deterministic transition dynamics; and/or 4) assume the existence of a known safe policy for any states. Addressing the issues mentioned above, we thus propose Long-term Binaryfeedback Safe RL (LoBiSaRL), a safe RL algorithm for constrained Markov decision processes (CMDPs) with binary safety feedback and an unknown, stochastic state transition function. LoBiSaRL optimizes a policy to maximize rewards while guaranteeing a long-term safety that an agent executes only safe state-action pairs throughout each episode with high probability. Specifically, LoBiSaRL models the binary safety function via a generalized linear model (GLM) and conservatively takes only a safe action at every time step while inferring its effect on future safety under proper assumptions. Our theoretical results show that LoBiSaRL guarantees the long-term safety constraint, with high probability. Finally, our empirical results demonstrate that our algorithm is safer than existing methods without significantly compromising performance in terms of reward., Comment: Accepted to AAAI-24

In this paper, we investigate the problem of mitigating epidemics by applying an event-triggered control strategy. We consider a susceptible-infected-removed-susceptible (SIRS) model, which builds upon the foundational SIR model by accounting for reinfection cases. The event-triggered control strategy is formulated based on the condition in which the control input (e.g., the level of public measures) is updated only when the fraction of the infected subjects changes over a designed threshold. To synthesize the event-triggered controller, we leverage the notion of a symbolic model, which represents an abstracted expression of the transition system associated with the SIRS model under the event-triggered control strategy. %The symbolic model is constructed based on an approximate alternating simulation relation. Then, by employing safety and reachability games, two event-triggered controllers are synthesized to ensure a desired specification, which is more sophisticated than the one given in previous works. The effectiveness of the proposed approach is illustrated through numerical simulations., Comment: accepted for publication in Nonlinear Analysis: Hybrid Systems (NAHS)

Safe exploration is essential for the practical use of reinforcement learning (RL) in many real-world scenarios. In this paper, we present a generalized safe exploration (GSE) problem as a unified formulation of common safe exploration problems. We then propose a solution of the GSE problem in the form of a meta-algorithm for safe exploration, MASE, which combines an unconstrained RL algorithm with an uncertainty quantifier to guarantee safety in the current episode while properly penalizing unsafe explorations before actual safety violation to discourage them in future episodes. The advantage of MASE is that we can optimize a policy while guaranteeing with a high probability that no safety constraint will be violated under proper assumptions. Specifically, we present two variants of MASE with different constructions of the uncertainty quantifier: one based on generalized linear models with theoretical guarantees of safety and near-optimality, and another that combines a Gaussian process to ensure safety with a deep RL algorithm to maximize the reward. Finally, we demonstrate that our proposed algorithm achieves better performance than state-of-the-art algorithms on grid-world and Safety Gym benchmarks without violating any safety constraints, even during training., Comment: Accepted to NeurIPS 2023

In this paper, we consider a way to safely navigate the robots in unknown environments using measurement data from sensory devices. The control barrier function (CBF) is one of the promising approaches to encode safety requirements of the system and the recent progress on learning-based approaches for CBF realizes online synthesis of CBF-based safe controllers with sensor measurements. However, the existing methods are inefficient in the sense that the trained CBF cannot be generalized to different environments and the re-synthesis of the controller is necessary when changes in the environment occur. Thus, this paper considers a way to learn CBF that can quickly adapt to a new environment with few amount of data by utilizing the currently developed Bayesian meta-learning framework. The proposed scheme realizes efficient online synthesis of the controller as shown in the simulation study and provides probabilistic safety guarantees on the resulting controller., Comment: submitted for publication

Signal Temporal Logic (STL) is capable of expressing a broad range of temporal properties that controlled dynamical systems must satisfy. In the literature, both mixed-integer programming (MIP) and nonlinear programming (NLP) methods have been applied to solve optimal control problems with STL specifications. However, neither approach has succeeded in solving problems with complex long-horizon STL specifications within a realistic timeframe. This study proposes a new optimization framework, called \textit{STLCCP}, which explicitly incorporates several structures of STL to mitigate this issue. The core of our framework is a structure-aware decomposition of STL formulas, which converts the original program into a difference of convex (DC) programs. This program is then solved as a convex quadratic program sequentially, based on the convex-concave procedure (CCP). Our numerical experiments on several commonly used benchmarks demonstrate that this framework can effectively handle complex scenarios over long horizons, which have been challenging to address even using state-of-the-art optimization methods., Comment: 17 pages

This study considers the control problem with signal temporal logic (STL) specifications. Prior works have adopted smoothing techniques to address this problem within a feasible time frame and solve the problem by applying sequential quadratic programming (SQP) methods naively. However, one of the drawbacks of this approach is that solutions can easily become trapped in local minima that do not satisfy the specification. In this study, we propose a new optimization method, termed CCP-based SQP, based on the convex-concave procedure (CCP). Our framework includes a new robustness decomposition method that decomposes the robustness function into a set of constraints, resulting in a form of difference of convex (DC) program that can be solved efficiently. We solve this DC program sequentially as a quadratic program by only approximating the disjunctive parts of the specifications. Our experimental results demonstrate that our method has a superior performance compared to the state-of-the-art SQP methods in terms of both robustness and computational time., Comment: 8 pages, 5 figures

In this paper, we propose a control synthesis method for signal temporal logic (STL) specifications with neural networks (NNs). Most of the previous works consider training a controller for only a given STL specification. These approaches, however, require retraining the NN controller if a new specification arises and needs to be satisfied, which results in large consumption of memory and inefficient training. To tackle this problem, we propose to construct NN controllers by introducing encoder-decoder structured NNs with an attention mechanism. The encoder takes an STL formula as input and encodes it into an appropriate vector, and the decoder outputs control signals that will meet the given specification. As the encoder, we consider three NN structures: sequential, tree-structured, and graph-structured NNs. All the model parameters are trained in an end-to-end manner to maximize the expected robustness that is known to be a quantitative semantics of STL formulae. We compare the control performances attained by the above NN structures through a numerical experiment of the path planning problem, showing the efficacy of the proposed approach., Comment: submitted for publication

This paper investigates the design of self-triggered control for networked control systems (NCS), where the dynamics of the plant is unknown apriori. To deal with the nature of the self-triggered control, in which state measurements are transmitted to the controller a-periodically, we propose to lift the continuous-time dynamics to a novel dynamical model by taking an inter-event time as an additional input, and then, the lifted model is learned by the Gaussian processes (GP) regression. Moreover, we propose a learning-based approach, in which a self-triggered controller is learned by minimizing a cost function, such that it can take inter-sample behavior into account. By employing the lifting approach, we can utilize a gradient-based policy update as an efficient method to optimize both control and communication policies. Finally, we summarize the overall algorithm and provide a numerical simulation to illustrate the effectiveness of the proposed approach., Comment: submitted for publication

Event-triggered control strategy is capable of significantly reducing the number of control task executions without sacrificing control performance. In this paper, we propose a novel learning-based approach towards an event-triggered model predictive control (MPC) for nonlinear control systems whose dynamics are unknown apriori. In particular, the optimal control problems (OCPs) are formulated based on predictive states learned by Gaussian process (GP) regression under a terminal constraint constructed by a symbolic abstraction. The event-triggered condition proposed in this paper is derived from the recursive feasibility so that the OCPs are solved only when an error between the predictive and the actual states exceeds a certain threshold. Based on the event-triggered condition, we analyze the stability of the closed-loop system and show that the finite-time convergence to the terminal set is achieved as the uncertainty of the GP model becomes smaller. Moreover, in order to reduce the uncertainty of the GP model and increase efficiency to find the optimal solution, we provide an overall learning-based event-triggered MPC algorithm based on an iterative task. Finally, we demonstrate the proposed approach through a tracking control problem., Comment: submitted for publication

In this paper, a method for learning a recurrent neural network (RNN) controller that maximizes the robustness of signal temporal logic (STL) specifications is presented. In contrast to previous methods, we consider synthesizing the RNN controller for which the user is able to select an STL specification arbitrarily from multiple STL specifications. To obtain such a controller, we propose a novel notion called STL2vec, which represents a vector representation of the STL specifications and exhibits their similarities. The construction of the STL2vec is useful since it allows us to enhance the efficiency and performance of the RNN controller. We validate our proposed method through the examples of the path planning problem., Comment: submitted for publication

This paper investigates a collaborative rover-copter path planning and exploration with temporal logic specifications under uncertain environments. The objective of the rover is to complete a mission expressed by a syntactically co-safe linear temporal logic (scLTL) formula, while the objective of the copter is to actively explore the environment and reduce its uncertainties, aiming at assisting the rover and enhancing the efficiency of the mission completion. To formalize our approach, we first capture the environmental uncertainties by environmental beliefs of the atomic propositions, under an assumption that it is unknown which properties (or, atomic propositions) are satisfied in each area of the environment. The environmental beliefs of the atomic propositions are updated according to the Bayes rule based on the Bernoulli-type sensor measurements provided by both the rover and the copter. Then, the optimal policy for the rover is synthesized by maximizing a belief of the satisfaction of the scLTL formula through an implementation of an automata-based model checking. An exploration policy for the copter is then synthesized by employing the notion of an entropy that is evaluated based on the environmental beliefs of the atomic propositions, and a path that the rover intends to follow according to the optimal policy. As such, the copter can actively explore regions whose uncertainties are high and that are relevant to the mission completion. Finally, some numerical examples illustrate the effectiveness of the proposed approach., Comment: This paper is accepted in ACM Transactions on Cyber Physical Systems (ACM TCPS)

In this paper, we investigate the problem of designing event-triggered controllers for containing epidemic processes in complex networks. We focus on a deterministic susceptible-infected-susceptible (SIS) model, which is one of the well-known, fundamental models that capture the epidemic spreading. The event-triggered control is particularly formulated in the context of viral spreading, in which control inputs (e.g., the amount of medical treatments, a level of traffic regulations) for each subpopulation are updated only when the fraction of the infected people in the subpopulation exceeds a prescribed threshold. We analyze stability of the proposed event-triggered controller, and derives a sufficient condition for a prescribed control objective to be achieved. Moreover, we propose a novel emulation-based approach towards the design of the event-triggered controller, and show that the problem of designing the event-triggered controller can be solved in polynomial time using geometric programming. We illustrate the effectiveness of the proposed approach through numerical simulations using an air transportation network., Comment: Accepted to Annual Reviews in Control.The first and second authors are contributed equally to this work

This paper presents a framework of learning parameter space for event-triggered control. In particular, our goal is to find a set of parameters for the event-triggered condition, such that certain specifications on safety and convergence properties are satisfied. The exploration strategy is based on the Gaussian process-based active learning, in which, for each iteration, the parameter with having the largest variance is evaluated. Moreover, we provide a theoretical analysis, so that the derived parameter space satisfies both convergence and safety. Finally, a numerical simulation is given to illustrate the effectiveness of the approach., Comment: Presented at 2020 IEEE International Conference on Control and Automation (IEEE ICCA)

Symbolic models or abstractions are known to be powerful tools for the control design of cyber-physical systems (CPSs) with logic specifications. In this paper, we investigate a novel learning-based approach to the construction of symbolic models for nonlinear control systems. In particular, the symbolic model is constructed based on learning the un-modeled part of the dynamics from training data based on state-space exploration, and the concept of an alternating simulation relation that represents behavioral relationships with respect to the original control system. Moreover, we aim at achieving safe exploration, meaning that the trajectory of the system is guaranteed to be in a safe region for all times while collecting the training data. In addition, we provide some techniques to reduce the computational load, in terms of memory and computation time, of constructing the symbolic models and the safety controller synthesis, so as to make our approach practical. Finally, a numerical simulation illustrates the effectiveness of the proposed approach., Comment: Accepted for publication in Automatica

This paper presents a novel method of synthesizing a fragment of a timed discrete event system(TDES),introducing a novel linear temporal logic(LTL), called ticked LTL$_f$. The ticked LTL$_f$ is given as an extension to LTL$_f$, where the semantics is defined over a finite execution fragment. Differently from the standard LTL$_f$, the formula is defined as a variant of metric temporal logic formula, where the temporal properties are described by counting the number of tick in the fragment of the TDES. Moreover, we provide a scheme that encodes the problem into a suitable one that can be solved by an integer linear programming (ILP). The effectiveness of the proposed approach is illustrated through a numerical example of a path planning.

This paper investigates the design of self-triggered controllers for networked control systems (NCSs), where the dynamics of the plant is \textit{unknown} apriori. To deal with the unknown transition dynamics, we employ the Gaussian process (GP) regression in order to learn the dynamics of the plant. To design the self-triggered controller, we formulate an optimal control problem, such that the optimal control and communication policies can be jointly designed based on the GP model of the plant. Moreover, we provide an overall implementation algorithm that jointly learns the dynamics of the plant and the self-triggered controller based on a reinforcement learning framework. Finally, a numerical simulation illustrates the effectiveness of the proposed approach., Comment: appear in IEEE Transactions on Cybernetics

In this paper, we investigate novel self-triggered controllers for nonlinear control systems with reachability and safety specifications. To synthesize the self-triggered controller, we leverage the notion of symbolic models, or abstractions, which represent abstracted expressions of control systems. The symbolic models will be constructed through the concepts of approximate alternating simulation relations, based on which, and by employing a reachability game, the self-triggered controller is synthesized. We illustrate the effectiveness of the proposed approach through numerical simulations., Comment: published in IEEE Control Systems Letters

We propose control and communication strategies for nonlinear networked control systems subject to state and input constraints. The objective is to steer the state of the system towards a prescribed target set in finite time (\textit{reachability}), while at the same time remaining inside a safety set for all time (\textit{safety}). By leveraging the notion of $\delta$-ISS control Lyapunov function, we derive a sufficient condition to generate a communication scheduling, such that the resulting state trajectory guarantees reachability and safety. Moreover, in order to alleviate computational burden we present a way to find a suitable communication scheduling by implementing abstraction schemes and standard graph search methodologies. Simulation examples validate the effectiveness of the proposed approach., Comment: submitted to IEEE TAC

In this paper, we propose a distributed state-and-fault estimation scheme for multi-agent systems. The proposed estimator is based on an $\ell_1$-norm optimization problem, which is inspired by sparse signal recovery in the field of compressive sampling. Two theoretical results are given to analyze the correctness of the proposed approach. First, we provide a necessary and sufficient condition such that state and fault signals are correctly estimated. The result presents a fundamental limitation of the algorithm, which shows how many faulty nodes are allowed to ensure a correct estimation. Second, we provide a sufficient condition for the estimation error of fault signals when numerical errors of solving the optimization problem are present. An illustrative example is given to validate the effectiveness of the proposed approach.

In recent years, event and self-triggered control have been proposed as energy-aware control strategies to expand the life-time of battery powered devices in Networked Control Systems (NCSs). In contrast to the previous works in which their control objective is to achieve stability, this paper presents a novel energy-aware control scheme for achieving high level specifications, or more specifically, temporal logic specifications. Inspired by the standard hierarchical strategy that has been proposed in the field of formal control synthesis paradigm, we propose a new abstraction procedure for jointly synthesizing control and communication strategies, such that the communication reduction in NCSs and the satisfaction of the temporal logic specifications are guaranteed. The benefits of the proposal are illustrated through a numerical example.

Event-triggered and self-triggered control have been proposed in recent years as promising control strategies to reduce communication resources in Networked Control Systems (NCSs). Based on the notion of set-invariance theory, this note presents new self-triggered control strategies for linear discrete-time systems subject to input and state constraints. The proposed schemes not only achieve communication reduction for NCSs, but also ensure both asymptotic stability of the origin and constraint satisfactions. A numerical simulation example validates the effectiveness of the proposed approaches., Comment: To appear in IEEE Transactions on Automatic Control

In this paper, we propose a new aperiodic formulation of model predictive control for nonlinear continuous-time systems. Unlike earlier approaches, we provide event-triggered conditions without using the optimal cost as a Lyapunov function candidate. Instead, we evaluate the time interval when the optimal state trajectory enters a local set around the origin. The obtained event-triggered strategy is more suitable for practical applications than the earlier approaches in two directions. First, it does not include parameters (e.g., Lipschitz constant parameters of stage and terminal costs) which may be a potential source of conservativeness for the event-triggered conditions. Second, the event-triggered conditions are necessary to be checked only at certain sampling time instants, instead of continuously. This leads to the alleviation of the sensing cost and becomes more suitable for practical implementations under a digital platform. The proposed event-triggered scheme is also validated through numerical simulations., Comment: To appear in Automatica

In this paper, a self-triggered control scheme for constrained discrete-time control systems is presented. The key idea of our approach is to construct a transition system or a graph structure from a collection of polyhedral sets, which are generated based on the notion of {set-invariance theory}. The inter-event time steps are then determined through a standard graph search algorithm to obtain the minimal total cost to a terminal state. The proposed strategy is illustrated through a numerical example., Comment: Extended version of ACC 2017

In this paper, we propose a self-triggered formulation of Model Predictive Control for continuous-time nonlinear input-affine networked control systems. Our control method specifies not only when to execute control tasks but also provides a way to discretize the optimal control trajectory into several control samples, so that the reduction of communication load will be obtained. Stability analysis under the sample-and-hold implementation is also given, which guarantees that the state converges to a terminal region where the system can be stabilized by a local state feedback controller. Some simulation examples validate our proposed framework., Comment: To appear in IEEE TAC