Search

Developing trustworthy multi-agent systems for practical applications is challenging due to the complicated communication of situational awareness (SA) among agents. This paper showcases a novel efficient and easy-to-use software framework for multi-agent simulation, named SymAware which provides a rich set of predefined data structures to compute, store, and communicate SA for agents. It also provides an abstract interface for the agents to compute their control inputs taking into account the awareness of the situation, knowledge, and risk of surrounding agents. Besides, utilizing a cluster of specialized components, SymAware hides the heavy computation of physical rendering and communication interfacing of simulation engines behind the control threads, resulting in high implementation efficiency in bridging the gap between conceptual prototyping and practical applications. Three multi-agent case studies are used to validate the efficacy and efficiency of this software framework., Comment: Submitted to ICRA2025

Decision-making for autonomous driving incorporating different types of risks is a challenging topic. This paper proposes a novel risk metric to facilitate the driving task specified by linear temporal logic (LTL) by balancing the risk brought up by different uncertain events. Such a balance is achieved by discounting the costs of these uncertain events according to their timing and severity, thereby reflecting a human-like awareness of risk. We have established a connection between this risk metric and the occupation measure, a fundamental concept in stochastic reachability problems, such that a risk-aware control synthesis problem under LTL specifications is formulated for autonomous vehicles using occupation measures. As a result, the synthesized policy achieves balanced decisions across different types of risks with associated costs, showcasing advantageous versatility and generalizability. The effectiveness and scalability of the proposed approach are validated by three typical traffic scenarios in Carla simulator.

It has been an ambition of many to control a robot for a complex task using natural language (NL). The rise of large language models (LLMs) makes it closer to coming true. However, an LLM-powered system still suffers from the ambiguity inherent in an NL and the uncertainty brought up by LLMs. This paper proposes a novel LLM-based robot motion planner, named \textit{VernaCopter}, with signal temporal logic (STL) specifications serving as a bridge between NL commands and specific task objectives. The rigorous and abstract nature of formal specifications allows the planner to generate high-quality and highly consistent paths to guide the motion control of a robot. Compared to a conventional NL-prompting-based planner, the proposed VernaCopter planner is more stable and reliable due to less ambiguous uncertainty. Its efficacy and advantage have been validated by two small but challenging experimental scenarios, implying its potential in designing NL-driven robots.

This paper develops a correct-by-design controller for an autonomous vehicle interacting with opponent vehicles with unknown intentions. We define an intention-aware control problem incorporating epistemic uncertainties of the opponent vehicles and model their intentions as discrete-valued random variables. Then, we focus on a control objective specified as belief-space temporal logic specifications. From this stochastic control problem, we derive a sound deterministic control problem using stochastic expansion and solve it using shrinking-horizon model predictive control. The solved intention-aware controller allows a vehicle to adjust its behaviors according to its opponents' intentions. It ensures provable safety by restricting the probabilistic risk under a desired level. We show with experimental studies that the proposed method ensures strict limitation of risk probabilities, validating its efficacy in autonomous driving cases. This work provides a novel solution for the risk-aware control of interactive vehicles with formal safety guarantees.

This paper addresses the control synthesis of heterogeneous stochastic linear multi-agent systems with real-time allocation of signal temporal logic (STL) specifications. Based on previous work, we decompose specifications into sub-specifications on the individual agent level. To leverage the efficiency of task allocation, a heuristic filter evaluates potential task allocation based on STL robustness, and subsequently, an auctioning algorithm determines the definitive allocation of specifications. Finally, a control strategy is synthesized for each agent-specification pair using tube-based model predictive control (MPC), ensuring provable probabilistic satisfaction. We demonstrate the efficacy of the proposed methods using a multi-shuttle scenario that highlights a promising extension to automated driving applications like vehicle routing., Comment: 7 pages, 5 figures, Accepted for CDC 2024. arXiv admin note: text overlap with arXiv:2402.03165

This paper concerns the risk-aware control of stochastic systems with temporal logic specifications dynamically assigned during runtime. Conventional risk-aware control typically assumes that all specifications are predefined and remain unchanged during runtime. In this paper, we propose a novel, provably correct model predictive control scheme for linear systems with additive unbounded stochastic disturbances that dynamically evaluates the feasibility of runtime signal temporal logic specifications and automatically reschedules the control inputs accordingly. The control method guarantees the probabilistic satisfaction of newly accepted specifications without sacrificing the satisfaction of the previously accepted ones. The proposed control method is validated by a robotic motion planning case study., Comment: 7 pages, 4 figures, 1 table, Accepted for ADHS 2024

Model Predictive Control (MPC) has been widely applied to the motion planning of autonomous vehicles. An MPC-controlled vehicle is required to predict its own trajectories in a finite prediction horizon according to its model. Beyond this, the vehicle should also incorporate the prediction of the trajectory of its nearby vehicles, or target vehicles (TVs) into its decision-making. The conventional trajectory prediction methods, such as the constant-speed-based ones, are too trivial to accurately capture the potential collision risks. In this report, we propose a novel MPC-based motion planning method for an autonomous vehicle with a set of risk-aware constraints. These constraints incorporate the predicted trajectory of a TV learned using a deep-learning-based method. A recurrent neural network (RNN) is used to predict the TV's future trajectory based on its historical data. Then, the predicted TV trajectory is incorporated into the optimization of the MPC of the ego vehicle to generate collision-free motion. Simulation studies are conducted to showcase the prediction accuracy of the RNN model and the collision-free trajectories generated by the MPC.

The driving style of an Autonomous Vehicle (AV) refers to how it behaves and interacts with other AVs. In a multi-vehicle autonomous driving system, an AV capable of identifying the driving styles of its nearby AVs can reliably evaluate the risk of collisions and make more reasonable driving decisions. However, there has not been a consistent definition of driving styles for an AV in the literature, although it is considered that the driving style is encoded in the AV's trajectories and can be identified using Maximum Entropy Inverse Reinforcement Learning (ME-IRL) methods as a cost function. Nevertheless, an important indicator of the driving style, i.e., how an AV reacts to its nearby AVs, is not fully incorporated in the feature design of previous ME-IRL methods. In this paper, we describe the driving style as a cost function of a series of weighted features. We design additional novel features to capture the AV's reaction-aware characteristics. Then, we identify the driving styles from the demonstration trajectories generated by the Stochastic Model Predictive Control (SMPC) using a modified ME-IRL method with our newly proposed features. The proposed method is validated using MATLAB simulation and an off-the-shelf experiment.

Reinforcement learning (RL) for motion planning of multi-degree-of-freedom robots still suffers from low efficiency in terms of slow training speed and poor generalizability. In this paper, we propose a novel RL-based robot motion planning framework that uses implicit behavior cloning (IBC) and dynamic movement primitive (DMP) to improve the training speed and generalizability of an off-policy RL agent. IBC utilizes human demonstration data to leverage the training speed of RL, and DMP serves as a heuristic model that transfers motion planning into a simpler planning space. To support this, we also create a human demonstration dataset using a pick-and-place experiment that can be used for similar studies. Comparison studies in simulation reveal the advantage of the proposed method over the conventional RL agents with faster training speed and higher scores. A real-robot experiment indicates the applicability of the proposed method to a simple assembly task. Our work provides a novel perspective on using motion primitives and human demonstration to leverage the performance of RL for robot applications.

Reinforcement learning (RL) is an effective approach to motion planning in autonomous driving, where an optimal driving policy can be automatically learned using the interaction data with the environment. Nevertheless, the reward function for an RL agent, which is significant to its performance, is challenging to be determined. The conventional work mainly focuses on rewarding safe driving states but does not incorporate the awareness of risky driving behaviors of the vehicles. In this paper, we investigate how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. Based on the essential requirements that prescribe the safety specifications for general autonomous driving in practice, we propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors. A simulation study in OpenAI Gym indicates the advantage of risk-aware reward shaping for various RL agents. Also, we point out that proximal policy optimization (PPO) is likely to be the best RL method that works with risk-aware reward shaping.

This paper proposes a novel distributed coverage controller for a multi-agent system with constant-speed unicycle robots (CSUR). The work is motivated by the limitation of the conventional method that does not ensure the satisfaction of hard state- and input-dependent constraints and leads to feasibility issues for multi-CSUR systems. In this paper, we solve these problems by designing a novel coverage cost function and a saturated gradient-search-based control law. Invariant set theory and Lyapunov-based techniques are used to prove the state-dependent confinement and the convergence of the system state to the optimal coverage configuration, respectively. The controller is implemented in a distributed manner based on a novel communication standard among the agents. A series of simulation case studies are conducted to validate the effectiveness of the proposed coverage controller in different initial conditions and with control parameters. A comparison study in simulation reveals the advantage of the proposed method in terms of avoiding infeasibility. The experiment study verifies the applicability of the method to real robots with uncertainties. The development procedure of the method from theoretical analysis to experimental validation provides a novel framework for multi-agent system coordinate control with complex agent dynamics.

Finding an efficient way to adapt robot trajectory is a priority to improve overall performance of robots. One approach for trajectory planning is through transferring human-like skills to robots by Learning from Demonstrations (LfD). The human demonstration is considered the target motion to mimic. However, human motion is typically optimal for human embodiment but not for robots because of the differences between human biomechanics and robot dynamics. The Dynamic Movement Primitives (DMP) framework is a viable solution for this limitation of LfD, but it requires tuning the second-order dynamics in the formulation. Our contribution is introducing a systematic method to extract the dynamic features from human demonstration to auto-tune the parameters in the DMP framework. In addition to its use with LfD, another utility of the proposed method is that it can readily be used in conjunction with Reinforcement Learning (RL) for robot training. In this way, the extracted features facilitate the transfer of human skills by allowing the robot to explore the possible trajectories more efficiently and increasing robot compliance significantly. We introduced a methodology to extract the dynamic features from multiple trajectories based on the optimization of human-likeness and similarity in the parametric space. Our method was implemented into an actual human-robot setup to extract human dynamic features and used to regenerate the robot trajectories following both LfD and RL with DMP. It resulted in a stable performance of the robot, maintaining a high degree of human-likeness based on accumulated distance error as good as the best heuristic tuning.

Observer-based methods are widely used to estimate the disturbances of different dynamic systems. However, a drawback of the conventional disturbance observers is that they all assume persistent excitation (PE) of the systems. As a result, they may lead to poor estimation precision when PE is not ensured, for instance, when the disturbance gain of the system is close to the singularity. In this paper, we propose a novel disturbance observer based on concurrent learning (CL) with time-variant history stacks, which ensures high estimation precision even in PE-free cases. The disturbance observer is designed in both continuous and discrete time. The estimation errors of the proposed method are proved to converge to a bounded set using the Lyapunov method. A history-sample-selection procedure is proposed to reduce the estimation error caused by the accumulation of old history samples. A simulation study on epidemic control shows that the proposed method produces higher estimation precision than the conventional disturbance observer when PE is not satisfied. This justifies the correctness of the proposed CL-based disturbance observer and verifies its applicability to solving practical problems.

Reinforcement learning demonstrates significant potential in automatically building control policies in numerous domains, but shows low efficiency when applied to robot manipulation tasks due to the curse of dimensionality. To facilitate the learning of such tasks, prior knowledge or heuristics that incorporate inherent simplification can effectively improve the learning performance. This paper aims to define and incorporate the natural symmetry present in physical robotic environments. Then, sample-efficient policies are trained by exploiting the expert demonstrations in symmetrical environments through an amalgamation of reinforcement and behavior cloning, which gives the off-policy learning process a diverse yet compact initiation. Furthermore, it presents a rigorous framework for a recent concept and explores its scope for robot manipulation tasks. The proposed method is validated via two point-to-point reaching tasks of an industrial arm, with and without an obstacle, in a simulation experiment study. A PID controller, which tracks the linear joint-space trajectories with hard-coded temporal logic to produce interim midpoints, is used to generate demonstrations in the study. The results of the study present the effect of the number of demonstrations and quantify the magnitude of behavior cloning to exemplify the possible improvement of model-free reinforcement learning in common manipulation tasks. A comparison study between the proposed method and a traditional off-policy reinforcement learning algorithm indicates its advantage in learning performance and potential value for applications.

Simulation is essential to reinforcement learning (RL) before implementation in the real world, especially for safety-critical applications like robot manipulation. Conventionally, RL agents are sensitive to the discrepancies between the simulation and the real world, known as the sim-to-real gap. The application of domain randomization, a technique used to fill this gap, is limited to the imposition of heuristic-randomized models. {We investigate the properties of intrinsic stochasticity of real-time simulation (RT-IS) of off-the-shelf simulation software and its potential to improve RL performance. This improvement includes a higher tolerance to noise and model imprecision and superiority to conventional domain randomization in terms of ease of use and automation. Firstly, we conduct analytical studies to measure the correlation of RT-IS with the utilization of computer hardware and validate its comparability with the natural stochasticity of a physical robot. Then, we exploit the RT-IS feature in the training of an RL agent. The simulation and physical experiment results verify the feasibility and applicability of RT-IS to robust agent training for robot manipulation tasks. The RT-IS-powered RL agent outperforms conventional agents on robots with modeling uncertainties. RT-IS requires less heuristic randomization, is not task-dependent, and achieves better generalizability than the conventional domain-randomization-powered agents. Our findings provide a new perspective on the sim-to-real problem in practical applications like robot manipulation tasks.

In many practical scenarios, multi-robot systems are envisioned to support humans in executing complicated tasks within structured environments, such as search-and-rescue tasks. We propose a framework for a multi-robot swarm to fulfill complex tasks represented by temporal logic specifications. Given temporal logic specifications on the swarm formation and navigation, we develop a controller with runtime safety and convergence guarantees that drive the swarm to formally satisfy the specification. In addition, the synthesized controller will autonomously switch formations as necessary and react to uncontrollable events from the environment. The efficacy of the proposed framework is validated with a simulation study on the navigation of multiple quadrotor robots.

The control synthesis of a dynamic system subject to a signal temporal logic (STL) specification is commonly formulated as a mixed-integer linear/convex programming (MILP/MICP) problem. Solving such a problem is computationally expensive when the specification is long and complex. In this paper, we propose a framework to transform a long and complex specification into separate forms in time, to be more specific, the logical combination of a series of short and simple subformulas with non-overlapping timing intervals. In this way, one can easily modularize the synthesis of a long specification by solving its short subformulas, which improves the efficiency of the control problem. We first propose a syntactic timing separation form for a type of complex specifications based on a group of separation principles. Then, we further propose a complete specification split form with subformulas completely separated in time. Based on this, we develop a modularized synthesis algorithm that ensures the soundness of the solution to the original synthesis problem. The efficacy of the methods is validated with a robot monitoring case study in simulation. Our work is promising to promote the efficiency of control synthesis for systems with complicated specifications.

A novel procedure for the online identification of a class of discrete-time switched linear systems, which simultaneously estimates the parameters and switching manifolds of the systems, is proposed in this paper. Firstly, to estimate the parameters of the subsystems, a discrete-time concurrent learning-based recursive parameter estimator is designed to guarantee the exponential convergence of the estimation errors to zero. Secondly, as an assistant procedure of the identification framework, an online switching detection method is proposed by making use of the history stacks produced by the concurrent learning estimators. Thirdly, techniques of incremental support vector machine are applied to develop the recursive algorithm to estimate the system switching manifolds, with its stability proven by a Lynapunov-based method. At the end of the paper, the stability and precision of the proposed identification methods are confirmed by the numerical simulation of a 2-order switched linear system. Compared to the traditional offline identification methods, the proposed online identification framework possesses superior efficiency with respect to large amounts of data, while the limitations and outlook of this framework are also discussed within the conclusion.

This paper proposes a novel observer-based disturbance estimation method for high degree-of-freedom Euler-Lagrangian systems using an unknown input-output (UIO) sliding mode observer (SMO). Different from the previous SMO methods, this approach does not assume the matching condition of the disturbances. Besides, compared to the conventional disturbance estimation methods, the proposed method does not require the calculation of the inverse inertia matrices and accurate measurement of system velocities. This advantage resolves the concerns of heavy computational load and amplified noise for practical problems like external torque estimation of high degree-of-freedom manipulators in safe human-robot collaboration. We achieve this by defining a novel linearized model for the Euler-Lagrangian system and designing a sliding-mode-based disturbance observer. The estimation precision of the observer is ensured by the Lyapunov-based stability proof and the equivalent control theory of sliding mode. At the end of this paper, the method is implemented on a seven-degree-of-freedom robot manipulator, and the experimental results confirm its decent performance and potential to be applied in practice.

Quantifying the average communication rate (ACR) of a networked event-triggered stochastic control system (NET-SCS) with deterministic thresholds is challenging due to the non-stationary nature of the system's stochastic processes. For a NET-SCS, the nonlinear statistics propagation of the network communication status brought up by deterministic thresholds makes the precise computation of ACR difficult. Previous work used to over-simplify the computation using a Gaussian distribution without incorporating this nonlinearity, leading to sacrificed precision. This paper proposes both analytical and numerical approaches to predict the exact ACR for a NET-SCS using a recursive model. We use theoretical analysis and a numerical study to qualitatively evaluate the deviation gap of the conventional approach that ignores the side information. The accuracy of our proposed method, alongside its comparison with the simplified results of the conventional approach, is validated by experimental studies. Our work is promising to benefit the efficient resource planning of networked control systems with limited communication resources by providing accurate ACR computation.

In this note, we develop an adaptive observer for a class of nonlinear systems with switched unknown parameters to estimate the states and parameters simultaneously. The main challenge lies in how to eliminate the disturbance effect of zero-input responses caused by the switching on the parameter estimation. These responses depend on the unknown states at switching instants (SASI) and constitute an additive disturbance to the parameter estimation, which obstructs parameter convergence to zero. Our solution is to treat the zero-input responses as excitations instead of disturbances. This is realized by first augmenting the system parameter with the SASI and then developing an estimator for the augmented parameter using the \textit{dynamic regression extension and mixing} (DREM) technique. Thanks to its property of element-wise parameter adaptation, the system parameter estimation is decoupled from the SASI. As a result, the estimation errors of system states and parameters converge to zero asymptotically. Furthermore, the robustness of the proposed adaptive observer is guaranteed in the presence of disturbances and noise. A numerical example validates the effectiveness of the proposed approach.

Mobile robots are desired with resilience to safely interact with prior-unknown environments and finally accomplish given tasks. This paper utilizes instantaneous local sensory data to stimulate the safe feedback motion planning (SFMP) strategy with adaptability to diverse prior-unknown environments without building a global map. This is achieved by the numerical optimization with the constraints, referred to as instantaneous local control barrier functions (IL-CBFs) and goal-driven control Lyapunov functions (GD-CLFs), learned from perceptional signals. In particular, the IL-CBFs reflecting potential collisions and GD-CLFs encoding incrementally discovered subgoals are first online learned from local perceptual data. Then, the learned IL-CBFs are united with GD-CLFs in the context of quadratic programming (QP) to generate the safe feedback motion planning strategy. Rather importantly, an optimization over the admissible control space of IL-CBFs is conducted to enhance the solution feasibility of QP. The SFMP strategy is developed with theoretically guaranteed collision avoidance and convergence to destinations. Numerical simulations are conducted to reveal the effectiveness of the proposed SFMP strategy that drives mobile robots to safely reach the destination incrementally in diverse prior-unknown environments.