Search

The copper metaborate CuB2O4 (CBO) has been studied within the parametric space of magnetic field (B) and temperature (T), showing a series of distinct phases, including commensurate and incommensurate magnetic orders, as well as magnetic soliton phases. However, no single study has simultaneously demonstrated all these phases. This work presents the first comprehensive construction of the complex B vs T phase diagram for B = 0-9 T (B perpendicular to the c axis) and T = 0.1-25 K, utilizing thermodynamic probes, magnetic measurements, and neutron scattering on a single batch of CBO crystals. Our findings elucidate new magnetic phases and regimes. First-principles calculations help to gain a deeper and intricate understanding of this complex phase diagram. The emergence of various phases within such a narrow B and T regime is attributed to two primary mechanisms: the delicate competition between ferromagnetic and antiferromagnetic interactions, and the weak exchange and superexchange interactions within and between the Cu(A) and Cu(B) sublattices. This competition also leads to a magnetically frustrated ground state at absolute zero where the total magnetic entropy release from high to low temperatures is only 0.76Rln2, lower than Rln2 expected for copper spin 1/2. Taking CBO as an example, the present work elucidates the mechanisms underlying the formation of complex magnetic phases.

In this letter, we address the task of adaptive sampling to model vector fields. When modeling environmental phenomena with a robot, gathering high resolution information can be resource intensive. Actively gathering data and modeling flows with the data is a more efficient alternative. However, in such scenarios, data is often sparse and thus requires flow modeling techniques that are effective at capturing the relevant dynamical features of the flow to ensure high prediction accuracy of the resulting models. To accomplish this effectively, regions with high informative value must be identified. We propose EnKode, an active sampling approach based on Koopman Operator theory and ensemble methods that can build high quality flow models and effectively estimate model uncertainty. For modeling complex flows, EnKode provides comparable or better estimates of unsampled flow regions than Gaussian Process Regression models with hyperparameter optimization. Additionally, our active sensing scheme provides more accurate flow estimates than comparable strategies that rely on uniform sampling. We evaluate EnKode using three common benchmarking systems: the Bickley Jet, Lid-Driven Cavity flow with an obstacle, and real ocean currents from the National Oceanic and Atmospheric Administration (NOAA)., Comment: This work has been submitted to the IEEE for possible publication

Flying quadrotors in tight formations is a challenging problem. It is known that in the near-field airflow of a quadrotor, the aerodynamic effects induced by the propellers are complex and difficult to characterize. Although machine learning tools can potentially be used to derive models that capture these effects, these data-driven approaches can be sample inefficient and the resulting models often do not generalize as well as their first-principles counterparts. In this work, we propose a framework that combines the benefits of first-principles modeling and data-driven approaches to construct an accurate and sample efficient representation of the complex aerodynamic effects resulting from quadrotors flying in formation. The data-driven component within our model is lightweight, making it amenable for optimization-based control design. Through simulations and physical experiments, we show that incorporating the model into a novel learning-based nonlinear model predictive control (MPC) framework results in substantial performance improvements in terms of trajectory tracking and disturbance rejection. In particular, our framework significantly outperforms nominal MPC in physical experiments, achieving a 40.1% improvement in the average trajectory tracking errors and a 57.5% reduction in the maximum vertical separation errors. Our framework also achieves exceptional sample efficiency, using only a total of 46 seconds of flight data for training across both simulations and physical experiments. Furthermore, with our proposed framework, the quadrotors achieve an exceptionally tight formation, flying with an average separation of less than 1.5 body lengths throughout the flight. A video illustrating our framework and physical experiments is given here: https://youtu.be/Hv-0JiVoJGo, Comment: 7 pages, 5 figures

Traditionally, unmanned aerial vehicles (UAVs) rely on CMOS-based cameras to collect images about the world below. One of the most successful applications of UAVs is to generate orthomosaics or orthomaps, in which a series of images are integrated together to develop a larger map. However, the use of CMOS-based cameras with global or rolling shutters mean that orthomaps are vulnerable to challenging light conditions, motion blur, and high-speed motion of independently moving objects under the camera. Event cameras are less sensitive to these issues, as their pixels are able to trigger asynchronously on brightness changes. This work introduces the first orthomosaic approach using event cameras. In contrast to existing methods relying only on CMOS cameras, our approach enables map generation even in challenging light conditions, including direct sunlight and after sunset., Comment: 7 pages, 7 figures

Coordinating the motion of multiple robots in cluttered environments remains a computationally challenging task. We study the problem of minimizing the execution time of a set of geometric paths by a team of robots with state-dependent actuation constraints. We propose a Time-Optimal Path Parameterization (TOPP) algorithm for multiple car-like agents, where the modulation of the timing of every robot along its assigned path is employed to ensure collision avoidance and dynamic feasibility. This is achieved through the use of a priority queue to determine the order of trajectory execution for each robot while taking into account all possible collisions with higher priority robots in a spatiotemporal graph. We show a 10-20% reduction in makespan against existing state-of-the-art methods and validate our approach through simulations and hardware experiments.

Accurate and efficient fluid flow models are essential for applications relating to many physical phenomena including geophysical, aerodynamic, and biological systems. While these flows may exhibit rich and multiscale dynamics, in many cases underlying low-rank structures exist which describe the bulk of the motion. These structures tend to be spatially large and temporally slow, and may contain most of the energy in a given flow. The extraction and parsimonious representation of these low-rank dynamics from high-dimensional data is a key challenge. Inspired by the success of physics-informed machine learning methods, we propose a spectrally-informed approach to extract low-rank models of fluid flows by leveraging known spectral properties in the learning process. We incorporate this knowledge by imposing regularizations on the learned dynamics, which bias the training process towards learning low-frequency structures with corresponding higher power. We demonstrate the effectiveness of this method to improve prediction and produce learned models which better match the underlying spectral properties of prototypical fluid flows., Comment: 13 pages, 10 figures

Soft robots have many advantages over rigid robots thanks to their compliant and passive nature. However, it is generally challenging to model the dynamics of soft robots due to their high spatial dimensionality, making it difficult to use model-based methods to accurately control soft robots. It often requires direct numerical simulation of partial differential equations to simulate soft robots. This not only requires an accurate numerical model, but also makes soft robot modeling slow and expensive. Deep learning algorithms have shown promises in data-driven modeling of soft robots. However, these algorithms usually require a large amount of data, which are difficult to obtain in either simulation or real-world experiments of soft robots. In this work, we propose KNODE-Cosserat, a framework that combines first-principle physics models and neural ordinary differential equations. We leverage the best from both worlds -- the generalization ability of physics-based models and the fast speed of deep learning methods. We validate our framework in both simulation and real-world experiments. In both cases, we show that the robot model significantly improves over the baseline models under different metrics., Comment: 8 pages, 11 figures, 4 tables

One common and desirable application of robots is exploring potentially hazardous and unstructured environments. Air-ground collaboration offers a synergistic approach to addressing such exploration challenges. In this paper, we demonstrate a system for large-scale exploration using a team of aerial and ground robots. Our system uses semantics as lingua franca, and relies on fully opportunistic communications. We highlight the unique challenges from this approach, explain our system architecture and showcase lessons learned during our experiments. All our code is open-source, encouraging researchers to use it and build upon., Comment: 6 pages, 5 figres

Climate change has increased the frequency and severity of extreme weather events such as hurricanes and winter storms. The complex interplay of floods with tides, runoff, and sediment creates additional hazards -- including erosion and the undermining of urban infrastructure -- consequently impacting the health of our rivers and ecosystems. Observations of these underwater phenomena are rare, because satellites and sensors mounted on aerial vehicles cannot penetrate the murky waters. Autonomous Surface Vehicles (ASVs) provides a means to track and map these complex and dynamic underwater phenomena. This work highlights preliminary results of high-resolution data gathering with ASVs, equipped with a suite of sensors capable of measuring physical and chemical parameters of the river. Measurements were acquired along the lower Schuylkill River in the Philadelphia area at high-tide and low-tide conditions. The data will be leveraged to improve our understanding of changes in bathymetry due to floods; the dynamics of mixing and stagnation zones and their impact on water quality; and the dynamics of suspension and resuspension of fine sediment. The data will also provide insight into the development of adaptive sampling strategies for ASVs that can maximize the information gain for future field experiments., Comment: Published to IEEE OCEANS 2022

Multi-robot collaboration in large-scale environments with limited-sized teams and without external infrastructure is challenging, since the software framework required to support complex tasks must be robust to unreliable and intermittent communication links. In this work, we present MOCHA (Multi-robot Opportunistic Communication for Heterogeneous Collaboration), a framework for resilient multi-robot collaboration that enables large-scale exploration in the absence of continuous communications. MOCHA is based on a gossip communication protocol that allows robots to interact opportunistically whenever communication links are available, propagating information on a peer-to-peer basis. We demonstrate the performance of MOCHA through real-world experiments with commercial-off-the-shelf (COTS) communication hardware. We further explore the system's scalability in simulation, evaluating the performance of our approach as the number of robots increases and communication ranges vary. Finally, we demonstrate how MOCHA can be tightly integrated with the planning stack of autonomous robots. We show a communication-aware planning algorithm for a high-altitude aerial robot executing a collaborative task while maximizing the amount of information shared with ground robots. The source code for MOCHA and the high-altitude UAV planning system is available open source: http://github.com/KumarRobotics/MOCHA, http://github.com/KumarRobotics/air_router., Comment: 7 pages, 8 figures

Planning time-optimal trajectories for quadrotors in cluttered environments is a challenging, non-convex problem. This paper addresses minimizing the traversal time of a given collision-free geometric path without violating bounds on individual motor thrusts of the vehicle. Previous approaches have either relied on convex relaxations that do not guarantee dynamic feasibility, or have generated overly conservative time parametrizations. We propose TOPPQuad, a time-optimal path parameterization algorithm for quadrotors which explicitly incorporates quadrotor rigid body dynamics and constraints such as bounds on inputs (including motor speeds) and state of the vehicle (including the pose, linear and angular velocity and acceleration). We demonstrate the ability of the planner to generate faster trajectories that respect hardware constraints of the robot compared to several planners with relaxed notions of dynamic feasibility. We also demonstrate how TOPPQuad can be used to plan trajectories for quadrotors that utilize bidirectional motors. Overall, the proposed approach paves a way towards maximizing the efficacy of autonomous micro aerial vehicles while ensuring their safety.

With the increase in data availability, it has been widely demonstrated that neural networks (NN) can capture complex system dynamics precisely in a data-driven manner. However, the architectural complexity and nonlinearity of the NNs make it challenging to synthesize a provably safe controller. In this work, we propose a novel safety filter that relies on convex optimization to ensure safety for a NN system, subject to additive disturbances that are capable of capturing modeling errors. Our approach leverages tools from NN verification to over-approximate NN dynamics with a set of linear bounds, followed by an application of robust linear MPC to search for controllers that can guarantee robust constraint satisfaction. We demonstrate the efficacy of the proposed framework numerically on a nonlinear pendulum system., Comment: This paper has been accepted to the 2023 62nd IEEE Conference on Decision and Control (CDC)

The recent increase in data availability and reliability has led to a surge in the development of learning-based model predictive control (MPC) frameworks for robot systems. Despite attaining substantial performance improvements over their non-learning counterparts, many of these frameworks rely on an offline learning procedure to synthesize a dynamics model. This implies that uncertainties encountered by the robot during deployment are not accounted for in the learning process. On the other hand, learning-based MPC methods that learn dynamics models online are computationally expensive and often require a significant amount of data. To alleviate these shortcomings, we propose a novel learning-enhanced MPC framework that incorporates components from $\mathcal{L}_1$ adaptive control into learning-based MPC. This integration enables the accurate compensation of both matched and unmatched uncertainties in a sample-efficient way, enhancing the control performance during deployment. In our proposed framework, we present two variants and apply them to the control of a quadrotor system. Through simulations and physical experiments, we demonstrate that the proposed framework not only allows the synthesis of an accurate dynamics model on-the-fly, but also significantly improves the closed-loop control performance under a wide range of spatio-temporal uncertainties., Comment: 7 pages, 7 figures. Accepted for publication in the proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023)

Event-based sensors have recently drawn increasing interest in robotic perception due to their lower latency, higher dynamic range, and lower bandwidth requirements compared to standard CMOS-based imagers. These properties make them ideal tools for real-time perception tasks in highly dynamic environments. In this work, we demonstrate an application where event cameras excel: accurately estimating the impact location of fast-moving objects. We introduce a lightweight event representation called Binary Event History Image (BEHI) to encode event data at low latency, as well as a learning-based approach that allows real-time inference of a confidence-enabled control signal to the robot. To validate our approach, we present an experimental catching system in which we catch fast-flying ping-pong balls. We show that the system is capable of achieving a success rate of 81% in catching balls targeted at different locations, with a velocity of up to 13 m/s even on compute-constrained embedded platforms such as the Nvidia Jetson NX., Comment: 8 pages, 6 figures, IEEE Robotics and Automation Letters ( Volume: 7, Issue: 4, October 2022)

Persistent monitoring of a spatiotemporal fluid process requires data sampling and predictive modeling of the process being monitored. In this paper we present PASST algorithm: Predictive-model based Adaptive Sampling of a Spatio-Temporal process. PASST is an adaptive robotic sampling algorithm that leverages predictive models to efficiently and persistently monitor a fluid process in a given region of interest. Our algorithm makes use of the predictions from a learned prediction model to plan a path for an autonomous vehicle to adaptively and efficiently survey the region of interest. In turn, the sampled data is used to obtain better predictions by giving an updated initial state to the predictive model. For predictive model, we use Knowledged-based Neural Ordinary Differential Equations to train models of fluid processes. These models are orders of magnitude smaller in size and run much faster than fluid data obtained from direct numerical simulations of the partial differential equations that describe the fluid processes or other comparable computational fluids models. For path planning, we use reinforcement learning based planning algorithms that use the field predictions as reward functions. We evaluate our adaptive sampling path planning algorithm on both numerically simulated fluid data and real-world nowcast ocean flow data to show that we can sample the spatiotemporal field in the given region of interest for long time horizons. We also evaluate PASST algorithm's generalization ability to sample from fluid processes that are not in the training repertoire of the learned models.

We seek methods to model, control, and analyze robot teams performing environmental monitoring tasks. During environmental monitoring, the goal is to have teams of robots collect various data throughout a fixed region for extended periods of time. Standard bottom-up task assignment methods do not scale as the number of robots and task locations increases and require computationally expensive replanning. Alternatively, top-down methods have been used to combat computational complexity, but most have been limited to the analysis of methods which focus on transition times between tasks. In this work, we study a class of nonlinear macroscopic models which we use to control a time-varying distribution of robots performing different tasks throughout an environment. Our proposed ensemble model and control maintains desired time-varying populations of robots by leveraging naturally occurring interactions between robots performing tasks. We validate our approach at multiple fidelity levels including experimental results, suggesting the effectiveness of our approach to perform environmental monitoring.

Any strategy used to distribute a robot ensemble over a set of sequential tasks is subject to inaccuracy due to robot-level uncertainties and environmental influences on the robots' behavior. We approach the problem of inaccuracy during task allocation by modeling and controlling the overall ensemble behavior. Our model represents the allocation problem as a stochastic jump process and we regulate the mean and variance of such a process. The main contributions of this paper are: Establishing a structure for the transition rates of the equivalent stochastic jump process and formally showing that this approach leads to decoupled parameters that allow us to adjust the first- and second-order moments of the ensemble distribution over tasks, which gives the flexibility to decrease the variance in the desired final distribution. This allows us to directly shape the impact of uncertainties on the group allocation over tasks. We introduce a detailed procedure to design the gains to achieve the desired mean and show how the additional parameters impact the covariance matrix, which is directly associated with the degree of task allocation precision. Our simulation and experimental results illustrate the successful control of several robot ensembles during task allocation.

This paper focuses on the broadcast of information on robot networks with stochastic network interconnection topologies. Problematic communication networks are almost unavoidable in areas where we wish to deploy multi-robotic systems, usually due to a lack of environmental consistency, accessibility, and structure. We tackle this problem by modeling the broadcast of information in a multi-robot communication network as a stochastic process with random arrival times, which can be produced by irregular robot movements, wireless attenuation, and other environmental factors. Using this model, we provide and analyze a receding horizon control strategy to control the statistics of the information broadcast. The resulting strategy compels the robots to re-direct their communication resources to different neighbors according to the current propagation process to fulfill global broadcast requirements. Based on this method, we provide an approach to compute the expected time to broadcast the message to all nodes. Numerical examples are provided to illustrate the results.

Transfer operators offer linear representations and global, physically meaningful features of nonlinear dynamical systems. Discovering transfer operators, such as the Koopman operator, require careful crafted dictionaries of observables, acting on states of the dynamical system. This is ad hoc and requires the full dataset for evaluation. In this paper, we offer an optimization scheme to allow joint learning of the observables and Koopman operator with online data. Our results show we are able to reconstruct the evolution and represent the global features of complex dynamical systems., Comment: Accepted to 5th L4DC Conference. Proceedings of The 5th Annual Learning for Dynamics and Control Conference, PMLR 211:1271-1283, 2023. 13 pages 6 figures

Nonlinear model predictive control (MPC) is a flexible and increasingly popular framework used to synthesize feedback control strategies that can satisfy both state and control input constraints. In this framework, an optimization problem, subjected to a set of dynamics constraints characterized by a nonlinear dynamics model, is solved at each time step. Despite its versatility, the performance of nonlinear MPC often depends on the accuracy of the dynamics model. In this work, we leverage deep learning tools, namely knowledge-based neural ordinary differential equations (KNODE) and deep ensembles, to improve the prediction accuracy of this model. In particular, we learn an ensemble of KNODE models, which we refer to as the KNODE ensemble, to obtain an accurate prediction of the true system dynamics. This learned model is then integrated into a novel learning-enhanced nonlinear MPC framework. We provide sufficient conditions that guarantees asymptotic stability of the closed-loop system and show that these conditions can be implemented in practice. We show that the KNODE ensemble provides more accurate predictions and illustrate the efficacy and closed-loop performance of the proposed nonlinear MPC framework using two case studies., Comment: 16 pages, 4 figures, includes Appendix. Accepted at Learning for Dynamics and Control Conference (L4DC) 2023

State estimation is an important aspect in many robotics applications. In this work, we consider the task of obtaining accurate state estimates for robotic systems by enhancing the dynamics model used in state estimation algorithms. Existing frameworks such as moving horizon estimation (MHE) and the unscented Kalman filter (UKF) provide the flexibility to incorporate nonlinear dynamics and measurement models. However, this implies that the dynamics model within these algorithms has to be sufficiently accurate in order to warrant the accuracy of the state estimates. To enhance the dynamics models and improve the estimation accuracy, we utilize a deep learning framework known as knowledge-based neural ordinary differential equations (KNODEs). The KNODE framework embeds prior knowledge into the training procedure and synthesizes an accurate hybrid model by fusing a prior first-principles model with a neural ordinary differential equation (NODE) model. In our proposed LEARNEST framework, we integrate the data-driven model into two novel model-based state estimation algorithms, which are denoted as KNODE-MHE and KNODE-UKF. These two algorithms are compared against their conventional counterparts across a number of robotic applications; state estimation for a cartpole system using partial measurements, localization for a ground robot, as well as state estimation for a quadrotor. Through simulations and tests using real-world experimental data, we demonstrate the versatility and efficacy of the proposed learning-enhanced state estimation framework., Comment: 7 pages, 3 figures, 1 table. Accepted at 2023 IEEE International Conference on Robotics and Automation (ICRA)

In this paper, we present an online adaptive planning strategy for a team of robots with heterogeneous sensors to sample from a latent spatial field using a learned model for decision making. Current robotic sampling methods seek to gather information about an observable spatial field. However, many applications, such as environmental monitoring and precision agriculture, involve phenomena that are not directly observable or are costly to measure, called latent phenomena. In our approach, we seek to reason about the latent phenomenon in real-time by effectively sampling the observable spatial fields using a team of robots with heterogeneous sensors, where each robot has a distinct sensor to measure a different observable field. The information gain is estimated using a learned model that maps from the observable spatial fields to the latent phenomenon. This model captures aleatoric uncertainty in the relationship to allow for information theoretic measures. Additionally, we explicitly consider the correlations among the observable spatial fields, capturing the relationship between sensor types whose observations are not independent. We show it is possible to learn these correlations, and investigate the impact of the learned correlation models on the performance of our sampling approach. Through our qualitative and quantitative results, we illustrate that empirically learned correlations improve the overall sampling efficiency of the team. We simulate our approach using a data set of sensor measurements collected on Lac Hertel, in Quebec, which we make publicly available., Comment: (c) 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

In this work, we consider the task of improving the accuracy of dynamic models for model predictive control (MPC) in an online setting. Although prediction models can be learned and applied to model-based controllers, these models are often learned offline. In this offline setting, training data is first collected and a prediction model is learned through an elaborated training procedure. However, since the model is learned offline, it does not adapt to disturbances or model errors observed during deployment. To improve the adaptiveness of the model and the controller, we propose an online dynamics learning framework that continually improves the accuracy of the dynamic model during deployment. We adopt knowledge-based neural ordinary differential equations (KNODE) as the dynamic models, and use techniques inspired by transfer learning to continually improve the model accuracy. We demonstrate the efficacy of our framework with a quadrotor, and verify the framework in both simulations and physical experiments. Results show that our approach can account for disturbances that are possibly time-varying, while maintaining good trajectory tracking performance., Comment: Accepted into Conference on Robot Learning (CoRL) 2022, 8 pages, 4 figures

The multi-robot adaptive sampling problem aims at finding trajectories for a team of robots to efficiently sample the phenomenon of interest within a given endurance budget of the robots. In this paper, we propose a robust and scalable approach using Multi-Agent Reinforcement Learning for cooperated Adaptive Sampling (MARLAS) of quasi-static environmental processes. Given a prior on the field being sampled, the proposed method learns decentralized policies for a team of robots to sample high-utility regions within a fixed budget. The multi-robot adaptive sampling problem requires the robots to coordinate with each other to avoid overlapping sampling trajectories. Therefore, we encode the estimates of neighbor positions and intermittent communication between robots into the learning process. We evaluated MARLAS over multiple performance metrics and found it to outperform other baseline multi-robot sampling techniques. Additionally, we demonstrate scalability with both the size of the robot team and the size of the region being sampled. We further demonstrate robustness to communication failures and robot failures. The experimental evaluations are conducted both in simulations on real data and in real robot experiments on demo environmental setup., Comment: 15 pages, 9 figures, accepted to DARS 2022

We present a flow-based control strategy that enables resource-constrained marine robots to patrol gyre-like flow environments on an orbital trajectory with a periodicity in a given range. The controller does not require a detailed model of the flow field and relies only on the robot's location relative to the center of the gyre. Instead of precisely tracking a pre-defined trajectory, the robots are tasked to stay in between two bounding trajectories with known periodicity. Furthermore, the proposed strategy leverages the surrounding flow field to minimize control effort. We prove that the proposed strategy enables robots to cycle in the flow satisfying the desired periodicity requirements. Our method is tested and validated both in simulation and in experiments using a low-cost, underactuated, surface swimming robot, i.e. the Modboat., Comment: 7 pages. Published at 2022 International Conference on Robotics and Automation (ICRA)

This paper studies how global dynamics and knowledge of high-level features can inform decision-making for robots in flow-like environments. Specifically, we investigate how coherent sets, an environmental feature found in these environments, inform robot awareness within these scenarios. The proposed approach is an online environmental feature generator which can be used for robot reasoning. We compute coherent sets online with techniques from machine learning and design frameworks for robot behavior that leverage coherent set features. We demonstrate the effectiveness of online methods over offline methods. Notably, we apply these online methods for robot monitoring of pedestrian behaviors and robot navigation through water. Environmental features such as coherent sets provide rich context to robots for smarter, more efficient behavior., Comment: 8 pages, 8 figures. To be published in IEEE Robotics and Automation Letters (RA-L)

Understanding decentralized dynamics from collective behaviors in swarms is crucial for informing robot controller designs in artificial swarms and multiagent robotic systems. However, the complexity in agent-to-agent interactions and the decentralized nature of most swarms pose a significant challenge to the extraction of single-robot control laws from global behavior. In this work, we consider the important task of learning decentralized single-robot controllers based solely on the state observations of a swarm's trajectory. We present a general framework by adopting knowledge-based neural ordinary differential equations (KNODE) -- a hybrid machine learning method capable of combining artificial neural networks with known agent dynamics. Our approach distinguishes itself from most prior works in that we do not require action data for learning. We apply our framework to two different flocking swarms in 2D and 3D respectively, and demonstrate efficient training by leveraging the graphical structure of the swarms' information network. We further show that the learnt single-robot controllers can not only reproduce flocking behavior in the original swarm but also scale to swarms with more robots., Comment: This work has been submitted to the IEEE for possible publication. 8 pages, 11 figures

In this work, we consider the problem of deriving and incorporating accurate dynamic models for model predictive control (MPC) with an application to quadrotor control. MPC relies on precise dynamic models to achieve the desired closed-loop performance. However, the presence of uncertainties in complex systems and the environments they operate in poses a challenge in obtaining sufficiently accurate representations of the system dynamics. In this work, we make use of a deep learning tool, knowledge-based neural ordinary differential equations (KNODE), to augment a model obtained from first principles. The resulting hybrid model encompasses both a nominal first-principle model and a neural network learnt from simulated or real-world experimental data. Using a quadrotor, we benchmark our hybrid model against a state-of-the-art Gaussian Process (GP) model and show that the hybrid model provides more accurate predictions of the quadrotor dynamics and is able to generalize beyond the training data. To improve closed-loop performance, the hybrid model is integrated into a novel MPC framework, known as KNODE-MPC. Results show that the integrated framework achieves 60.2% improvement in simulations and more than 21% in physical experiments, in terms of trajectory tracking performance., Comment: 8pg. \c{opyright} 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Deformable image registration (DIR), aiming to find spatial correspondence between images, is one of the most critical problems in the domain of medical image analysis. In this paper, we present a novel, generic, and accurate diffeomorphic image registration framework that utilizes neural ordinary differential equations (NODEs). We model each voxel as a moving particle and consider the set of all voxels in a 3D image as a high-dimensional dynamical system whose trajectory determines the targeted deformation field. Our method leverages deep neural networks for their expressive power in modeling dynamical systems, and simultaneously optimizes for a dynamical system between the image pairs and the corresponding transformation. Our formulation allows various constraints to be imposed along the transformation to maintain desired regularities. Our experiment results show that our method outperforms the benchmarks under various metrics. Additionally, we demonstrate the feasibility to expand our framework to register multiple image sets using a unified form of transformation,which could possibly serve a wider range of applications., Comment: Accepted by the IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) 2022

Autonomous robotic surgery has the potential to provide efficacy, safety, and consistency independent of individual surgeons skill and experience. Autonomous soft-tissue surgery in unstructured and deformable environments is especially challenging as it necessitates intricate imaging, tissue tracking and surgical planning techniques, as well as a precise execution via highly adaptable control strategies. In the laparoscopic setting, soft-tissue surgery is even more challenging due to the need for high maneuverability and repeatability under motion and vision constraints. We demonstrate the first robotic laparoscopic soft tissue surgery with a level of autonomy of 3 out of 5, which allows the operator to select among autonomously generated surgical plans while the robot executes a wide range of tasks independently. We also demonstrate the first in vivo autonomous robotic laparoscopic surgery via intestinal anastomosis on porcine models. We compared the criteria including needle placement corrections, suture spacing, suture bite size, completion time, lumen patency, and leak pressure between the developed system, manual laparoscopic surgery, and robot-assisted surgery (RAS). The ex vivo results indicate that our system outperforms expert surgeons and RAS techniques in terms of consistency and accuracy, and it leads to a remarkable anastomosis quality in living pigs. These results demonstrate that surgical robots exhibiting high levels of autonomy have the potential to improve consistency, patient outcomes, and access to a standard surgical technique.

This paper presents a distributed scalable multi-robot planning algorithm for informed sampling of quasistatic spatial fields. We address the problem of efficient data collection using multiple autonomous vehicles and consider the effects of communication between multiple robots, acting independently, on the overall sampling performance of the team. We focus on the distributed sampling problem where the robots operate independent of their teammates, but have the ability to communicate their current state to other neighbors within a fixed communication range. Our proposed approach is scalable and adaptive to various environmental scenarios, changing robot team configurations, and runs in real-time, which are important features for many real-world applications. We compare the performance of our proposed algorithm to baseline strategies through simulated experiments that utilize models derived from both synthetic and field deployment data. The results show that our sampling algorithm is efficient even when robots in the team are operating with a limited communication range, thus demonstrating the scalability of our method in sampling large-scale environments., Comment: Accepted for publication on Autonomous Robots (Journal), Spl. Issue on Robot Swarms in the Real World: from Design to Deployment

This paper presents a framework to enable a team of heterogeneous mobile robots to model and sense a multiscale system. We propose a coupled strategy, where robots of one type collect high-fidelity measurements at a slow time scale and robots of another type collect low-fidelity measurements at a fast time scale, for the purpose of fusing measurements together. The multiscale measurements are fused to create a model of a complex, nonlinear spatiotemporal process. The model helps determine optimal sensing locations and predict the evolution of the process. Key contributions are: i) consolidation of multiple types of data into one cohesive model, ii) fast determination of optimal sensing locations for mobile robots, and iii) adaptation of models online for various monitoring scenarios. We illustrate the proposed framework by modeling and predicting the evolution of an artificial plasma cloud. We test our approach using physical marine robots adaptively sampling a process in a water tank., Comment: 22 pages, 13 figures, submitted to Autonomous Robots

Swarming patterns that emerge from the interaction of many mobile agents are a subject of great interest in fields ranging from biology to physics and robotics. In some application areas, multiple swarms effectively interact and collide, producing complex spatiotemporal patterns. Recent studies have begun to address swarm-on-swarm dynamics, and in particular the scattering of two large, colliding swarms with nonlinear interactions. To build on early numerical insights, we develop a mean-field approach that can be used to predict the parameters under which colliding swarms are expected to form a milling state. Our analytical method relies on the assumption that, upon collision, two swarms oscillate near a limit-cycle, where each swarm rotates around the other while maintaining an approximately constant and uniform density. Using this approach we are able to predict the critical swarm-on-swarm interaction coupling, below which two colliding swarms merely scatter, for near head-on collisions as a function of control parameters. We show that the critical coupling corresponds to a saddle-node bifurcation of a stable limit cycle in the uniform, constant density approximation. Our results are tested and found to agree with large multi-agent simulations.

Loading quantum information deterministically onto a quantum node is an important step towards a quantum network. Here, we demonstrate that coherent-state microwave photons, with an optimal temporal waveform, can be efficiently loaded onto a single superconducting artificial atom in a semi-infinite one-dimensional (1D) transmission-line waveguide. Using a weak coherent state (average photon number N<<1 with an exponentially rising waveform, whose time constant matches the decoherence time of the artificial atom, we demonstrate a loading efficiency of above 94% from 1D semi-free space to the artificial atom. We also show that Fock-state microwave photons can be deterministically loaded with an efficiency of 98.5%. We further manipulate the phase of the coherent state exciting the atom, enabling coherent control of the loading process. Our results open up promising applications in realizing quantum networks based on waveguide quantum electrodynamics (QED).

Extracting predictive models from nonlinear systems is a central task in scientific machine learning. One key problem is the reconciliation between modern data-driven approaches and first principles. Despite rapid advances in machine learning techniques, embedding domain knowledge into data-driven models remains a challenge. In this work, we present a universal learning framework for extracting predictive models from nonlinear systems based on observations. Our framework can readily incorporate first principle knowledge because it naturally models nonlinear systems as continuous-time systems. This both improves the extracted models' extrapolation power and reduces the amount of data needed for training. In addition, our framework has the advantages of robustness to observational noise and applicability to irregularly sampled data. We demonstrate the effectiveness of our scheme by learning predictive models for a wide variety of systems including a stiff Van der Pol oscillator, the Lorenz system, and the Kuramoto-Sivashinsky equation. For the Lorenz system, different types of domain knowledge are incorporated to demonstrate the strength of knowledge embedding in data-driven system identification., Comment: 8 pages, 12 figures in main text, 6 figures and 8 tables in supplement

We present a data-driven modeling strategy to overcome improperly modeled dynamics for systems exhibiting complex spatio-temporal behaviors. We propose a Deep Learning framework to resolve the differences between the true dynamics of the system and the dynamics given by a model of the system that is either inaccurately or inadequately described. Our machine learning strategy leverages data generated from the improper system model and observational data from the actual system to create a neural network to model the dynamics of the actual system. We evaluate the proposed framework using numerical solutions obtained from three increasingly complex dynamical systems. Our results show that our system is capable of learning a data-driven model that provides accurate estimates of the system states both in previously unobserved regions as well as for future states. Our results show the power of state-of-the-art machine learning frameworks in estimating an accurate prior of the system's true dynamics that can be used for prediction up to a finite horizon.

Swarms of coupled mobile agents subject to inter-agent wireless communication delays are known to exhibit multiple dynamic patterns in space that depend on the strength of the interactions and the magnitude of the communication delays. We experimentally demonstrate communication delay-induced bifurcations in the spatio-temporal patterns of robot swarms using two distinct hardware platforms in a mixed reality framework. Additionally, we make steps toward experimentally validating theoretically predicted parameter regions where transitions between swarm patterns occur. We show that multiple rotation patterns persist even when collision-avoidance strategies are incorporated, and we show the existence of multi-stable, co-existing rotational patterns not predicted by usual mean field dynamics. Our experiments are the first significant steps towards validating existing theory and the existence and robustness of the delay-induced patterns in real robotic swarms., Comment: 13 pages, 12 figures

Dynamical emergent patterns of swarms are now fairly well established in nature, and include flocking and rotational states. Recently, there has been great interest in engineering and physics to create artificial self-propelled agents that communicate over a network and operate with simple rules, with the goal of creating emergent self-organizing swarm patterns. In this paper, we show that when communicating networks have range dependent delays, rotational states which are typically periodic, undergo a bifurcation and create swarm dynamics on a torus. The observed bifurcation yields additional frequencies into the dynamics, which may lead to quasi-periodic behavior of the swarm., Comment: 7 pages 8 figures