Search

Despite significant research, robotic swarms have yet to be useful in solving real-world problems, largely due to the difficulty of creating and controlling swarming behaviors in multi-agent systems. Traditional top-down approaches in which a desired emergent behavior is produced often require complex, resource-heavy robots, limiting their practicality. This paper introduces a bottom-up approach by employing an Embodied Agent-Based Modeling and Simulation approach, emphasizing the use of simple robots and identifying conditions that naturally lead to self-organized collective behaviors. Using the Reality-to-Simulation-to-Reality for Swarms (RSRS) process, we tightly integrate real-world experiments with simulations to reproduce known swarm behaviors as well as discovering a novel emergent behavior without aiming to eliminate or even reduce the sim2real gap. This paper presents the development of an Agent-Based Embodiment and Emulation process that balances the importance of running physical swarming experiments and the prohibitively time-consuming process of even setting up and running a single experiment with 20+ robots by leveraging low-fidelity lightweight simulations to enable hypothesis-formation to guide physical experiments. We demonstrate the usefulness of our methods by emulating two known behaviors from the literature and show a third behavior `discovered' by accident., Comment: 8 pages, 6 figures, submitted to ICRA 2025

Drones which can swarm and loiter in a certain area cost hundreds of dollars, but mosquitos can do the same and are essentially worthless. To control swarms of low-cost robots, researchers may end up spending countless hours brainstorming robot configurations and policies to ``organically" create behaviors which do not need expensive sensors and perception. Existing research explores the possible emergent behaviors in swarms of robots with only a binary sensor and a simple but hand-picked controller structure. Even agents in this highly limited sensing, actuation, and computational capability class can exhibit relatively complex global behaviors such as aggregation, milling, and dispersal, but finding the local interaction rules that enable more collective behaviors remains a significant challenge. This paper investigates the feasibility of training spiking neural networks to find those local interaction rules that result in particular emergent behaviors. In this paper, we focus on simulating a specific milling behavior already known to be producible using very simple binary sensing and acting agents. To do this, we use evolutionary algorithms to evolve not only the parameters (the weights, biases, and delays) of a spiking neural network, but also its structure. To create a baseline, we also show an evolutionary search strategy over the parameters for the incumbent hand-picked binary controller structure. Our simulations show that spiking neural networks can be evolved in binary sensing agents to form a mill., Comment: 8 pages, 7 figures, presented at the 2024 International Conference on Neuromorphic Systems

With the oncologist acting as the ``game leader'', we employ a Stackelberg game-theoretic model involving multiple populations to study prostate cancer. We refine the drug dosing schedule using an empirical Bayes feed-forward analysis, based on clinical data that reflects each patient's prostate-specific drug response. Our methodology aims for a quantitative grasp of the parameter landscape of this adaptive multi-population model, focusing on arresting the growth of drug-resistant prostate cancer by promoting competition across drug-sensitive cancer cell populations. Our findings indicate that not only is it is feasible to considerably extend cancer suppression duration through careful optimization, but even transform metastatic prostate cancer into a chronic condition instead of an acute one for most patients, with supporting clinical and analytical evidence.

Recent breakthroughs in machine learning and artificial intelligence, fueled by scientific data, are revolutionizing the discovery of new materials. Despite the wealth of existing scientific literature, the availability of both structured experimental data and chemical domain knowledge that can be easily integrated into data-driven workflows is limited. The motivation to integrate this information, as well as additional context from first-principle calculations and physics-informed deep learning surrogate models, is to enable efficient exploration of the relevant chemical space and to predict structure-property relationships of new materials a priori. Ultimately, such a framework could replicate the expertise of human subject-matter experts. In this work, we present dZiner, a chemist AI agent, powered by large language models (LLMs), that discovers new compounds with desired properties via inverse design (property-to-structure). In specific, the agent leverages domain-specific insights from foundational scientific literature to propose new materials with enhanced chemical properties, iteratively evaluating them using relevant surrogate models in a rational design process, while accounting for design constraints. The model supports both closed-loop and human-in-the-loop feedback cycles, enabling human-AI collaboration in molecular design with real-time property inference, and uncertainty and chemical feasibility assessment. We demonstrate the flexibility of this agent by applying it to various materials target properties, including surfactants, ligand and drug candidates, and metal-organic frameworks. Our approach holds promise to both accelerate the discovery of new materials and enable the targeted design of materials with desired functionalities. The methodology is available as an open-source software on https://github.com/mehradans92/dZiner.

Nambu-determinant brackets on $R^d\ni x=(x^1,...,x^d)$, $\{f,g\}_d(x)=\rho(x) \det(\partial(f,g,a_1,...,a_{d-2})/\partial(x^1,...,x^d))$, with $a_i\in C^\infty(R^d)$ and $\rho\partial_x\in\mathfrak{X}^d(R^d)$, are a class of Poisson structures with (non)linear coefficients, e.g., polynomials of arbitrarily high degree. With good cocycles in the graph complex, Kontsevich associated universal -- for all Poisson bivectors $P$ on affine $R^d_{aff}$ -- elements $\dot{P}=Q^\gamma(P)\in H^2_{P}(R^d_{aff})$ in the Lichnerowicz-Poisson second cohomology groups; we note that known graph cocycles $\gamma$ preserve the Nambu-Poisson class $\{P(\rho,a)\}$, and we express, directly from $\gamma$, the evolution $\dot{\rho}$,$\dot{a}$ that induces $\dot{P}$. Over all $d\geq2$ at once, there is no universal mechanism for the bivector cocycles $Q^\gamma_d$ to be trivial, $Q^\gamma_d=[\![P,\vec{X}^\gamma_d(P)]\!]$, w.r.t. vector fields defined uniformly for all dimensions $d$ by the same graph formula. While over $R^2$, the graph flows $\dot{P} = Q^{\gamma_i}_{2D}(P(\rho))$ for $\gamma\in\{\gamma_3,\gamma_5,\gamma_7,...\}$ are trivialized by vector fields $\vec{X}^{\gamma_i}_{2D}=(dx\wedge dy)^{-1}d_{dR}(Ham^{\gamma_i}(P))$ of peculiar shape, we detect that in $d\geq3$, the 1-vectors from 2D, now with $P(\rho,a_1,...,a_{d-2})$ inside, do not solve the problems $Q^{\gamma_i}_{d\geq3}=[\![P,{\vec{X}^{\gamma_i}_{d\geq3}}(P(\rho,a))]\!]$, yet they do yield good Ansatz where we find solutions $\vec{X}^{\gamma_i}_{d=3,4}(P(\rho,a))$. In the study of the step $d\mapsto d+1$, by adapting the Kontsevich graph calculus to the Nambu-Poisson class of brackets, we discover more identities for the Jacobian determinants within $P(\rho,a)$, i.e. for multivector-valued $GL(d)$-invariants on $R^d_{aff}$., Comment: Plenary talk given by AVK at the international conference on Integrable Systems and Quantum Symmetries (ISQS28) held on 1-5 July 2024 at CVUT Prague, Czech Republic; 13 pages, 1 table; followed by papers II. (arXiv:2409.12555 [math.QA]) and III. (arXiv:2409.15932 [math.QA]) by the same authors

Kontsevich constructed a map between `good' graph cocycles $\gamma$ and infinitesimal deformations of Poisson bivectors on affine manifolds, that is, Poisson cocycles in the second Lichnerowicz--Poisson cohomology. For the tetrahedral graph cocycle $\gamma_3$ and for the class of Nambu-determinant Poisson bivectors $P$ over $\mathbb{R}^2$, $\mathbb{R}^3$ and $\mathbb{R}^4$, we know the fact of trivialization, $\dot{P}=[[ P, \vec{X}^{\gamma_3}_{\text{dim}}]]$, by using dimension-dependent vector fields $\vec{X}^{\gamma_3}_{\text{dim}}$ expressed by Kontsevich (micro-) graphs. We establish that these trivializing vector fields $\vec{X}^{\gamma_3}_{\text{dim}}$ are unique modulo Hamiltonian vector fields $\vec{X}_{H}=d_P(H)= [[ P, H]]$, where $d_P$ is the Lichnerowicz--Poisson differential and where the Hamiltonians $H$ are also represented by Kontsevich (micro-)graphs. However, we find that the choice of Kontsevich (micro-)graphs to represent the aforementioned multivectors is not unique.

Kontsevich constructed a map from suitable cocycles in the graph complex to infinitesimal deformations of Poisson bi-vector fields. Under the deformations, the bi-vector fields remain Poisson. We ask, are these deformations trivial, meaning, do they amount to a change of coordinates along a vector field? We examine this question for the tetrahedron, the smallest nontrivial suitable graph cocycle in the Kontsevich graph complex, and for the class of Nambu--Poisson brackets on $\mathbb{R}^d$. Within Kontsevich's graph calculus, we use dimension-specific micro-graphs, in which each vertex represents an ingredient of the Nambu--Poisson bracket. For the tetrahedron, Kontsevich knew that the deformation is trivial for $d = 2$ (1996). In 2020, Buring and the third author found that the deformation is trivial for $d = 3$. Building on these discoveries, we now establish that the deformation is trivial for $d = 4$.

We study the problem of cross-embodiment inverse reinforcement learning, where we wish to learn a reward function from video demonstrations in one or more embodiments and then transfer the learned reward to a different embodiment (e.g., different action space, dynamics, size, shape, etc.). Learning reward functions that transfer across embodiments is important in settings such as teaching a robot a policy via human video demonstrations or teaching a robot to imitate a policy from another robot with a different embodiment. However, prior work has only focused on cases where near-optimal demonstrations are available, which is often difficult to ensure. By contrast, we study the setting of cross-embodiment reward learning from mixed-quality demonstrations. We demonstrate that prior work struggles to learn generalizable reward representations when learning from mixed-quality data. We then analyze several techniques that leverage human feedback for representation learning and alignment to enable effective cross-embodiment learning. Our results give insight into how different representation learning techniques lead to qualitatively different reward shaping behaviors and the importance of human feedback when learning from mixed-quality, mixed-embodiment data., Comment: First Two Authors Share Equal Contribution. 19 Pages, 4 Figures

A popular method for enhancing images involves learning the style of a professional photo editor using pairs of training images comprised of the original input with the editor-enhanced version. When manipulating images, many editing tools offer a feature that allows the user to manipulate a limited selection of familiar colors. Editing by color name allows easy adjustment of elements like the "blue" of the sky or the "green" of trees. Inspired by this approach to color manipulation, we propose NamedCurves, a learning-based image enhancement technique that separates the image into a small set of named colors. Our method learns to globally adjust the image for each specific named color via tone curves and then combines the images using an attention-based fusion mechanism to mimic spatial editing. We demonstrate the effectiveness of our method against several competing methods on the well-known Adobe 5K dataset and the PPR10K dataset, showing notable improvements., Comment: European Conference on Computer Vision ECCV 2024

We use variations on Lax type operators to find explicit formulas for certain elements of finite $W$-algebras. These give a complete set of generators for all finite $W$-algebras of types B,C,D for which the Dynkin grading is even.

Automating robotic surgery via learning from demonstration (LfD) techniques is extremely challenging. This is because surgical tasks often involve sequential decision-making processes with complex interactions of physical objects and have low tolerance for mistakes. Prior works assume that all demonstrations are fully observable and optimal, which might not be practical in the real world. This paper introduces a sample-efficient method that learns a robust reward function from a limited amount of ranked suboptimal demonstrations consisting of partial-view point cloud observations. The method then learns a policy by optimizing the learned reward function using reinforcement learning (RL). We show that using a learned reward function to obtain a policy is more robust than pure imitation learning. We apply our approach on a physical surgical electrocautery task and demonstrate that our method can perform well even when the provided demonstrations are suboptimal and the observations are high-dimensional point clouds. Code and videos available here: https://sites.google.com/view/lfdinelectrocautery, Comment: In proceedings of the International Symposium on Medical Robotics (ISMR) 2024. Equal contribution from two first authors

Tendon-driven continuum robot kinematic models are frequently computationally expensive, inaccurate due to unmodeled effects, or both. In particular, unmodeled effects produce uncertainties that arise during the robot's operation that lead to variability in the resulting geometry. We propose a novel solution to these issues through the development of a Gaussian mixture kinematic model. We train a mixture density network to output a Gaussian mixture model representation of the robot geometry given the current tendon displacements. This model computes a probability distribution that is more representative of the true distribution of geometries at a given configuration than a model that outputs a single geometry, while also reducing the computation time. We demonstrate one use of this model through a trajectory optimization method that explicitly reasons about the workspace uncertainty to minimize the probability of collision.

It is crucial for robots to be aware of the presence of constraints in order to acquire safe policies. However, explicitly specifying all constraints in an environment can be a challenging task. State-of-the-art constraint inference algorithms learn constraints from demonstrations, but tend to be computationally expensive and prone to instability issues. In this paper, we propose a novel Bayesian method that infers constraints based on preferences over demonstrations. The main advantages of our proposed approach are that it 1) infers constraints without calculating a new policy at each iteration, 2) uses a simple and more realistic ranking of groups of demonstrations, without requiring pairwise comparisons over all demonstrations, and 3) adapts to cases where there are varying levels of constraint violation. Our empirical results demonstrate that our proposed Bayesian approach infers constraints of varying severity, more accurately than state-of-the-art constraint inference methods.

The Postsecondary Pathways for Former Foster Care Youth (PP-FFCY) study is a research project that focuses on the transitions of foster care alums to and through postsecondary institutions. Through the Ofice of Community College Research and Leadership (OCCRL), the PP-FFCY project team is conducting a comprehensive needs assessment report that: (1) provides a descriptive profile of Illinois foster care alums; (2) examines postsecondary access, opportunities, and supports for Illinois foster youth; and (3) addresses disparities in career and technical education (CTE) programs. Although the PP-FFCY project focuses on Illinois foster youth, it is beneficial to examine programs and resources for foster care alums in other states. Consequently, in this research brief, the authors highlight California postsecondary programs and services that support youth in care, in addition to young adults aging out of foster care. Outlined in this report is a comprehensive review of the general and foster youth demographic profiles in California, followed by an overview of various forms of student aid (e.g., national-, state-, and campuslevel support) available for individuals with foster care backgrounds, as well as a synopsis of foster youth CTE program enrollment.

We study the problem of determining the emergent behaviors that are possible given a functionally heterogeneous swarm of robots with limited capabilities. Prior work has considered behavior search for homogeneous swarms and proposed the use of novelty search over either a hand-specified or learned behavior space followed by clustering to return a taxonomy of emergent behaviors to the user. In this paper, we seek to better understand the role of novelty search and the efficacy of using clustering to discover novel emergent behaviors. Through a large set of experiments and ablations, we analyze the effect of representations, evolutionary search, and various clustering methods in the search for novel behaviors in a heterogeneous swarm. Our results indicate that prior methods fail to discover many interesting behaviors and that an iterative human-in-the-loop discovery process discovers more behaviors than random search, swarm chemistry, and automated behavior discovery. The combined discoveries of our experiments uncover 23 emergent behaviors, 18 of which are novel discoveries. To the best of our knowledge, these are the first known emergent behaviors for heterogeneous swarms of computation-free agents. Videos, code, and appendix are available at the project website: https://sites.google.com/view/heterogeneous-bd-methods, Comment: 11 pages, 9 figures, To be published in Proceedings IEEE International Symposium on Multi-Robot & Multi-Agent Systems (MRS 2023)

Our ultimate goal is to build robust policies for robots that assist people. What makes this hard is that people can behave unexpectedly at test time, potentially interacting with the robot outside its training distribution and leading to failures. Even just measuring robustness is a challenge. Adversarial perturbations are the default, but they can paint the wrong picture: they can correspond to human motions that are unlikely to occur during natural interactions with people. A robot policy might fail under small adversarial perturbations but work under large natural perturbations. We propose that capturing robustness in these interactive settings requires constructing and analyzing the entire natural-adversarial frontier: the Pareto-frontier of human policies that are the best trade-offs between naturalness and low robot performance. We introduce RIGID, a method for constructing this frontier by training adversarial human policies that trade off between minimizing robot reward and acting human-like (as measured by a discriminator). On an Assistive Gym task, we use RIGID to analyze the performance of standard collaborative Reinforcement Learning, as well as the performance of existing methods meant to increase robustness. We also compare the frontier RIGID identifies with the failures identified in expert adversarial interaction, and with naturally-occurring failures during user interaction. Overall, we find evidence that RIGID can provide a meaningful measure of robustness predictive of deployment performance, and uncover failure cases in human-robot interaction that are difficult to find manually. https://ood-human.github.io.

Emergence and emergent behaviors are often defined as cases where changes in local interactions between agents at a lower level effectively changes what occurs in the higher level of the system (i.e., the whole swarm) and its properties. However, the manner in which these collective emergent behaviors self-organize is less understood. The focus of this paper is in presenting a new framework for characterizing the conditions that lead to different macrostates and how to predict/analyze their macroscopic properties, allowing us to indirectly engineer the same behaviors from the bottom up by tuning their environmental conditions rather than local interaction rules. We then apply this framework to a simple system of binary sensing and acting agents as an example to see if a re-framing of this swarms problem can help us push the state of the art forward. By first creating some working definitions of macrostates in a particular swarm system, we show how agent-based modeling may be combined with control theory to enable a generalized understanding of controllable emergent processes without needing to simulate everything. Whereas phase diagrams can generally only be created through Monte Carlo simulations or sweeping through ranges of parameters in a simulator, we develop closed-form functions that can immediately produce them revealing an infinite set of swarm parameter combinations that can lead to a specifically chosen self-organized behavior. While the exact methods are still under development, we believe simply laying out a potential path towards solutions that have evaded our traditional methods using a novel method is worth considering. Our results are characterized through both simulations and real experiments on ground robots., Comment: 8 pages, 13 figures, submitted to IROS 2024 conference

Autoexposure (AE) is a critical step applied by camera systems to ensure properly exposed images. While current AE algorithms are effective in well-lit environments with constant illumination, these algorithms still struggle in environments with bright light sources or scenes with abrupt changes in lighting. A significant hurdle in developing new AE algorithms for challenging environments, especially those with time-varying lighting, is the lack of suitable image datasets. To address this issue, we have captured a new 4D exposure dataset that provides a large solution space (i.e., shutter speed range from (1/500 to 15 seconds) over a temporal sequence with moving objects, bright lights, and varying lighting. In addition, we have designed a software platform to allow AE algorithms to be used in a plug-and-play manner with the dataset. Our dataset and associate platform enable repeatable evaluation of different AE algorithms and provide a much-needed starting point to develop better AE methods. We examine several existing AE strategies using our dataset and show that most users prefer a simple saliency method for challenging lighting conditions., Comment: ICCV 2023

Deep neural networks often fail catastrophically by relying on spurious correlations. Most prior work assumes a clear dichotomy into spurious and reliable features; however, this is often unrealistic. For example, most of the time we do not want an autonomous car to simply copy the speed of surrounding cars -- we don't want our car to run a red light if a neighboring car does so. However, we cannot simply enforce invariance to next-lane speed, since it could provide valuable information about an unobservable pedestrian at a crosswalk. Thus, universally ignoring features that are sometimes (but not always) reliable can lead to non-robust performance. We formalize a new setting called contextual reliability which accounts for the fact that the "right" features to use may vary depending on the context. We propose and analyze a two-stage framework called Explicit Non-spurious feature Prediction (ENP) which first identifies the relevant features to use for a given context, then trains a model to rely exclusively on these features. Our work theoretically and empirically demonstrates the advantages of ENP over existing methods and provides new benchmarks for contextual reliability., Comment: ICML 2023 Camera Ready Version

Reinforcement Learning from Human Feedback (RLHF) has emerged as a popular paradigm for capturing human intent to alleviate the challenges of hand-crafting the reward values. Despite the increasing interest in RLHF, most works learn black box reward functions that while expressive are difficult to interpret and often require running the whole costly process of RL before we can even decipher if these frameworks are actually aligned with human preferences. We propose and evaluate a novel approach for learning expressive and interpretable reward functions from preferences using Differentiable Decision Trees (DDTs). Our experiments across several domains, including CartPole, Visual Gridworld environments and Atari games, provide evidence that the tree structure of our learned reward function is useful in determining the extent to which the reward function is aligned with human preferences. We also provide experimental evidence that not only shows that reward DDTs can often achieve competitive RL performance when compared with larger capacity deep neural network reward functions but also demonstrates the diagnostic utility of our framework in checking alignment of learned reward functions. We also observe that the choice between soft and hard (argmax) output of reward DDT reveals a tension between wanting highly shaped rewards to ensure good RL performance, while also wanting simpler, more interpretable rewards. Videos and code, are available at: https://sites.google.com/view/ddt-rlhf

3D lookup tables (3D LUTs) are a key component for image enhancement. Modern image signal processors (ISPs) have dedicated support for these as part of the camera rendering pipeline. Cameras typically provide multiple options for picture styles, where each style is usually obtained by applying a unique handcrafted 3D LUT. Current approaches for learning and applying 3D LUTs are notably fast, yet not so memory-efficient, as storing multiple 3D LUTs is required. For this reason and other implementation limitations, their use on mobile devices is less popular. In this work, we propose a Neural Implicit LUT (NILUT), an implicitly defined continuous 3D color transformation parameterized by a neural network. We show that NILUTs are capable of accurately emulating real 3D LUTs. Moreover, a NILUT can be extended to incorporate multiple styles into a single network with the ability to blend styles implicitly. Our novel approach is memory-efficient, controllable and can complement previous methods, including learned ISPs. Code, models and dataset available at: https://github.com/mv-lab/nilut, Comment: AAAI 2024 - The 38th Annual AAAI Conference on Artificial Intelligence

Robot swarms often exhibit emergent behaviors that are fascinating to observe; however, it is often difficult to predict what swarm behaviors can emerge under a given set of agent capabilities. We seek to efficiently leverage human input to automatically discover a taxonomy of collective behaviors that can emerge from a particular multi-agent system, without requiring the human to know beforehand what behaviors are interesting or even possible. Our proposed approach adapts to user preferences by learning a similarity space over swarm collective behaviors using self-supervised learning and human-in-the-loop queries. We combine our learned similarity metric with novelty search and clustering to explore and categorize the space of possible swarm behaviors. We also propose several general-purpose heuristics that improve the efficiency of our novelty search by prioritizing robot controllers that are likely to lead to interesting emergent behaviors. We test our approach in simulation on two robot capability models and show that our methods consistently discover a richer set of emergent behaviors than prior work. Code, videos, and datasets are available at https://sites.google.com/view/evolving-novel-swarms., Comment: 13 pages, 10 figures, To be published in Proceedings Genetic and Evolutionary Computation Conference (GECCO 2023)

Cameras and image-editing software often process images in the wide-gamut ProPhoto color space, encompassing 90% of all visible colors. However, when images are encoded for sharing, this color-rich representation is transformed and clipped to fit within the small-gamut standard RGB (sRGB) color space, representing only 30% of visible colors. Recovering the lost color information is challenging due to the clipping procedure. Inspired by neural implicit representations for 2D images, we propose a method that optimizes a lightweight multi-layer-perceptron (MLP) model during the gamut reduction step to predict the clipped values. GamutMLP takes approximately 2 seconds to optimize and requires only 23 KB of storage. The small memory footprint allows our GamutMLP model to be saved as metadata in the sRGB image -- the model can be extracted when needed to restore wide-gamut color values. We demonstrate the effectiveness of our approach for color recovery and compare it with alternative strategies, including pre-trained DNN-based gamut expansion networks and other implicit neural representation methods. As part of this effort, we introduce a new color gamut dataset of 2200 wide-gamut/small-gamut images for training and testing. Our code and dataset can be found on the project website: https://gamut-mlp.github.io.