Search

Successfully addressing a wide variety of tasks is a core ability of autonomous agents, requiring flexibly adapting the underlying decision-making strategies and, as we argue in this work, also adapting the perception modules. An analogical argument would be the human visual system, which uses top-down signals to focus attention determined by the current task. Similarly, we adapt pre-trained large vision models conditioned on specific downstream tasks in the context of multi-task policy learning. We introduce task-conditioned adapters that do not require finetuning any pre-trained weights, combined with a single policy trained with behavior cloning and capable of addressing multiple tasks. We condition the visual adapters on task embeddings, which can be selected at inference if the task is known, or alternatively inferred from a set of example demonstrations. To this end, we propose a new optimization-based estimator. We evaluate the method on a wide variety of tasks from the CortexBench benchmark and show that, compared to existing work, it can be addressed with a single policy. In particular, we demonstrate that adapting visual features is a key design choice and that the method generalizes to unseen tasks given a few demonstrations.

First, two examples of 1D distributed port-Hamiltonian systems with dissipation, given in explicit (descriptor) form, are considered: the Dzekster model for the seepage of underground water and a nanorod model with non-local viscous damping. Implicit representations in Stokes-Lagrange subspaces are formulated. These formulations lead to modified Hamiltonian functions with spatial differential operators. The associated power balance equations are derived, together with the new boundary ports. Second, the port-Hamiltonian formulations for the Timoshenko and the Euler-Bernoulli beams are recalled, the latter being a flow-constrained version of the former. Implicit representations of these models in Stokes-Lagrange subspaces and corresponding power balance equations are derived. Bijective transformations are proposed between the explicit and implicit representations. It is proven these transformations commute with the flow-constraint projection operator., Comment: 14 pages, submitted to MTNS 2024

This paper presents a systematic methodology for the discretization and reduction of a class of one-dimensional Partial Differential Equations (PDEs) with inputs and outputs collocated at the spatial boundaries. The class of system that we consider is known as Boundary-Controlled Port-Hamiltonian Systems (BC-PHSs) and covers a wide class of Hyperbolic PDEs with a large type of boundary inputs and outputs. This is, for instance, the case of waves and beams with Neumann, Dirichlet, or mixed boundary conditions. Based on a Partitioned Finite Element Method (PFEM), we develop a numerical scheme for the structure-preserving spatial discretization for the class of one-dimensional BC-PHSs. We show that if the initial PDE is passive (or impedance energy preserving), the discretized model also is. In addition and since the discretized model or Full Order Model (FOM) can be of large dimension, we recall the standard Loewner framework for the Model Order Reduction (MOR) using frequency domain interpolation. We recall the main steps to produce a Reduced Order Model (ROM) that approaches the FOM in a given range of frequencies. We summarize the steps to follow in order to obtain a ROM that preserves the passive structure as well. Finally, we provide a constructive way to build a projector that allows to recover the physical meaning of the state variables from the ROM to the FOM. We use the one-dimensional wave equation and the Timoshenko beam as examples to show the versatility of the proposed approach.

A structure-preserving Finite Element Method (FEM) for the transport equation in one- and two-dimensional domains is presented. This Distributed Parameter System (DPS) has non-collocated boundary control and observation, and reveals a scattering-energy preserving structure. We show that the discretized model preserves the aforementioned structure from the original infinite-dimensional system. Moreover, we analyse the case of moving meshes for the one-dimensional case. The moving mesh requires less states than the fixed one to produce solutions with a comparable accuracy, and it can also reduce the overshoot and oscillations of Gibbs phenomenon produced when using the FEM. Numerical simulations are provided for the case of a one-dimensional transport equation with fixed and moving meshes.

Learning robot navigation strategies among pedestrian is crucial for domain based applications. Combining perception, planning and prediction allows us to model the interactions between robots and pedestrians, resulting in impressive outcomes especially with recent approaches based on deep reinforcement learning (RL). However, these works do not consider multi-robot scenarios. In this paper, we present MultiSoc, a new method for learning multi-agent socially aware navigation strategies using RL. Inspired by recent works on multi-agent deep RL, our method leverages graph-based representation of agent interactions, combining the positions and fields of view of entities (pedestrians and agents). Each agent uses a model based on two Graph Neural Network combined with attention mechanisms. First an edge-selector produces a sparse graph, then a crowd coordinator applies node attention to produce a graph representing the influence of each entity on the others. This is incorporated into a model-free RL framework to learn multi-agent policies. We evaluate our approach on simulation and provide a series of experiments in a set of various conditions (number of agents / pedestrians). Empirical results show that our method learns faster than social navigation deep RL mono-agent techniques, and enables efficient multi-agent implicit coordination in challenging crowd navigation with multiple heterogeneous humans. Furthermore, by incorporating customizable meta-parameters, we can adjust the neighborhood density to take into account in our navigation strategy.

Implicit representations such as Neural Radiance Fields (NeRF) have been shown to be very effective at novel view synthesis. However, these models typically require manual and careful human data collection for training. In this paper, we present AutoNeRF, a method to collect data required to train NeRFs using autonomous embodied agents. Our method allows an agent to explore an unseen environment efficiently and use the experience to build an implicit map representation autonomously. We compare the impact of different exploration strategies including handcrafted frontier-based exploration, end-to-end and modular approaches composed of trained high-level planners and classical low-level path followers. We train these models with different reward functions tailored to this problem and evaluate the quality of the learned representations on four different downstream tasks: classical viewpoint rendering, map reconstruction, planning, and pose refinement. Empirical results show that NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment, and can be used for several downstream robotic tasks, and that modular trained exploration models outperform other classical and end-to-end baselines. Finally, we show that AutoNeRF can reconstruct large-scale scenes, and is thus a useful tool to perform scene-specific adaptation as the produced 3D environment models can be loaded into a simulator to fine-tune a policy of interest.

In this paper we prove that a large class of linear evolution PDEs defines a Stokes-Dirac structure over Hilbert spaces. To do so, the theory of boundary control system is employed. This definition encompasses problems from mechanics, that cannot be handled by the seminal geometric setting given in [van der Schaft and Maschke, Hamiltonian formulation of distributed-parameter systems with boundary energy flow, 2002 ]. Many worked-out examples stemming from continuum mechanics and physics are presented in detail, and a particular focus is given on the functional spaces in duality at the boundary of the geometrical domain. For each example, the connection between the differential operators and the associated Hilbert complexes is illustrated.

In this work, we detail a procedure to construct a reduced order model on the basis of frequency-domain data, that preserves the non-strictly passive property and the port-Hamiltonian structure. The proposed scheme is based on Benner et al. (2020) contribution, which has been adapted (i) to handle non-strictly passive model, and (ii) to handle numerical issues observed when applying the Loewner framework on complex configurations. We validate the proposed scheme on a very complex two-dimensional wave equation, for which the discretized version preserves the port-Hamiltoninan form., Comment: Accepted at European Control Conference 2023

Understanding and mapping a new environment are core abilities of any autonomously navigating agent. While classical robotics usually estimates maps in a stand-alone manner with SLAM variants, which maintain a topological or metric representation, end-to-end learning of navigation keeps some form of memory in a neural network. Networks are typically imbued with inductive biases, which can range from vectorial representations to birds-eye metric tensors or topological structures. In this work, we propose to structure neural networks with two neural implicit representations, which are learned dynamically during each episode and map the content of the scene: (i) the Semantic Finder predicts the position of a previously seen queried object; (ii) the Occupancy and Exploration Implicit Representation encapsulates information about explored area and obstacles, and is queried with a novel global read mechanism which directly maps from function space to a usable embedding space. Both representations are leveraged by an agent trained with Reinforcement Learning (RL) and learned online during each episode. We evaluate the agent on Multi-Object Navigation and show the high impact of using neural implicit representations as a memory source.

The reinforcement learning (RL) research area is very active, with an important number of new contributions; especially considering the emergent field of deep RL (DRL). However a number of scientific and technical challenges still need to be resolved, amongst which we can mention the ability to abstract actions or the difficulty to explore the environment in sparse-reward settings which can be addressed by intrinsic motivation (IM). We propose to survey these research works through a new taxonomy based on information theory: we computationally revisit the notions of surprise, novelty and skill learning. This allows us to identify advantages and disadvantages of methods and exhibit current outlooks of research. Our analysis suggests that novelty and surprise can assist the building of a hierarchy of transferable skills that further abstracts the environment and makes the exploration process more robust.

The existence of certain Fq-spaces of differential forms of the projective line over a field K containing Fq leads us to prove an identity linking the determinant of the Moore matrix of n indeterminates with the determinant of the Moore matrix of the cofactors of its first row. These same spaces give an interpretation of Elkies pairing in terms of residues of differential forms. This pairing puts in duality the Fq-vector space of the roots of a Fq-linear polynomial and that of the roots of its reversed polynomial.

In the context of visual navigation, the capacity to map a novel environment is necessary for an agent to exploit its observation history in the considered place and efficiently reach known goals. This ability can be associated with spatial reasoning, where an agent is able to perceive spatial relationships and regularities, and discover object characteristics. Recent work introduces learnable policies parametrized by deep neural networks and trained with Reinforcement Learning (RL). In classical RL setups, the capacity to map and reason spatially is learned end-to-end, from reward alone. In this setting, we introduce supplementary supervision in the form of auxiliary tasks designed to favor the emergence of spatial perception capabilities in agents trained for a goal-reaching downstream objective. We show that learning to estimate metrics quantifying the spatial relationships between an agent at a given location and a goal to reach has a high positive impact in Multi-Object Navigation settings. Our method significantly improves the performance of different baseline agents, that either build an explicit or implicit representation of the environment, even matching the performance of incomparable oracle agents taking ground-truth maps as input. A learning-based agent from the literature trained with the proposed auxiliary losses was the winning entry to the Multi-Object Navigation Challenge, part of the CVPR 2021 Embodied AI Workshop.

The optimal way for a deep reinforcement learning (DRL) agent to explore is to learn a set of skills that achieves a uniform distribution of states. Following this,we introduce DisTop, a new model that simultaneously learns diverse skills and focuses on improving rewarding skills. DisTop progressively builds a discrete topology of the environment using an unsupervised contrastive loss, a growing network and a goal-conditioned policy. Using this topology, a state-independent hierarchical policy can select where the agent has to keep discovering skills in the state space. In turn, the newly visited states allows an improved learnt representation and the learning loop continues. Our experiments emphasize that DisTop is agnostic to the ground state representation and that the agent can discover the topology of its environment whether the states are high-dimensional binary data, images, or proprioceptive inputs. We demonstrate that this paradigm is competitiveon MuJoCo benchmarks with state-of-the-art algorithms on both single-task dense rewards and diverse skill discovery. By combining these two aspects, we showthat DisTop achieves state-of-the-art performance in comparison with hierarchical reinforcement learning (HRL) when rewards are sparse. We believe DisTop opens new perspectives by showing that bottom-up skill discovery combined with representation learning can unlock the exploration challenge in DRL.

The anisotropic and heterogeneous $N$-dimensional wave equation, controlled and observed at the boundary, is considered as a port-Hamiltonian system. A recent structure-preserving mixed Galerkin method is applied, leading directly to a finite-dimensional port-Hamiltonian system: its numerical analysis is carried out in a general framework. Optimal choices of mixed finite elements are then proved to reach the best trade-off between the convergence rate and the number of degrees of freedom for the state error. Exta compatibility conditions are identified for the Hamiltonian error to be twice that of the state error, and numerical evidence is provided that some combinations of finite element families meet these conditions. Numerical simulations in 2D are performed to illustrate the main theorems among several choices of classical finite element families. Several test cases are provided, including non-convex domain, anisotropic or hetergoneous cases and absorbing boundary conditions., Comment: 36 pages, 12 figure, submitted

Taking inspiration from developmental learning, we present a novel reinforcement learning architecture which hierarchically learns and represents self-generated skills in an end-to-end way. With this architecture, an agent focuses only on task-rewarded skills while keeping the learning process of skills bottom-up. This bottom-up approach allows to learn skills that 1- are transferable across tasks, 2- improves exploration when rewards are sparse. To do so, we combine a previously defined mutual information objective with a novel curriculum learning algorithm, creating an unlimited and explorable tree of skills. We test our agent on simple gridworld environments to understand and visualize how the agent distinguishes between its skills. Then we show that our approach can scale on more difficult MuJoCo environments in which our agent is able to build a representation of skills which improve over a baseline both transfer learning and exploration when rewards are sparse., Comment: Accepted at ECML 2020

Methods for discretizing port-Hamiltonian systems are of interest both for simulation and control purposes. Despite the large literature on mixed finite elements, no rigorous analysis of the connections between mixed elements and port-Hamiltonian systems has been carried out. In this paper we demonstrate how existing methods can be employed to discretize dynamical plate problems in a structure-preserving way. Based on convergence results of existing schemes, new error estimates are conjectured; numerical simulations confirm the expected behaviors.

In this paper we consider the problem of obtaining a general port-Hamiltonian formulation of Newtonian fluids. We propose the port-Hamiltonian models to describe the energy flux of rotational three-dimensional isentropic and non-isentropic fluids, whose boundary flows and efforts can be used for control purposes or for power-preserving interconnection with other physical systems. In case of two-dimensional flows, we include the considerations about the operators associated with fluid vorticity, preserving the port-Hamiltonian structure of the models proposed., Comment: 7 pages

A new formulation for the modular construction of flexible multibody systems is presented. By rearranging the equations for a flexible floating body and introducing the appropriate canonical momenta, the model is recast into a coupled system of ordinary and partial differential equations in port-Hamiltonian (pH) form. This approach relies on a floating frame description and remains valid under the assumption of small deformations. This allows including mechanical models that cannot be easily formulated in terms of differential forms. Once a pH model is established, a finite element based method is then introduced to discretize the dynamics in a structure-preserving manner. Thanks to the features of the pH framework, complex multibody systems are constructed in a modular way. Constraints are imposed at the velocity level, leading to an index 2 quasi-linear differential-algebraic system. Numerical tests are carried out to assess the validity of the proposed approach.

The reinforcement learning (RL) research area is very active, with an important number of new contributions; especially considering the emergent field of deep RL (DRL). However a number of scientific and technical challenges still need to be addressed, amongst which we can mention the ability to abstract actions or the difficulty to explore the environment which can be addressed by intrinsic motivation (IM). In this article, we provide a survey on the role of intrinsic motivation in DRL. We categorize the different kinds of intrinsic motivations and detail for each category, its advantages and limitations with respect to the mentioned challenges. Additionnally, we conduct an in-depth investigation of substantial current research questions, that are currently under study or not addressed at all in the considered research area of DRL. We choose to survey these research works, from the perspective of learning how to achieve tasks. We suggest then, that solving current challenges could lead to a larger developmental architecture which may tackle most of the tasks. We describe this developmental architecture on the basis of several building blocks composed of a RL algorithm and an IM module compressing information.

This paper presents a new algorithm: TSRuleGrowth, looking for partially-ordered rules over a time series. This algorithm takes principles from the state of the art of rule mining and applies them to time series via a new notion of support. We apply this algorithm to real data from a connected environment, which extract user habits through different connected objects., Comment: in French. Conf\'erence Nationale sur les Applications Pratiques de l'Intelligence Artificielle (APIA), Jul 2019, Toulouse, France

In 2011, Khurana, Lam and Wang define the following property. (*)A commutative unital ring A satisfies the property ''power stable range one'' if for all a, b $\in$ A with aA + bA = A there are an integer N = N (a, b) $\ge$ 1 and $\lambda$ = $\lambda$(a, b) $\in$ A such that b N + $\lambda$a $\in$ A x , the unit group of A. In 2019, Berman and Erman consider rings with the following property (**) A commutative unital ring A has enough homogeneous polynomials if for any k $\ge$ 1 and set S := {p 1 , p 2 , ..., p k } , of primitive points in A n and any n $\ge$ 2, there exists an homogeneous polynomial P (X 1 , X 2 , ..., X n) $\in$ A[X 1 , X 2 , ..., X n ]) with deg P $\ge$ 1 and P (p i) $\in$ A x for 1 $\le$ i $\le$ k. We show in this article that the two properties (*) and (**) are equivalent and we shall call a commutative unital ring with these properties a good ring. When A is a commutative unital ring of pictorsion as defined by Gabber, Lorenzini and Liu in 2015, we show that A is a good ring. Using a Dedekind domain we built by Goldman in 1963,we show that the converse is false., Comment: The paper is reorganized, some historical comments are added and some proofs are streamlined following referee's suggestions

This paper presents a structure-preserving spatial discretization method for distributed parameter port-Hamiltonian systems. The class of considered systems are hyperbolic systems of two conservation laws in arbitrary spatial dimension and geometries. For these systems, a partioned finite element method is derived, based on the integration by parts of one of the two conservation laws written in weak form. The nonlinear 1D Shallow Water Equation (SWE) is first considered as a motivation example. Then the method is investigated on the example of the nonlinear 2D SWE. Complete derivation of the reduced finite-dimensional port-Hamiltonian system is provided and numerical experiments are performed. Extensions to curvilinear (polar) coordinate systems, space-varying coefficients and higher-order port-Hamiltonian systems (Euler-Bernoulli beam equation) are provided.

This paper proves the asymptotic stability of the multidimensional wave equation posed on a bounded open Lipschitz set, coupled with various classes of positive-real impedance boundary conditions, chosen for their physical relevance: time-delayed, standard diffusive (which includes the Riemann-Liouville fractional integral) and extended diffusive (which includes the Caputo fractional derivative). The method of proof consists in formulating an abstract Cauchy problem on an extended state space using a dissipative realization of the impedance operator, be it finite or infinite-dimensional. The asymptotic stability of the corresponding strongly continuous semigroup is then obtained by verifying the sufficient spectral conditions derived by Arendt and Batty (Trans. Amer. Math. Soc., 306 (1988)) as well as Lyubich and V\~u (Studia Math., 88 (1988)).

The port-Hamiltonian formulation is a powerful method for modeling and interconnecting systems of different natures. In this paper, the port-Hamiltonian formulation in tensorial form of a thick plate described by the Mindlin-Reissner model is presented. Boundary control and observation are taken into account. Thanks to tensorial calculus, it can be seen that the Mindlin plate model mimics the interconnection structure of its one-dimensional counterpart, i.e. the Timoshenko beam. The Partitioned Finite Element Method (PFEM) is then extended to both the vectorial and tensorial formulations in order to obtain a suitable, i.e. structure-preserving, finite-dimensional port-Hamiltonian system (PHs), which preserves the structure and properties of the original distributed parameter system. Mixed boundary conditions are finally handled by introducing some algebraic constraints. Numerical examples are finally presented to validate this approach.

The mechanical model of a thin plate with boundary control and observation is presented as a port-Hamiltonian system (pHs), both in vectorial and tensorial forms: the Kirchhoff-Love model of a plate is described by using a Stokes-Dirac structure and this represents a novelty with respect to the existing literature. This formulation is carried out both in vectorial and tensorial forms. Thanks to tensorial calculus, this model is found to mimic the interconnection structure of its one-dimensional counterpart, i.e. the Euler-Bernoulli beam. The Partitioned Finite Element Method (PFEM) is then extended to obtain a suitable, i.e. structure-preserving, weak form. The discretization procedure, performed on the vectorial formulation, leads to a finite-dimensional port-Hamiltonian system. This part II of the companion paper extends part I, dedicated to the Mindlin model for thick plates. The thin plate model comes along with additional difficulties, because of the higher order of the differential operator under consideration., Comment: arXiv admin note: text overlap with arXiv:1809.11131

A fractional time derivative is introduced into the Burger's equation to model losses of nonlinear waves. This term amounts to a time convolution product, which greatly penalizes the numerical modeling. A diffusive representation of the fractional derivative is adopted here, replacing this nonlocal operator by a continuum of memory variables that satisfy local-in-time ordinary differential equations. Then a quadrature formula yields a system of local partial differential equations, well-suited to numerical integration. The determination of the quadrature coefficients is crucial to ensure both the well-posedness of the system and the computational efficiency of the diffusive approximation. For this purpose, optimization with constraint is shown to be a very efficient strategy. Strang splitting is used to solve successively the hyperbolic part by a shock-capturing scheme, and the diffusive part exactly. Numerical experiments are proposed to assess the efficiency of the numerical modeling, and to illustrate the effect of the fractional attenuation on the wave propagation., Comment: submitted to Siam SIAP