Search

The framework of Mean-field Games (MFGs) is used for modelling the collective dynamics of large populations of non-cooperative decision-making agents. We formulate and analyze a kinetic MFG model for an interacting system of non-cooperative motile agents with inertial dynamics and finite-range interactions, where each agent is minimizing a biologically inspired cost function. By analyzing the associated coupled forward-backward in time system of nonlinear Fokker-Planck and Hamilton-Jacobi-Bellman equations, we obtain conditions for closed-loop linear stability of the spatially homogeneous MFG equilibrium that corresponds to an ordered state with non-zero mean speed. Using a combination of analysis and numerical simulations, we show that when energetic cost of control is reduced below a critical value, this equilibrium loses stability, and the system transitions to a travelling wave solution. Our work provides a game-theoretic perspective to the problem of collective motion in non-equilibrium biological and bio-inspired systems.

Mean-field games (MFG) provide a statistical physics inspired modeling framework for decision making in large-populations of strategic, non-cooperative agents. Mathematically, these systems consist of a forward-backward in time system of two coupled nonlinear partial differential equations (PDEs), namely the Fokker-Plank and the Hamilton-Jacobi-Bellman equations, governing the agent state and control distribution, respectively. In this work, we study a finite-time MFG with a rich global bifurcation structure using a reduced-order model (ROM). The ROM is a 4D two-point boundary value problem obtained by restricting the controlled dynamics to first two moments of the agent state distribution, i.e., the mean and the variance. Phase space analysis of the ROM reveals that the invariant manifolds of periodic orbits around the so-called `ergodic MFG equilibrium' play a crucial role in determining the bifurcation diagram, and impart a topological signature to various solution branches. We show a qualitative agreement of these results with numerical solutions of the full-order MFG PDE system., Comment: 32 pages, 16 figures

This work is a unified study of stable and unstable steady states of 2D active nematic channel flow using the framework of Exact Coherent Structures (ECS). ECS are stationary, periodic, quasiperiodic, or traveling wave solutions of the governing equations that, together with their invariant manifolds, organize the dynamics of nonlinear continuum systems. We extend our earlier work on ECS in the preturbulent regime by performing a comprehensive study of stable and unstable ECS for a wide range of activity values spanning the preturbulent and turbulent regimes. In the weakly turbulent regime, we compute more than 200 unstable ECS that co-exist at a single set of parameters, and uncover the role of symmetries in organizing the phase space geometry. We provide conclusive numerical evidence that in the preturbulent regime, generic trajectories shadow a series of unstable ECS before settling onto an attractor. Finally, our studies hint at shadowing of quasiperiodic type ECS in the turbulent regime., Comment: 10 figures

We propose a reaction-diffusion system that converts topological information of an active nematic into chemical signals. We show that a curvature-activated reaction dipole is sufficient for creating a system that dynamically senses topology by producing a concentration field possessing local extrema coinciding with $\pm\frac{1}{2}$ defects. The enabling term is analogous to polarization charge density seen in dielectric materials. We demonstrate the ability of this system to identify defects in both passive and active nematics. The model demonstrates that a relatively simple feedback scheme in the form of a PDE system is capable of producing chemical signals in response to inherently non-local structures in anisotropic media. We posit that such coarse-grained systems can help generate testable hypotheses for regulated processes in biological systems such as morphogenesis and motivate the creation of bioinspired materials that utilize dynamic coupling between nematic structure and biochemistry.

We study the dynamics of solitary waves traveling in a one-dimensional chain of bistable elements in the presence of a local inhomogeneity (defect). Numerical simulations reveal that depending upon its initial speed, an incoming solitary wave can get transmitted, captured or reflected upon interaction with the defect. The dynamics are dominated by energy exchange between the wave and a breather mode localized at the defect. We derive a reduced-order two degree of freedom Hamiltonian model for wave-breather interaction, and analyze it using dynamical systems techniques. Lobe dynamics analysis reveals the fine structure of phase space that leads to the complicated dynamics in this system. This work is a step towards developing a rational approach to defect engineering for manipulating nonlinear waves in mechanical metamaterials., Comment: 36 pages, 13 figures

Confined active nematics exhibit rich dynamical behavior, including spontaneous flows, periodic defect dynamics, and chaotic `active turbulence'. Here, we study these phenomena using the framework of Exact Coherent Structures, which has been successful in characterizing the routes to high Reynolds number turbulence of passive fluids. Exact Coherent Structures are stationary, periodic, quasiperiodic, or traveling wave solutions of the hydrodynamic equations that, together with their invariant manifolds, serve as an organizing template of the dynamics. We compute the dominant Exact Coherent Structures and connecting orbits in a pre-turbulent active nematic channel flow, which enables a fully nonlinear but highly reduced order description in terms of a directed graph. Using this reduced representation, we compute instantaneous perturbations that switch the system between disparate spatiotemporal states occupying distant regions of the infinite dimensional phase space. Our results lay the groundwork for a systematic means of understanding and controlling active nematic flows in the moderate to high activity regime.

In this work we present the first systematic framework to sculpt active nematics systems, using optimal control theory and a hydrodynamic model of active nematics. We demonstrate the use of two different control fields, (1) applied vorticity and (2) activity strength, to shape the dynamics of an extensile active nematic that is confined to a disk. In the absence of control inputs, the system exhibits two attractors, clockwise and counterclockwise circulating states characterized by two co-rotating topological $+\frac{1}{2}$ defects. We specifically seek spatiotemporal inputs that switch the system from one attractor to the other; we also examine phase-shifting perturbations. We identify control inputs by optimizing a penalty functional with three contributions: total control effort, spatial gradients in the control, and deviations from the desired trajectory. This work demonstrates that optimal control theory can be used to calculate non-trivial inputs capable of restructuring active nematics in a manner that is economical, smooth, and rapid, and therefore will serve as a guide to experimental efforts to control active matter.

We consider the problem of incentivization and optimal control of autonomous vehicles for improving traffic congestion. In our scenario, autonomous vehicles must be incentivized in order to participate in traffic improvement. Using the theory and methods of optimal transport, we propose a constrained optimization framework over dynamics governed by partial differential equations, so that we can optimally select a portion of vehicles to be incentivized and controlled. The goal of the optimization is to obtain a uniform distribution of vehicles over the spatial domain. To achieve this, we consider two types of penalties on vehicle density, one is the $L^2$ cost and the other is a multiscale-norm cost, commonly used in fluid-mixing problems. To solve this non-convex optimization problem, we introduce a novel algorithm, which iterates between solving a convex optimization problem and propagating the flow of uncontrolled vehicles according to the Lighthill-Whitham-Richards model. We perform numerical simulations, which suggest that the optimization of the $L^2$ cost is ineffective while optimization of the multiscale norm is effective. The results also suggest the use of a dedicated lane for this type of control in practice., Comment: 6 pages, 10 figures, submitted to the 2020 IEEE Intelligent Vehicles Symposium

Control of continuous time dynamics with multiplicative noise is a classic topic in stochastic optimal control. This work addresses the problem of designing infinite horizon optimal controls with stability guarantees for \textit{a single agent or large population systems} of identical, non-cooperative and non-networked agents, with multi-dimensional and nonlinear stochastic dynamics excited by multiplicative noise. For agent dynamics belonging to the class of reversible diffusion processes, we provide constraints on the state and control cost functions which guarantee stability of the closed-loop system under the action of the individual optimal controls. A condition relating the state-dependent control cost and volatility is introduced to prove the stability of the equilibrium density. This condition is a special case of the constraint required to use the path integral Feynman-Kac formula for computing the control. We investigate the connection between the stabilizing optimal control and the path integral formalism, leading us to a control law formulation expressed exclusively in terms of the desired equilibrium density., Comment: 6 pages, 0 figures, IEEE Conference on Decision and Control 2020 (accepted)

Recent work has shown that reinforcement learning (RL) is a promising approach to control dynamical systems described by partial differential equations (PDE). This paper shows how to use RL to tackle more general PDE control problems that have continuous high-dimensional action spaces with spatial relationship among action dimensions. In particular, we propose the concept of action descriptors, which encode regularities among spatially-extended action dimensions and enable the agent to control high-dimensional action PDEs. We provide theoretical evidence suggesting that this approach can be more sample efficient compared to a conventional approach that treats each action dimension separately and does not explicitly exploit the spatial regularity of the action space. The action descriptor approach is then used within the deep deterministic policy gradient algorithm. Experiments on two PDE control problems, with up to 256-dimensional continuous actions, show the advantage of the proposed approach over the conventional one., Comment: ICML2018

A One-Dimensional (1D) Reduced-Order Model (ROM) has been developed for a 3D Rayleigh-B\'enard convection system in the turbulent regime with Rayleigh number $\mathrm{Ra}=10^6$. The state vector of the 1D ROM is horizontally averaged temperature. Using the Green's Function (GRF) method, which involves applying many localized, weak forcings to the system one at a time and calculating the responses using long-time averaged Direct Numerical Simulations (DNS), the system's Linear Response Function (LRF) has been computed. Another matrix, called the Eddy Flux Matrix (EFM), that relates changes in the divergence of vertical eddy heat fluxes to changes in the state vector, has also been calculated. Using various tests, it is shown that the LRF and EFM can accurately predict the time-mean responses of temperature and eddy heat flux to external forcings, and that the LRF can well predict the forcing needed to change the mean flow in a specified way (inverse problem). The non-normality of the LRF is discussed and its eigen/singular vectors are compared with the leading Proper Orthogonal Decomposition (POD) modes of the DNS data. Furthermore, it is shown that if the LRF and EFM are simply scaled by the square-root of Rayleigh number, they perform equally well for flows at other $\mathrm{Ra}$, at least in the investigated range of $ 5 \times 10^5 \le \mathrm{Ra} \le 1.25 \times 10^6$. The GRF method can be applied to develop 1D or 3D ROMs for any turbulent flow, and the calculated LRF and EFM can help with better analyzing and controlling the nonlinear system., Comment: Accepted for publication in Physical Review Fluids

This paper considers the relaxed version of the transport problem for general nonlinear control systems, where the objective is to design time-varying feedback laws that transport a given initial probability measure to a target probability measure under the action of the closed-loop system. To make the problem analytically tractable, we consider control laws that are stochastic, i.e., the control laws are maps from the state space of the control system to the space of probability measures on the set of admissible control inputs. Under some controllability assumptions on the control system as defined on the state space, we show that the transport problem, considered as a controllability problem for the lifted control system on the space of probability measures, is well-posed for a large class of initial and target measures. We use this to prove the well-posedness of a fixed-endpoint optimal control problem defined on the space of probability measures, where along with the terminal constraints, the goal is to optimize an objective functional along the trajectory of the control system. This optimization problem can be posed as an infinite-dimensional linear programming problem. This formulation facilitates numerical solutions of the transport problem for low-dimensional control systems, as we show in two numerical examples., Comment: Final version

Empirically derived continuum models of collective behavior among large populations of dynamic agents are a subject of intense study in several fields, including biology, engineering and finance. We formulate and study a mean-field game model whose behavior mimics an empirically derived non-local homogeneous flocking model for agents with gradient self-propulsion dynamics. The mean-field game framework provides a non-cooperative optimal control description of the behavior of a population of agents in a distributed setting. In this description, each agent's state is driven by optimally controlled dynamics that result in a Nash equilibrium between itself and the population. The optimal control is computed by minimizing a cost that depends only on its own state, and a mean-field term. The agent distribution in phase space evolves under the optimal feedback control policy. We exploit the low-rank perturbative nature of the non-local term in the forward-backward system of equations governing the state and control distributions, and provide a linear stability analysis demonstrating that our model exhibits bifurcations similar to those found in the empirical model. The present work is a step towards developing a set of tools for systematic analysis, and eventually design, of collective behavior of non-cooperative dynamic agents via an inverse modeling approach., Comment: Revised to incorporate reviewers' suggestions. Accepted to Chaos journal

We present a set-oriented graph-based computational framework for continuous-time optimal transport over nonlinear dynamical systems. We recover provably optimal control laws for steering a given initial distribution in phase space to a final distribution in prescribed finite time for the case of non-autonomous nonlinear control-affine systems, while minimizing a quadratic control cost. The resulting control law can be used to obtain approximate feedback laws for individual agents in a swarm control application. Using infinitesimal generators, the optimal control problem is reduced to a modified Monge-Kantorovich optimal transport problem, resulting in a convex Benamou-Brenier type fluid dynamics formulation on a graph. The well-posedness of this problem is shown to be a consequence of the graph being strongly-connected, which in turn is shown to result from controllability of the underlying dynamical system. Using our computational framework, we study optimal transport of distributions where the underlying dynamical systems are chaotic, and non-holonomic. The solutions to the optimal transport problem elucidate the role played by invariant manifolds, lobe-dynamics and almost-invariant sets in efficient transport of distributions in finite time. Our work connects set-oriented operator-theoretic methods in dynamical systems with optimal mass transportation theory, and opens up new directions in design of efficient feedback control strategies for nonlinear multi-agent and swarm systems operating in nonlinear ambient flow fields., Comment: 34 pages, 12 figures. Incorporated reviewers' suggestions

We formulate and solve a class of finite-time transport and mixing problems in the set-oriented framework. The aim is to obtain optimal discrete-time perturbations in nonlinear dynamical systems to transport a specified initial measure on the phase space to a final measure in finite time. The measure is propagated under system dynamics in between the perturbations via the associated transfer operator. Each perturbation is described by a deterministic map in the measure space that implements a version of Monge-Kantorovich optimal transport with quadratic cost. Hence, the optimal solution minimizes a sum of quadratic costs on phase space transport due to the perturbations applied at specified times. The action of the transport map is approximated by a continuous pseudo-time flow on a graph, resulting in a tractable convex optimization problem. This problem is solved via state-of-the-art solvers to global optimality. We apply this algorithm to a problem of transport between measures supported on two disjoint almost-invariant sets in a chaotic fluid system, and to a finite-time optimal mixing problem by choosing the final measure to be uniform. In both cases, the optimal perturbations are found to exploit the phase space structures, such as lobe dynamics, leading to efficient global transport. As the time-horizon of the problem is increased, the optimal perturbations become increasingly localized. Hence, by combining the transfer operator approach with ideas from the theory of optimal mass transportation, we obtain a discrete-time graph-based algorithm for optimal transport and mixing in nonlinear systems., Comment: With additional numerics. 23 pages, 17 figures. To appear in CNSNS

We study the problem of optimizing the performance of a nonlinear spring-mass-damper attached to a class of multiple-degree-of-freedom systems. We aim to maximize the rate of one-way energy transfer from primary system to the attachment, and focus on impulsive excitation of a two-degree-of-freedom primary system with an essentially nonlinear attachment. The nonlinear attachment is shown to be able to perform as a `nonlinear energy sink' (NES) by taking away energy from the primary system irreversibly for some types of impulsive excitations. Using perturbation analysis and exploiting separation of time scales, we perform dimensionality reduction of this strongly nonlinear system. Our analysis shows that efficient energy transfer to nonlinear attachment in this system occurs for initial conditions close to homoclinic orbit of the slow time-scale undamped system, a phenomenon that has been previously observed for the case of single-degree-of-freedom primary systems. Analytical formulae for optimal parameters for given impulsive excitation input are derived. Generalization of this framework to systems with arbitrary number of degrees-of-freedom of the primary system is also discussed. The performance of both linear and nonlinear optimally tuned attachments is compared. While NES performance is sensitive to magnitude of the initial impulse, our results show that NES performance is more robust than linear tuned-mass-damper to several parametric perturbations. Hence, our work provides evidence that homoclinic orbits of the underlying Hamiltonian system play a crucial role in efficient nonlinear energy transfers, even in high dimensional systems, and gives new insight into robustness of systems with essential nonlinearity., Comment: Updated results. 34 pages, 29 figures

We present a sparse sensing framework based on Dynamic Mode Decomposition (DMD) to identify flow regimes and bifurcations in large-scale thermo-fluid systems. Motivated by real-time sensing and control of thermal-fluid flows in buildings and equipment, we apply this method to a Direct Numerical Simulation (DNS) data set of a 2D laterally heated cavity. The resulting flow solutions can be divided into several regimes, ranging from steady to chaotic flow. The DMD modes and eigenvalues capture the main temporal and spatial scales in the dynamics belonging to different regimes. Our proposed classification method is data-driven, robust w.r.t measurement noise, and exploits the dynamics extracted from the DMD method. Namely, we construct an augmented DMD basis, with "built-in" dynamics, given by the DMD eigenvalues. This allows us to employ a short time-series of data from sensors, to more robustly classify flow regimes, particularly in the presence of measurement noise. We also exploit the incoherence exhibited among the data generated by different regimes, which persists even if the number of measurements is small compared to the dimension of the DNS data. The data-driven regime identification algorithm can enable robust low-order modeling of flows for state estimation and control., Comment: Expanded discussion. Fixed some typos and figures

We present results on stabilization for reduced order models (ROM) of partial differential equations using learning. Stabilization is achieved via closure models for ROMs, where we use a model-free extremum seeking (ES) dither-based algorithm to learn the best closure models' parameters, for optimal ROM stabilization. We first propose to auto-tune linear closure models using ES, and then extend the results to a closure model combining linear and nonlinear terms, for better stabilization performance. The coupled Burgers' equation is employed as a test-bed for the proposed tuning method.

The deformable and continuum nature of soft robots promises versatility and adaptability. However, control of modular, multi-limbed soft robots for terrestrial locomotion is challenging due to the complex robot structure, actuator mechanics and robot-environment interaction. Traditionally, soft robot control is performed by modeling kinematics using exact geometric equations and finite element analysis. The research presents an alternative, model-free, data-driven, reinforcement learning inspired approach, for controlling multi-limbed soft material robots. This control approach can be summarized as a four-step process of discretization, visualization, learning and optimization. The first step involves identification and subsequent discretization of key factors that dominate robot-environment, in turn, the robot control. Graph theory is used to visualize relationships and transitions between the discretized states. The graph representation facilitates mathematical definition of periodic control patterns (simple cycles) and locomotion gaits. Rewards corresponding to individual arcs of the graph are weighted displacement and orientation change for robot state-to-state transitions. These rewards are specific to surface of locomotion and are learned. Finally, the control patterns result from optimization of reward dependent locomotion task (e.g. translation) cost function. The optimization problem is an Integer Linear Programming problem which can be quickly solved using standard solvers. The framework is generic and independent of type of actuator, soft material properties or the type of friction mechanism, as the control exists in the robot's task space. Furthermore, the data-driven nature of the framework imparts adaptability to the framework toward different locomotion surfaces by re-learning rewards., Comment: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015

In certain two-dimensional time-dependent flows, the braiding of periodic orbits provides a way to analyze chaos in the system through application of the Thurston-Nielsen classification theorem (TNCT). We expand upon earlier work that introduced the application of the TNCT to braiding of almost-cyclic sets, which are individual components of almost-invariant sets [Stremler, Ross, Grover, Kumar, Topological chaos and periodic braiding of almost-cyclic sets. Physical Review Letters 106 (2011), 114101]. In this context, almost-cyclic sets are periodic regions in the flow with high local residence time that act as stirrers or `ghost rods' around which the surrounding fluid appears to be stretched and folded. In the present work, we discuss the bifurcation of the almost-cyclic sets as a system parameter is varied, which results in a sequence of topologically distinct braids. We show that, for Stokes' flow in a lid-driven cavity, these various braids give good lower bounds on the topological entropy over the respective parameter regimes in which they exist. We make the case that a topological analysis based on spatiotemporal braiding of almost-cyclic sets can be used for analyzing chaos in fluid flows. Hence we further develop a connection between set-oriented statistical methods and topological methods, which promises to be an important analysis tool in the study of complex systems., Comment: Submitted to Chaos

For low energy spacecraft trajectories such as multi-moon orbiters for the Jupiter system, multiple gravity assists by moons could be used in conjunction with ballistic capture to drastically decrease fuel usage. In this paper, we outline a procedure to obtain a family of zero-fuel multi-moon orbiter trajectories, using a family of Keplerian maps derived by the first author previously. The maps capture well the dynamics of the full equations of motion; the phase space contains a connected chaotic zone where intersections between unstable resonant orbit manifolds provide the template for lanes of fast migration between orbits of different semimajor axes. Patched three body approach is used and the four body problem is broken down into two three-body problems, and the search space is considerably reduced by the use of properties of the Keplerian maps. We also introduce the notion of Switching Region where the perturbations due to the two perturbing moons are of comparable strength, and which separates the domains of applicability of the corresponding two Keplerian maps.