Search

The reconstruction and estimation of spatio-temporal patterns poses significant challenges when sensor measurements are limited. The use of mobile sensors adds additional complexity due to the change in sensor locations over time. In such cases, historical measurement and sensor information are useful for better performance, including models such as Kalman filters, recurrent neural networks (RNNs) or transformer models. However, many of these approaches often fail to efficiently handle long sequences of data in such scenarios and are sensitive to noise. In this paper, we consider a model-free approach using the {\em structured state space sequence} (S4D) model as a deep learning layer in traditional sequence models to learn a better representation of historical sensor data. Specifically, it is integrated with a shallow decoder network for reconstruction of the high-dimensional state space. We also introduce a novel initialization of the S4D model using a Butterworth filter design to reduce noise in the inputs. Consequently, we construct a robust S4D (rS4D) model by appending the filtering S4D layer before the original S4D structure. This robust variant enhances the capability to accurately reconstruct spatio-temporal patterns with noisy mobile sensor measurements in long sequence. Numerical experiments demonstrate that our model achieves better performance compared with previous approaches. Our results underscore the efficacy of leveraging state space models within the context of spatio-temporal data reconstruction and estimation using limited mobile sensor resources, particularly in terms of long-sequence dependency and robustness to noise., Comment: 18 pages, 11 figures

The simulation of many complex phenomena in engineering and science requires solving expensive, high-dimensional systems of partial differential equations (PDEs). To circumvent this, reduced-order models (ROMs) have been developed to speed up computations. However, when governing equations are unknown or partially known, typically ROMs lack interpretability and reliability of the predicted solutions. In this work we present a data-driven, non-intrusive framework for building ROMs where the latent variables and dynamics are identified in an interpretable manner and uncertainty is quantified. Starting from a limited amount of high-dimensional, noisy data the proposed framework constructs an efficient ROM by leveraging variational autoencoders for dimensionality reduction along with a newly introduced, variational version of sparse identification of nonlinear dynamics (SINDy), which we refer to as Variational Identification of Nonlinear Dynamics (VINDy). In detail, the method consists of Variational Encoding of Noisy Inputs (VENI) to identify the distribution of reduced coordinates. Simultaneously, we learn the distribution of the coefficients of a pre-determined set of candidate functions by VINDy. Once trained offline, the identified model can be queried for new parameter instances and new initial conditions to compute the corresponding full-time solutions. The probabilistic setup enables uncertainty quantification as the online testing consists of Variational Inference naturally providing Certainty Intervals (VICI). In this work we showcase the effectiveness of the newly proposed VINDy method in identifying interpretable and accurate dynamical system for the R\"ossler system with different noise intensities and sources. Then the performance of the overall method - named VENI, VINDy, VICI - is tested on PDE benchmarks including structural mechanics and fluid dynamics.

We introduce the optimized dynamic mode decomposition algorithm for constructing an adaptive and computationally efficient reduced order model and forecasting tool for global atmospheric chemistry dynamics. By exploiting a low-dimensional set of global spatio-temporal modes, interpretable characterizations of the underlying spatial and temporal scales can be computed. Forecasting is also achieved with a linear model that uses a linear superposition of the dominant spatio-temporal features. The DMD method is demonstrated on three months of global chemistry dynamics data, showing its significant performance in computational speed and interpretability. We show that the presented decomposition method successfully extracts known major features of atmospheric chemistry, such as summertime surface pollution and biomass burning activities. Moreover, the DMD algorithm allows for rapid reconstruction of the underlying linear model, which can then easily accommodate non-stationary data and changes in the dynamics., Comment: 13 pages, 16 figures

Deep reinforcement learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in environments with complicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamak fusion reactor or minimizing the drag force exerted on an object in a fluid flow. However, these algorithms require an abundance of training examples and may become prohibitively expensive for many applications. In addition, the reliance on deep neural networks often results in an uninterpretable, black-box policy that may be too computationally expensive to use with certain embedded systems. Recent advances in sparse dictionary learning, such as the sparse identification of nonlinear dynamics (SINDy), have shown promise for creating efficient and interpretable data-driven models in the low-data regime. In this work we introduce SINDy-RL, a unifying framework for combining SINDy and DRL to create efficient, interpretable, and trustworthy representations of the dynamics model, reward function, and control policy. We demonstrate the effectiveness of our approaches on benchmark control environments and challenging fluids problems. SINDy-RL achieves comparable performance to state-of-the-art DRL algorithms using significantly fewer interactions in the environment and results in an interpretable control policy orders of magnitude smaller than a deep neural network policy., Comment: 24 pages + 14 appendices (45 pages total). 25 figures, 7 tables. For code, see https://github.com/nzolman/sindy-rl

Crash simulations play an essential role in improving vehicle safety, design optimization, and injury risk estimation. Unfortunately, numerical solutions of such problems using state-of-the-art high-fidelity models require significant computational effort. Conventional data-driven surrogate modeling approaches create low-dimensional embeddings for evolving the dynamics in order to circumvent this computational effort. Most approaches directly operate on high-resolution data obtained from numerical discretization, which is both costly and complicated for mapping the flow of information over large spatial distances. Furthermore, working with a fixed resolution prevents the adaptation of surrogate models to environments with variable computing capacities, different visualization resolutions, and different accuracy requirements. We thus propose a multi-hierarchical framework for structurally creating a series of surrogate models for a kart frame, which is a good proxy for industrial-relevant crash simulations, at different levels of resolution. For multiscale phenomena, macroscale features are captured on a coarse surrogate, whereas microscale effects are resolved by finer ones. The learned behavior of the individual surrogates is passed from coarse to finer levels through transfer learning. In detail, we perform a mesh simplification on the kart model to obtain multi-resolution representations of it. We then train a graph-convolutional neural network-based surrogate that learns parameter-dependent low-dimensional latent dynamics on the coarsest representation. Subsequently, another, similarly structured surrogate is trained on the residual of the first surrogate using a finer resolution. This step can be repeated multiple times. By doing so, we construct multiple surrogates for the same system with varying hardware requirements and increasing accuracy.

The dynamic mode decomposition (DMD) is a simple and powerful data-driven modeling technique that is capable of revealing coherent spatiotemporal patterns from data. The method's linear algebra-based formulation additionally allows for a variety of optimizations and extensions that make the algorithm practical and viable for real-world data analysis. As a result, DMD has grown to become a leading method for dynamical system analysis across multiple scientific disciplines. PyDMD is a Python package that implements DMD and several of its major variants. In this work, we expand the PyDMD package to include a number of cutting-edge DMD methods and tools specifically designed to handle dynamics that are noisy, multiscale, parameterized, prohibitively high-dimensional, or even strongly nonlinear. We provide a complete overview of the features available in PyDMD as of version 1.0, along with a brief overview of the theory behind the DMD algorithm, information for developers, tips regarding practical DMD usage, and introductory coding examples. All code is available at https://github.com/PyDMD/PyDMD .

Sensing is one of the most fundamental tasks for the monitoring, forecasting and control of complex, spatio-temporal systems. In many applications, a limited number of sensors are mobile and move with the dynamics, with examples including wearable technology, ocean monitoring buoys, and weather balloons. In these dynamic systems (without regions of statistical-independence), the measurement time history encodes a significant amount of information that can be extracted for critical tasks. Most model-free sensing paradigms aim to map current sparse sensor measurements to the high-dimensional state space, ignoring the time-history all together. Using modern deep learning architectures, we show that a sequence-to-vector model, such as an LSTM (long, short-term memory) network, with a decoder network, dynamic trajectory information can be mapped to full state-space estimates. Indeed, we demonstrate that by leveraging mobile sensor trajectories with shallow recurrent decoder networks, we can train the network (i) to accurately reconstruct the full state space using arbitrary dynamical trajectories of the sensors, (ii) the architecture reduces the variance of the mean-square error of the reconstruction error in comparison with immobile sensors, and (iii) the architecture also allows for rapid generalization (parameterization of dynamics) for data outside the training set. Moreover, the path of the sensor can be chosen arbitrarily, provided training data for the spatial trajectory of the sensor is available. The exceptional performance of the network architecture is demonstrated on three applications: turbulent flows, global sea-surface temperature data, and human movement biomechanics., Comment: 11 pages, 5 figures, 2 tables

PyKoopman is a Python package for the data-driven approximation of the Koopman operator associated with a dynamical system. The Koopman operator is a principled linear embedding of nonlinear dynamics and facilitates the prediction, estimation, and control of strongly nonlinear dynamics using linear systems theory. In particular, PyKoopman provides tools for data-driven system identification for unforced and actuated systems that build on the equation-free dynamic mode decomposition (DMD) and its variants. In this work, we provide a brief description of the mathematical underpinnings of the Koopman operator, an overview and demonstration of the features implemented in PyKoopman (with code examples), practical advice for users, and a list of potential extensions to PyKoopman. Software is available at http://github.com/dynamicslab/pykoopman, Comment: 16 pages

Partial differential equations (PDEs) are among the most universal and parsimonious descriptions of natural physical laws, capturing a rich variety of phenomenology and multi-scale physics in a compact and symbolic representation. This review will examine several promising avenues of PDE research that are being advanced by machine learning, including: 1) the discovery of new governing PDEs and coarse-grained approximations for complex natural and engineered systems, 2) learning effective coordinate systems and reduced-order models to make PDEs more amenable to analysis, and 3) representing solution operators and improving traditional numerical algorithms. In each of these fields, we summarize key advances, ongoing challenges, and opportunities for further development., Comment: 16 pages, 6 figures

Sparse model identification enables nonlinear dynamical system discovery from data. However, the control of false discoveries for sparse model identification is challenging, especially in the low-data and high-noise limit. In this paper, we perform a theoretical study on ensemble sparse model discovery, which shows empirical success in terms of accuracy and robustness to noise. In particular, we analyse the bootstrapping-based sequential thresholding least-squares estimator. We show that this bootstrapping-based ensembling technique can perform a provably correct variable selection procedure with an exponential convergence rate of the error rate. In addition, we show that the ensemble sparse model discovery method can perform computationally efficient uncertainty estimation, compared to expensive Bayesian uncertainty quantification methods via MCMC. We demonstrate the convergence properties and connection to uncertainty quantification in various numerical studies on synthetic sparse linear regression and sparse model discovery. The experiments on sparse linear regression support that the bootstrapping-based sequential thresholding least-squares method has better performance for sparse variable selection compared to LASSO, thresholding least-squares, and bootstrapping-based LASSO. In the sparse model discovery experiment, we show that the bootstrapping-based sequential thresholding least-squares method can provide valid uncertainty quantification, converging to a delta measure centered around the true value with increased sample sizes. Finally, we highlight the improved robustness to hyperparameter selection under shifting noise and sparsity levels of the bootstrapping-based sequential thresholding least-squares method compared to other sparse regression methods., Comment: 32 pages, 7 figures

Sensing is a universal task in science and engineering. Downstream tasks from sensing include inferring full state estimates of a system (system identification), control decisions, and forecasting. These tasks are exceptionally challenging to achieve with limited sensors, noisy measurements, and corrupt or missing data. We propose a SHallow REcurrent Decoder (SHRED) neural network structure for sensing which incorporates (i) a recurrent neural network (LSTM) to learn a latent representation of the temporal dynamics of the sensors, and (ii) a shallow decoder that learns a mapping between this latent representation and the high-dimensional state space. By explicitly accounting for the time-history, or trajectory, of the sensor measurements, SHRED enables accurate reconstructions with far fewer sensors, outperforms existing techniques when more measurements are available, and is agnostic towards sensor placement. In addition, a compressed representation of the high-dimensional state is directly obtained from sensor measurements, which provides an on-the-fly compression for modeling physical and engineering systems. Forecasting is also achieved from the sensor time-series data alone, producing an efficient paradigm for predicting temporal evolution with an exceptionally limited number of sensors. In the example cases explored, including turbulent flows, complex spatio-temporal dynamics can be characterized with exceedingly limited sensors that can be randomly placed with minimal loss of performance., Comment: 12 pages, 7 figures

Damped-driven systems are ubiquitous in engineering and science. Despite the diversity of physical processes observed in a broad range of applications, the underlying instabilities observed in practice have a universal characterization which is determined by the overall gain and loss curves of a given system. The universal behavior of damped-driven systems can be understood from a geometrical description of the energy balance with a minimal number of assumptions. The assumptions on the energy dynamics are as follows: the energy increases monotonically as a function of increasing gain, and the losses become increasingly larger with increasing energy, i.e. there are many routes for dissipation in the system for large input energy. The intersection of the gain and loss curves define an energy balanced solution. By constructing an iterative map between the loss and gain curves, the dynamics can be shown to be homeomorphic to the logistic map, which exhibits a period doubling cascade to chaos. Indeed, the loss and gain curves allow for a geometrical description of the dynamics through a simple Verhulst diagram (cobweb plot). Thus irrespective of the physics and its complexities, this simple geometrical description dictates the universal set of logistic map instabilities that arise in complex damped-driven systems. More broadly, damped-driven systems are a class of non-equilibrium pattern forming systems which have a canonical set of instabilities that are manifest in practice., Comment: 26 pages, 31 figures

We develop a data-driven characterization of the pilot-wave hydrodynamic system in which a bouncing droplet self-propels along the surface of a vibrating bath. We consider drop motion in a confined one-dimensional geometry, and apply the {\em Dynamic mode decomposition} (DMD) in order to characterize the evolution of the wave field as the bath's vibrational acceleration is increased progressively. DMD provides a regression framework for adaptively learning a best-fit linear dynamics model over snapshots of spatio-temporal data. The DMD characterization of the wave field yields a fresh perspective on the bouncing-droplet problem that forges valuable new links with the mathematical machinery of quantum mechanics. Moreover, it provides a low-rank characterization of the bifurcation structure of the pilot wave physics. Specifically, the analysis shows that as the vibrational acceleration is increased, the pilot-wave field undergoes a series of Hopf bifurcations that ultimately lead to a chaotic wave field. The established relation between the mean pilot-wave field and the droplet statistics allows us to characterize the evolution of the emergent statistics with increased vibrational forcing from the evolution of the pilot-wave field. We thus develop a numerical framework with the same basic structure as quantum mechanics, specifically a wave theory that predicts particle statistics., Comment: 12 pages, 10 figures

Pendulums are simple mechanical systems that have been studied for centuries and exhibit many aspects of modern dynamical systems theory. In particular, the double pendulum is a prototypical chaotic system that is frequently used to illustrate a variety of phenomena in nonlinear dynamics. In this work, we explore the existence and implications of codimension-1 invariant manifolds in the double pendulum, which originate from unstable periodic orbits around saddle equilibria and act as separatrices that mediate the global phase space transport. Motivated in part by similar studies on the three-body problem, we are able to draw a direct comparison between the dynamics of the double pendulum and transport in the solar system, which exist on vastly different scales. Thus, the double pendulum may be viewed as a table-top benchmark for chaotic, saddle-mediated transport, with direct relevance to energy-efficient space mission design. The analytical results of this work provide an existence result, concerning arbitrarily long itineraries in phase space, that is applicable to a wide class of two degree of freedom Hamiltonian systems, including the three-body problem and the double pendulum. This manuscript details a variety of periodic orbits corresponding to acrobatic motions of the double pendulum that can be identified and controlled in a laboratory setting., Comment: 47 pages, 17 figures Version 2 made minor changes based on reviewers comments

Nonlinear ordinary differential equations can rarely be solved analytically. Koopman operator theory provides a way to solve nonlinear systems by mapping nonlinear dynamics to a linear space using eigenfunctions. Unfortunately, finding such eigenfunctions is difficult. We introduce a method for constructing eigenfunctions from a nonlinear ODE's invariant manifolds. This method, when successful, allows us to find analytical solutions for constant coefficient nonlinear systems. Previous data-driven methods have used Koopman theory to construct local Koopman eigenfunction approximations valid in different regions of phase space; our method finds analytic Koopman eigenfunctions that are exact and globally valid. We demonstrate our Koopman method of solving nonlinear systems on 1-dimensional and 2-dimensional ODEs. The nonlinear examples considered have simple expressions for their invariant manifolds which produce tractable analytical solutions. Thus our method allows for the construction of analytical solutions for previously unsolved ordinary differential equations. It also highlights the connection between invariant manifolds and eigenfunctions in nonlinear ordinary differential equations and presents avenues for extending this method to solve more nonlinear systems., Comment: 28 pages, 9 figures

We leverage data-driven model discovery methods to determine the governing equations for the emergent behavior of heterogeneous networked dynamical systems. Specifically, we consider networks of coupled nonlinear oscillators whose collective behaviour approaches a limit cycle. Stable limit-cycles are of interest in many biological applications as they model self-sustained oscillations (e.g. heart beats, chemical oscillations, neurons firing, circadian rhythm). For systems that display relaxation oscillations, our method automatically detects boundary (time) layer structures in the dynamics, fitting inner and outer solutions and matching them in a data-driven manner. We demonstrate the method on well-studied systems: the Rayleigh Oscillator and the Van der Pol Oscillator. We then apply the mathematical framework to discovering low-dimensional dynamics in networks of semi-synchronized Kuramoto, Rayleigh, Rossler, and Fitzhugh-Nagumo oscillators, as well as heterogeneous combinations thereof. We also provide a numerical exploration of the dimension of collective network dynamics as a function of several network parameters, showing that the discovery of coarse-grained variables and dynamics can be accomplished with the proposed architecture.

The single, double, and triple pendulum has served as an illustrative experimental benchmark system for scientists to study dynamical behavior for more than four centuries. The pendulum system exhibits a wide range of interesting behaviors, from simple harmonic motion in the single pendulum to chaotic dynamics in multi-arm pendulums. Under forcing, even the single pendulum may exhibit chaos, providing a simple example of a damped-driven system. All multi-armed pendulums are characterized by the existence of index-one saddle points, which mediate the transport of trajectories in the system, providing a simple mechanical analog of various complex transport phenomena, from biolocomotion to transport within the solar system. Further, pendulum systems have long been used to design and test both linear and nonlinear control strategies, with the addition of more arms making the problem more challenging. In this work, we provide extensive designs for the construction and operation of a high-performance, multi-link pendulum on a cart system. Although many experimental setups have been built to study the behavior of pendulum systems, such an extensive documentation on the design, construction, and operation is missing from the literature. The resulting experimental system is highly flexible, enabling a wide range of benchmark problems in dynamical systems modeling, system identification and learning, and control. To promote reproducible research, we have made our entire system open-source, including 3D CAD drawings, basic tutorial code, and data. Moreover, we discuss the possibility of extending our system capability to be operated remotely to enable researchers all around the world to use it, thus increasing access., Comment: 72 pages, 39 figures, 9 tables

We consider the dynamics of a droplet on a vibrating fluid bath. This hydrodynamic quantum analog system is shown to elicit the canonical behavior of damped-driven systems, including a period doubling route to chaos. By approximating the system as a compositional map between the gain and loss dynamics, the underlying nonlinear dynamics can be shown to be driven by energy balances in the systems. The gain-loss iterative mapping is similar to a normal form encoding for the pattern forming instabilities generated in such spatially-extended system. Similar to mode-locked lasers and rotating detonation engines, the underlying bifurcations persist for general forms of the loss and gain, both of which admit explicit representations in our approximation. Moreover, the resulting geometrical description of the particle-wave interaction completely characterizes the instabilities observed in experiments., Comment: 20 pages, 7 figures

Physics-based and first-principles models pervade the engineering and physical sciences, allowing for the ability to model the dynamics of complex systems with a prescribed accuracy. The approximations used in deriving governing equations often result in discrepancies between the model and sensor-based measurements of the system, revealing the approximate nature of the equations and/or the signal-to-noise ratio of the sensor itself. In modern dynamical systems, such discrepancies between model and measurement can lead to poor quantification, often undermining the ability to produce accurate and precise control algorithms. We introduce a discrepancy modeling framework to identify the missing physics and resolve the model-measurement mismatch with two distinct approaches: (i) by learning a model for the evolution of systematic state-space residual, and (ii) by discovering a model for the deterministic dynamical error. Regardless of approach, a common suite of data-driven model discovery methods can be used. The choice of method depends on one's intent (e.g., mechanistic interpretability) for discrepancy modeling, sensor measurement characteristics (e.g., quantity, quality, resolution), and constraints imposed by practical applications (e.g., modeling approaches using the suite of data-driven modeling methods on three continuous dynamical systems under varying signal-to-noise ratios. Finally, we emphasize structural shortcomings of each discrepancy modeling approach depending on error type. In summary, if the true dynamics are unknown (i.e., an imperfect model), one should learn a discrepancy model of the missing physics in the dynamical space. Yet, if the true dynamics are known yet model-measurement mismatch still exists, one should learn a discrepancy model in the state space., Comment: 27 pages, 13 figures

Dynamical systems have a wide range of applications in mechanics, electrical engineering, chemistry, and so on. In this work, we propose the adaptive spectral Koopman (ASK) method to solve nonlinear autonomous dynamical systems. This novel numerical method leverages the spectral-collocation (i.e., pseudo-spectral) method and properties of the Koopman operator to obtain the solution of a dynamical system. Specifically, this solution is represented as a linear combination of the multiplication of Koopman operator's eigenfunctions and eigenvalues, and these eigenpairs are approximated by the spectral method. Unlike conventional time evolution algorithms such as Euler's scheme and the Runge-Kutta scheme, ASK is mesh-free, and hence is more flexible when evaluating the solution. Numerical experiments demonstrate high accuracy of ASK for solving one-, two- and three-dimensional dynamical systems., Comment: 31 pages, 13 figures

Sensor placement is an important and ubiquitous problem across the engineering and physical sciences for tasks such as reconstruction, forecasting and control. Surprisingly, there are few principled mathematical techniques developed to date for optimizing sensor locations, with the leading sensor placement algorithms often based upon the discovery of linear, low-rank sub-spaces and the QR algorithm. QR is a computationally efficient greedy search algorithm which selects sensor locations from candidate positions with maximal variance exhibited in a training data set. More recently, neural networks, specifically shallow decoder networks (SDNs), have been shown to be very successful in mapping sensor measurements to the original high-dimensional state space. SDNs outperform linear subspace representations in reconstruction accuracy, noise tolerance, and robustness to sensor locations. However, SDNs lack principled mathematical techniques for determining sensor placement. In this work, we develop two algorithms for optimizing sensor locations for use with SDNs: one which is a linear selection algorithm based upon QR (Q-SDN), and one which is a nonlinear selection algorithm based upon neural network pruning (P-SDN). Such sensor placement algorithms promise to enhance the already impressive reconstruction capabilities of SDNs. We demonstrate our sensor selection algorithms on two example data sets from fluid dynamics. Moreover, we provide a detailed comparison between our linear (Q-SDN) and nonlinear (P-SDN) algorithms with traditional linear embedding techniques (proper orthogonal decomposition) and QR greedy selection. We show that QR selection with SDNs enhances performance. QR even out-performs our nonlinear selection method that uses magnitude-based pruning. Thus, the combination of a greedy linear selection (QR) with nonlinear encoding (SDN) provides a synergistic combination., Comment: 10 pages, 8 figures, to be submitted to IEEE Sensors Journal

A central challenge in data-driven model discovery is the presence of hidden, or latent, variables that are not directly measured but are dynamically important. Takens' theorem provides conditions for when it is possible to augment these partial measurements with time delayed information, resulting in an attractor that is diffeomorphic to that of the original full-state system. However, the coordinate transformation back to the original attractor is typically unknown, and learning the dynamics in the embedding space has remained an open challenge for decades. Here, we design a custom deep autoencoder network to learn a coordinate transformation from the delay embedded space into a new space where it is possible to represent the dynamics in a sparse, closed form. We demonstrate this approach on the Lorenz, R\"ossler, and Lotka-Volterra systems, learning dynamics from a single measurement variable. As a challenging example, we learn a Lorenz analogue from a single scalar variable extracted from a video of a chaotic waterwheel experiment. The resulting modeling framework combines deep learning to uncover effective coordinates and the sparse identification of nonlinear dynamics (SINDy) for interpretable modeling. Thus, we show that it is possible to simultaneously learn a closed-form model and the associated coordinate system for partially observed dynamics.

In this work, we demonstrate how physical principles -- such as symmetries, invariances, and conservation laws -- can be integrated into the dynamic mode decomposition (DMD). DMD is a widely-used data analysis technique that extracts low-rank modal structures and dynamics from high-dimensional measurements. However, DMD frequently produces models that are sensitive to noise, fail to generalize outside the training data, and violate basic physical laws. Our physics-informed DMD (piDMD) optimization, which may be formulated as a Procrustes problem, restricts the family of admissible models to a matrix manifold that respects the physical structure of the system. We focus on five fundamental physical principles -- conservation, self-adjointness, localization, causality, and shift-invariance -- and derive several closed-form solutions and efficient algorithms for the corresponding piDMD optimizations. With fewer degrees of freedom, piDMD models are less prone to overfitting, require less training data, and are often less computationally expensive to build than standard DMD models. We demonstrate piDMD on a range of challenging problems in the physical sciences, including energy-preserving fluid flow, travelling-wave systems, the Schr\"odinger equation, solute advection-diffusion, a system with causal dynamics, and three-dimensional transitional channel flow. In each case, piDMD significantly outperforms standard DMD in metrics such as spectral identification, state prediction, and estimation of optimal forcings and responses., Comment: 36 pages, 9 figures

Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from subsets of limited and noisy data. The aggregate model statistics are then used to produce inclusion probabilities of the candidate functions, which enables uncertainty quantification and probabilistic forecasts. We apply this ensemble-SINDy (E-SINDy) algorithm to several synthetic and real-world data sets and demonstrate substantial improvements to the accuracy and robustness of model discovery from extremely noisy and limited data. For example, E-SINDy uncovers partial differential equations models from data with more than twice as much measurement noise as has been previously reported. Similarly, E-SINDy learns the Lotka Volterra dynamics from remarkably limited data of yearly lynx and hare pelts collected from 1900-1920. E-SINDy is computationally efficient, with similar scaling as standard SINDy. Finally, we show that ensemble statistics from E-SINDy can be exploited for active learning and improved model predictive control.

Many dynamical systems of interest are nonlinear, with examples in turbulence, epidemiology, neuroscience, and finance, making them difficult to control using linear approaches. Model predictive control (MPC) is a powerful model-based optimization technique that enables the control of such nonlinear systems with constraints. However, modern systems often lack computationally tractable models, motivating the use of system identification techniques to learn accurate and efficient models for real-time control. In this tutorial article, we review emerging data-driven methods for model discovery and how they are used for nonlinear MPC. In particular, we focus on the sparse identification of nonlinear dynamics (SINDy) algorithm and show how it may be used with MPC on an infectious disease control example. We compare the performance against MPC based on a linear dynamic mode decomposition (DMD) model. Code is provided to run the tutorial examples and may be modified to extend this data-driven control framework to arbitrary nonlinear systems.

Dynamic mode decomposition (DMD) provides a regression framework for adaptively learning a best-fit linear dynamics model over snapshots of temporal, or spatio-temporal, data. A diversity of regression techniques have been developed for producing the linear model approximation whose solutions are exponentials in time. For spatio-temporal data, DMD provides low-rank and interpretable models in the form of dominant modal structures along with their exponential/oscillatory behavior in time. The majority of DMD algorithms, however, are prone to bias errors from noisy measurements of the dynamics, leading to poor model fits and unstable forecasting capabilities. The optimized DMD algorithm minimizes the model bias with a variable projection optimization, thus leading to stabilized forecasting capabilities. Here, the optimized DMD algorithm is improved by using statistical bagging methods whereby a single set of snapshots is used to produce an ensemble of optimized DMD models. The outputs of these models are averaged to produce a bagging, optimized dynamic mode decomposition (BOP-DMD). BOP-DMD not only improves performance, it also robustifies the model and provides both spatial and temporal uncertainty quantification (UQ). Thus unlike currently available DMD algorithms, BOP-DMD provides a stable and robust model for probabilistic, or Bayesian forecasting with comprehensive UQ metrics., Comment: 12 pages, 8 figures, 2 algorithms

We propose a probabilistic model discovery method for identifying ordinary differential equations (ODEs) governing the dynamics of observed multivariate data. Our method is based on the sparse identification of nonlinear dynamics (SINDy) framework, in which target ODE models are expressed as a sparse linear combinations of pre-specified candidate functions. Promoting parsimony through sparsity in SINDy leads to interpretable models that generalize to unknown data. Instead of targeting point estimates of the SINDy (linear combination) coefficients, in this work we estimate these coefficients via sparse Bayesian inference. The resulting method, uncertainty quantification SINDy (UQ-SINDy), quantifies not only the uncertainty in the values of the SINDy coefficients due to observation errors and limited data, but also the probability of inclusion of each candidate function in the linear combination. UQ-SINDy promotes robustness against observation noise and limited data, interpretability (in terms of model selection and inclusion probabilities), and generalization capacity for out-of-sample forecast. Sparse inference for UQ-SINDy employs Markov Chain Monte Carlo, and we explore two sparsifying priors: the spike-and-slab prior, and the regularized horseshoe prior. We apply UQ-SINDy to synthetic nonlinear data sets from a Lotka-Volterra model and a nonlinear oscillator, and to a real-world data set of lynx and hare populations. We find that UQ-SINDy is able to discover accurate and meaningful models even in the presence of noise and limited data samples., Comment: 22 pages, 8 figures, 4 tables

Complex systems manifest a small number of instabilities and bifurcations that are canonical in nature, resulting in universal pattern forming characteristics as a function of some parametric dependence. Such parametric instabilities are mathematically characterized by their universal un-foldings, or normal form dynamics, whereby a parsimonious model can be used to represent the dynamics. Although center manifold theory guarantees the existence of such low-dimensional normal forms, finding them has remained a long standing challenge. In this work, we introduce deep learning autoencoders to discover coordinate transformations that capture the underlying parametric dependence of a dynamical system in terms of its canonical normal form, allowing for a simple representation of the parametric dependence and bifurcation structure. The autoencoder constrains the latent variable to adhere to a given normal form, thus allowing it to learn the appropriate coordinate transformation. We demonstrate the method on a number of example problems, showing that it can capture a diverse set of normal forms associated with Hopf, pitchfork, transcritical and/or saddle node bifurcations. This method shows how normal forms can be leveraged as canonical and universal building blocks in deep learning approaches for model discovery and reduced-order modeling., Comment: 18 pages, 7 figures

Despite many of the most common chaotic dynamical systems being continuous in time, it is through discrete time mappings that much of the understanding of chaos is formed. Henri Poincar\'e first made this connection by tracking consecutive iterations of the continuous flow with a lower-dimensional, transverse subspace. The mapping that iterates the dynamics through consecutive intersections of the flow with the subspace is now referred to as a Poincar\'e map, and it is the primary method available for interpreting and classifying chaotic dynamics. Unfortunately, in all but the simplest systems, an explicit form for such a mapping remains outstanding. This work proposes a method for obtaining explicit Poincar\'e mappings by using deep learning to construct an invertible coordinate transformation into a conjugate representation where the dynamics are governed by a relatively simple chaotic mapping. The invertible change of variable is based on an autoencoder, which allows for dimensionality reduction, and has the advantage of classifying chaotic systems using the equivalence relation of topological conjugacies. Indeed, the enforcement of topological conjugacies is the critical neural network regularization for learning the coordinate and dynamics pairing. We provide expository applications of the method to low-dimensional systems such as the R\"ossler and Lorenz systems, while also demonstrating the utility of the method on infinite-dimensional systems, such as the Kuramoto--Sivashinsky equation.

The field of dynamical systems is being transformed by the mathematical tools and algorithms emerging from modern computing and data science. First-principles derivations and asymptotic reductions are giving way to data-driven approaches that formulate models in operator theoretic or probabilistic frameworks. Koopman spectral theory has emerged as a dominant perspective over the past decade, in which nonlinear dynamics are represented in terms of an infinite-dimensional linear operator acting on the space of all possible measurement functions of the system. This linear representation of nonlinear dynamics has tremendous potential to enable the prediction, estimation, and control of nonlinear systems with standard textbook methods developed for linear systems. However, obtaining finite-dimensional coordinate systems and embeddings in which the dynamics appear approximately linear remains a central open challenge. The success of Koopman analysis is due primarily to three key factors: 1) there exists rigorous theory connecting it to classical geometric approaches for dynamical systems, 2) the approach is formulated in terms of measurements, making it ideal for leveraging big-data and machine learning techniques, and 3) simple, yet powerful numerical algorithms, such as the dynamic mode decomposition (DMD), have been developed and extended to reduce Koopman theory to practice in real-world applications. In this review, we provide an overview of modern Koopman operator theory, describing recent theoretical and algorithmic developments and highlighting these methods with a diverse range of applications. We also discuss key advances and challenges in the rapidly growing field of machine learning that are likely to drive future developments and significantly transform the theoretical landscape of dynamical systems., Comment: 110 pages, 27 figures

Time-delay embeddings and dimensionality reduction are powerful techniques for discovering effective coordinate systems to represent the dynamics of physical systems. Recently, it has been shown that models identified by dynamic mode decomposition (DMD) on time-delay coordinates provide linear representations of strongly nonlinear systems, in the so-called Hankel alternative view of Koopman (HAVOK) approach. Curiously, the resulting linear model has a matrix representation that is approximately antisymmetric and tridiagonal with a zero diagonal; for chaotic systems, there is an additional forcing term in the last component. In this paper, we establish a new theoretical connection between HAVOK and the Frenet-Serret frame from differential geometry, and also develop an improved algorithm to identify more stable and accurate models from less data. In particular, we show that the sub- and super-diagonal entries of the linear model correspond to the intrinsic curvatures in Frenet-Serret frame. Based on this connection, we modify the algorithm to promote this antisymmetric structure, even in the noisy, low-data limit. We demonstrate this improved modeling procedure on data from several nonlinear synthetic and real-world examples., Comment: 34 pages, 9 figures

Periodic orbits are among the simplest non-equilibrium solutions to dynamical systems, and they play a significant role in our modern understanding of the rich structures observed in many systems. For example, it is known that embedded within any chaotic attractor are infinitely many unstable periodic orbits (UPOs) and so a chaotic trajectory can be thought of as `jumping' from one UPO to another in a seemingly unpredictable manner. A number of studies have sought to exploit the existence of these UPOs to control a chaotic system. These methods rely on introducing small, precise parameter manipulations each time the trajectory crosses a transverse section to the flow. Typically these methods suffer from the fact that they require a precise description of the Poincar\'e mapping for the flow, which is a difficult task since there is no systematic way of producing such a mapping associated to a given system. Here we employ recent model discovery methods for producing accurate and parsimonious parameter-dependent Poincar\'e mappings to stabilize UPOs in nonlinear dynamical systems. Specifically, we use the sparse identification of nonlinear dynamics (SINDy) method to frame model discovery as a sparse regression problem, which can be implemented in a computationally efficient manner. This approach provides an explicit Poincar\'e mapping that faithfully describes the dynamics of the flow in the Poincar\'e section and can be used to identify UPOs. For each UPO, we then determine the parameter manipulations that stabilize this orbit. The utility of these methods are demonstrated on a variety of differential equations, including the R\"ossler system in a chaotic parameter regime., Comment: Minor edits after review process. Current version is accepted to IEEE Access

The sparse identification of nonlinear dynamics (SINDy) is a regression framework for the discovery of parsimonious dynamic models and governing equations from time-series data. As with all system identification methods, noisy measurements compromise the accuracy and robustness of the model discovery procedure. In this work, we develop a variant of the SINDy algorithm that integrates automatic differentiation and recent time-stepping constrained motivated by Rudy et al. for simultaneously (i) denoising the data, (ii) learning and parametrizing the noise probability distribution, and (iii) identifying the underlying parsimonious dynamical system responsible for generating the time-series data. Thus within an integrated optimization framework, noise can be separated from signal, resulting in an architecture that is approximately twice as robust to noise as state-of-the-art methods, handling as much as 40% noise on a given time-series signal and explicitly parametrizing the noise probability distribution. We demonstrate this approach on several numerical examples, from Lotka-Volterra models to the spatio-temporal Lorenz 96 model. Further, we show the method can identify a diversity of probability distributions including Gaussian, uniform, Gamma, and Rayleigh., Comment: 30 pages, 16 figures, 3 tables

Computing derivatives of noisy measurement data is ubiquitous in the physical, engineering, and biological sciences, and it is often a critical step in developing dynamic models or designing control. Unfortunately, the mathematical formulation of numerical differentiation is typically ill-posed, and researchers often resort to an \textit{ad hoc} process for choosing one of many computational methods and its parameters. In this work, we take a principled approach and propose a multi-objective optimization framework for choosing parameters that minimize a loss function to balance the faithfulness and smoothness of the derivative estimate. Our framework has three significant advantages. First, the task of selecting multiple parameters is reduced to choosing a single hyper-parameter. Second, where ground-truth data is unknown, we provide a heuristic for automatically selecting this hyper-parameter based on the power spectrum and temporal resolution of the data. Third, the optimal value of the hyper-parameter is consistent across different differentiation methods, thus our approach unifies vastly different numerical differentiation methods and facilitates unbiased comparison of their results. Finally, we provide an extensive open-source Python library \texttt{pynumdiff} to facilitate easy application to diverse datasets (https://github.com/florisvb/PyNumDiff)., Comment: 14 pages, 7 figures

The direct monitoring of a rotating detonation engine (RDE) combustion chamber has enabled the observation of combustion front dynamics that are composed of a number of co- and/or counter-rotating coherent traveling shock waves whose nonlinear mode-locking behavior exhibit bifurcations and instabilities which are not well understood. Computational fluid dynamics simulations are ubiquitous in characterizing the dynamics of RDE's reactive, compressible flow. Such simulations are prohibitively expensive when considering multiple engine geometries, different operating conditions, and the long-time dynamics of the mode-locking interactions. Reduced-order models (ROMs) provide a critically enabling simulation framework because they exploit low-rank structure in the data to minimize computational cost and allow for rapid parameterized studies and long-time simulations. However, ROMs are inherently limited by translational invariances manifest by the combustion waves present in RDEs. In this work, we leverage machine learning algorithms to discover moving coordinate frames into which the data is shifted, thus overcoming limitations imposed by the underlying translational invariance of the RDE and allowing for the application of traditional dimensionality reduction techniques. We explore a diverse suite of data-driven ROM strategies for characterizing the complex shock wave dynamics and interactions in the RDE. Specifically, we employ the dynamic mode decomposition and a deep Koopman embedding to give new modeling insights and understanding of combustion wave interactions in RDEs., Comment: 17 pages, 12 figures

Inferring causal relations from time series measurements is an ill-posed mathematical problem, where typically an infinite number of potential solutions can reproduce the given data. We explore in depth a strategy to disambiguate between possible underlying causal networks by perturbing the network, where the actuations are either targeted or applied at random. The resulting transient dynamics provide the critical information necessary to infer causality. Two methods are shown to provide accurate causal reconstructions: Granger causality (GC) with perturbations, and our proposed perturbation cascade inference (PCI). Perturbed GC is capable of inferring smaller networks under low coupling strength regimes. Our proposed PCI method demonstrated consistently strong performance in inferring causal relations for small (2-5 node) and large (10-20 node) networks, with both linear and nonlinear dynamics. Thus the ability to apply a large and diverse set of perturbations/actuations to the network is critical for successfully and accurately determining causal relations and disambiguating between various viable networks., Comment: 11 pages, 9 figures

We develop a principled mathematical framework for controlling nonlinear, networked dynamical systems. Our method integrates dimensionality reduction, bifurcation theory and emerging model discovery tools to find low-dimensional subspaces where feed-forward control can be used to manipulate a system to a desired outcome. The method leverages the fact that many high-dimensional networked systems have many fixed points, allowing for the computation of control signals that will move the system between any pair of fixed points. The sparse identification of nonlinear dynamics (SINDy) algorithm is used to fit a nonlinear dynamical system to the evolution on the dominant, low-rank subspace. This then allows us to use bifurcation theory to find collections of constant control signals that will produce the desired objective path for a prescribed outcome. Specifically, we can destabilize a given fixed point while making the target fixed point an attractor. The discovered control signals can be easily projected back to the original high-dimensional state and control space. We illustrate our nonlinear control procedure on established bistable, low-dimensional biological systems, showing how control signals are found that generate switches between the fixed points. We then demonstrate our control procedure for high-dimensional systems on random high-dimensional networks and Hopfield memory networks.

Multiscale phenomena that evolve on multiple distinct timescales are prevalent throughout the sciences. It is often the case that the governing equations of the persistent and approximately periodic fast scales are prescribed, while the emergent slow scale evolution is unknown. Yet the course-grained, slow scale dynamics is often of greatest interest in practice. In this work we present an accurate and efficient method for extracting the slow timescale dynamics from signals exhibiting multiple timescales that are amenable to averaging. The method relies on tracking the signal at evenly-spaced intervals with length given by the period of the fast timescale, which is discovered using clustering techniques in conjunction with the dynamic mode decomposition. Sparse regression techniques are then used to discover a mapping which describes iterations from one data point to the next. We show that for sufficiently disparate timescales this discovered mapping can be used to discover the continuous-time slow dynamics, thus providing a novel tool for extracting dynamics on multiple timescales.

PySINDy is a Python package for the discovery of governing dynamical systems models from data. In particular, PySINDy provides tools for applying the sparse identification of nonlinear dynamics (SINDy) (Brunton et al. 2016) approach to model discovery. In this work we provide a brief description of the mathematical underpinnings of SINDy, an overview and demonstration of the features implemented in PySINDy (with code examples), practical advice for users, and a list of potential extensions to PySINDy. Software is available at https://github.com/dynamicslab/pysindy.

Recent whole-brain calcium imaging recordings of the nematode C. elegans have demonstrated that neural activity is dominated by dynamics on a low-dimensional manifold that can be clustered according to behavioral states. Despite progress in modeling the dynamics with linear or locally linear models, it remains unclear how a single network of neurons can produce the observed features. In particular, there are multiple clusters, or fixed points, observed in the data which cannot be characterized by a single linear model. We propose a nonlinear control model which is global and parameterized by only four free parameters that match the features displayed by the low-dimensional C. elegans neural activity. In addition to reproducing the average probability distribution of the data, long and short time-scale changes in transition statistics can be characterized via changes in a single parameter. Some of these macro-scale transitions have experimental correlates to single neuro-modulators that seem to act as biological controls, allowing this model to generate testable hypotheses about the effect of these neuro-modulators on the global dynamics. The theory provides an elegant characterization of the neuron population dynamics in C. elegans. Moreover, the mathematical structure of the nonlinear control framework provides a paradigm that can be generalized to more complex systems with an arbitrary number of behavioral states.

We develop a deep autoencoder architecture that can be used to find a coordinate transformation which turns a nonlinear PDE into a linear PDE. Our architecture is motivated by the linearizing transformations provided by the Cole-Hopf transform for Burgers equation and the inverse scattering transform for completely integrable PDEs. By leveraging a residual network architecture, a near-identity transformation can be exploited to encode intrinsic coordinates in which the dynamics are linear. The resulting dynamics are given by a Koopman operator matrix $\mathbf{K}$. The decoder allows us to transform back to the original coordinates as well. Multiple time step prediction can be performed by repeated multiplication by the matrix $\mathbf{K}$ in the intrinsic coordinates. We demonstrate our method on a number of examples, including the heat equation and Burgers equation, as well as the substantially more challenging Kuramoto-Sivashinsky equation, showing that our method provides a robust architecture for discovering interpretable, linearizing transforms for nonlinear PDEs., Comment: 23 pages, 18 figures

We develop an unsupervised machine learning algorithm for the automated discovery and identification of traveling waves in spatio-temporal systems governed by partial differential equations (PDEs). Our method uses sparse regression and subspace clustering to robustly identify translational invariances that can be leveraged to build improved reduced order models (ROMs). Invariances, whether translational or rotational, are well known to compromise the ability of ROMs to produce accurate and/or low-rank representations of the spatio-temporal dynamics. However, by discovering translations in a principled way, data can be shifted into a coordinate systems where quality, low-dimensional ROMs can be constructed. This approach can be used on either numerical or experimental data with or without knowledge of the governing equations. We demonstrate our method on a variety of PDEs of increasing difficulty, taken from the field of fluid dynamics, showing the efficacy and robustness of the proposed approach., Comment: 14 pages, 8 figures

Poincar\'e maps are an integral aspect to our understanding and analysis of nonlinear dynamical systems. Despite this fact, the construction of these maps remains elusive and is primarily left to simple motivating examples. In this manuscript we propose a method of data-driven discovery of Poincar\'e maps based upon sparse regression techniques, specifically the sparse identification of nonlinear dynamics (SINDy) algorithm. This work can be used to determine the dynamics on and near invariant manifolds of a given dynamical system, as well as provide long-time forecasting of the coarse-grained dynamics of multiscale systems. Moreover, the method provides a mathematical formalism for determining nonlinear Floquet theory for the stability of nonlinear periodic orbits. The methods are applied to a range of examples including both ordinary and partial differential equations that exhibit periodic, quasi-periodic, and chaotic behavior.

Dynamic mode decomposition (DMD) is a data-driven method that models high-dimensional time series as a sum of spatiotemporal modes, where the temporal modes are constrained by linear dynamics. For nonlinear dynamical systems exhibiting strongly coherent structures, DMD can be a useful approximation to extract dominant, interpretable modes. In many domains with large spatiotemporal data---including fluid dynamics, video processing, and finance---the dynamics of interest are often perturbations about fixed points or equilibria, which motivates the application of DMD to centered (i.e. mean-subtracted) data. In this work, we show that DMD with centered data is equivalent to incorporating an affine term in the dynamic model and is not equivalent to computing a discrete Fourier transform. Importantly, DMD with centering can always be used to compute eigenvalue spectra of the dynamics. However, in many cases DMD without centering cannot model the corresponding dynamics, most notably if the dynamics have full effective rank. Additionally, we generalize the notion of centering to extracting arbitrary, but known, fixed frequencies from the data. We corroborate these theoretical results numerically on three nonlinear examples: the Lorenz system, a surveillance video, and brain recordings. Since centering the data is simple and computationally efficient, we recommend it as a preprocessing step before DMD; furthermore, we suggest that it can be readily used in conjunction with many other popular implementations of the DMD algorithm.

The Koopman and Perron Frobenius transport operators are fundamentally changing how we approach dynamical systems, providing linear representations for even strongly nonlinear dynamics. Although there is tremendous potential benefit of such a linear representation for estimation and control, transport operators are infinite-dimensional, making them difficult to work with numerically. Obtaining low-dimensional matrix approximations of these operators is paramount for applications, and the dynamic mode decomposition has quickly become a standard numerical algorithm to approximate the Koopman operator. Related methods have seen rapid development, due to a combination of an increasing abundance of data and the extensibility of DMD based on its simple framing in terms of linear algebra. In this chapter, we review key innovations in the data-driven characterization of transport operators for control, providing a high-level and unified perspective. We emphasize important recent developments around sparsity and control, and discuss emerging methods in big data and machine learning., Comment: 37 pages, 4 figures

Optimal sensor and actuator selection is a central challenge in high-dimensional estimation and control. Nearly all subsequent control decisions are affected by these sensor/actuator locations, and optimal placement amounts to an intractable brute-force search among the combinatorial possibilities. In this work, we exploit balanced model reduction and greedy optimization to efficiently determine sensor and actuator selections that optimize observability and controllability. In particular, we determine locations that optimize scalar measures of observability and controllability via greedy matrix QR pivoting on the dominant modes of the direct and adjoint balancing transformations. Pivoting runtime scales linearly with the state dimension, making this method tractable for high-dimensional systems. The results are demonstrated on the linearized Ginzburg-Landau system, for which our algorithm approximates known optimal placements computed using costly gradient descent methods., Comment: 8 pages, 6 figures

Conserved quantities, i.e. constants of motion, are critical for characterizing many dynamical systems in science and engineering. These quantities are related to underlying symmetries and they provide fundamental knowledge about physical laws, describe the evolution of the system, and enable system reduction. In this work, we formulate a data-driven architecture for discovering conserved quantities based on Koopman theory. The Koopman operator has emerged as a principled linear embedding of nonlinear dynamics, and its eigenfunctions establish intrinsic coordinates along which the dynamics behave linearly. Interestingly, eigenfunctions of the Koopman operator associated with vanishing eigenvalues correspond to conserved quantities of the underlying system. In this paper, we show that these invariants may be identified with data-driven regression and power series expansions, based on the infinitesimal generator of the Koopman operator. We further establish a connection between the Koopman framework, conserved quantities, and the Lie-Poisson bracket. This data-driven method for discovering conserved quantities is demonstrated on the three-dimensional rigid body equations, where we simultaneously discover the total energy and angular momentum and use these intrinsic coordinates to develop a model predictive controller to track a given reference value., Comment: 7 pages, 2 figures, 57th IEEE Conference on Decision and Control (CDC 2018)

Nonlinear dynamical systems are ubiquitous in science and engineering, yet analysis and prediction of these systems remains a challenge. Koopman operator theory circumvents some of these issues by considering the dynamics in the space of observable functions on the state, in which the dynamics are intrinsically linear and thus amenable to standard techniques from numerical analysis and linear algebra. However, practical issues remain with this approach, as the space of observables is infinite-dimensional and selecting a subspace of functions in which to accurately represent the system is a nontrivial task. In this work we consider time-delay observables to represent nonlinear dynamics in the Koopman operator framework. We prove the surprising result that Koopman operators for different systems admit universal (system-independent) representations in these coordinates, and give analytic expressions for these representations. In addition, we show that for certain systems a restricted class of these observables form an optimal finite-dimensional basis for representing the Koopman operator, and that the analytic representation of the Koopman operator in these coordinates coincides with results computed by the dynamic mode decomposition. We provide numerical examples to complement our results. In addition to being theoretically interesting, these results have implications for a number of linearization algorithms for dynamical systems., Comment: 28 pages, 6 figures

Hybrid systems are traditionally difficult to identify and analyze using classical dynamical systems theory. Moreover, recently developed model identification methodologies largely focus on identifying a single set of governing equations solely from measurement data. In this article, we develop a new methodology, Hybrid-Sparse Identification of Nonlinear Dynamics (Hybrid-SINDy), which identifies separate nonlinear dynamical regimes, employs information theory to manage uncertainty, and characterizes switching behavior. Specifically, we utilize the nonlinear geometry of data collected from a complex system to construct a set of coordinates based on measurement data and augmented variables. Clustering the data in these measurement-based coordinates enables the identification of nonlinear hybrid systems. This methodology broadly empowers nonlinear system identification without constraining the data locally in time and has direct connections to hybrid systems theory. We demonstrate the success of this method on numerical examples including a mass-spring hopping model and an infectious disease model. Characterizing complex systems that switch between dynamic behaviors is integral to overcoming modern challenges such as eradication of infectious diseases, the design of efficient legged robots, and the protection of cyber infrastructures., Comment: 22 pages, 5 figures, 1 table. Code repository: https://github.com/niallmm/Hybrid-SINDy

A major challenge in the study of dynamical systems is that of model discovery: turning data into models that are not just predictive, but provide insight into the nature of the underlying dynamical system that generated the data. This problem is made more difficult by the fact that many systems of interest exhibit diverse behaviors across multiple time scales. We introduce a number of data-driven strategies for discovering nonlinear multiscale dynamical systems and their embeddings from data. We consider two canonical cases: (i) systems for which we have full measurements of the governing variables, and (ii) systems for which we have incomplete measurements. For systems with full state measurements, we show that the recent sparse identification of nonlinear dynamical systems (SINDy) method can discover governing equations with relatively little data and introduce a sampling method that allows SINDy to scale efficiently to problems with multiple time scales. Specifically, we can discover distinct governing equations at slow and fast scales. For systems with incomplete observations, we show that the Hankel alternative view of Koopman (HAVOK) method, based on time-delay embedding coordinates, can be used to obtain a linear model and Koopman invariant measurement system that nearly perfectly captures the dynamics of nonlinear quasiperiodic systems. We introduce two strategies for using HAVOK on systems with multiple time scales. Together, our approaches provide a suite of mathematical strategies for reducing the data required to discover and model nonlinear multiscale systems., Comment: 21 pages, 6 figures