Search

Machine-learning technologies for learning dynamical systems from data play an important role in engineering design. This research focuses on learning continuous linear models from data. Stability, a key feature of dynamic systems, is especially important in design tasks such as prediction and control. Thus, there is a need to develop methodologies that provide stability guarantees. To that end, we leverage the parameterization of stable matrices proposed in [Gillis/Sharma, Automatica, 2017] to realize the desired models. Furthermore, to avoid the estimation of derivative information to learn continuous systems, we formulate the inference problem in an integral form. We also discuss a few extensions, including those related to control systems. Numerical experiments show that the combination of a stable matrix parameterization and an integral form of differential equations allows us to learn stable systems without requiring derivative information, which can be challenging to obtain in situations with noisy or limited data.

In this work, we investigate a model order reduction scheme for high-fidelity nonlinear structured parametric dynamical systems. More specifically, we consider a class of nonlinear dynamical systems whose nonlinear terms are polynomial functions, and the linear part corresponds to a linear structured model, such as second-order, time-delay, or fractional-order systems. Our approach relies on the Volterra series representation of these dynamical systems. Using this representation, we identify the kernels and, thus, the generalized multivariate transfer functions associated with these systems. Consequently, we present results allowing the construction of reduced-order models whose generalized transfer functions interpolate these of the original system at pre-defined frequency points. For efficient calculations, we also need the concept of a symmetric Kronecker product representation of a tensor and derive particular properties of them. Moreover, we propose an algorithm that extracts dominant subspaces from the prescribed interpolation conditions. This allows the construction of reduced-order models that preserve the structure. We also extend these results to parametric systems and a special case (delay in input/output). We demonstrate the efficiency of the proposed method by means of various numerical benchmarks.

Model order reduction usually consists of two stages: the offline stage and the online stage. The offline stage is the expensive part that sometimes takes hours till the final reduced-order model is derived, especially when the original model is very large or complex. Once the reduced-order model is obtained, the online stage of querying the reduced-order model for simulation is very fast and often real-time capable. This work concerns a strategy to significantly speed up the offline stage of model order reduction for large and complex systems. In particular, it is successful in accelerating the greedy algorithm that is often used in the offline stage for reduced-order model construction. We propose multi-fidelity error estimators and replace the high-fidelity error estimator in the greedy algorithm. Consequently, the computational complexity at each iteration of the greedy algorithm is reduced and the algorithm converges more than 3 times faster without incurring noticeable accuracy loss., Comment: Compared to the first version, only the title is rephrased

The engineering design process often relies on mathematical modeling that can describe the underlying dynamic behavior. In this work, we present a data-driven methodology for modeling the dynamics of nonlinear systems. To simplify this task, we aim to identify a coordinate transformation that allows us to represent the dynamics of nonlinear systems using a common, simple model structure. The advantage of a common simple model is that customized design tools developed for it can be applied to study a large variety of nonlinear systems. The simplest common model -- one can think of -- is linear, but linear systems often fall short in accurately capturing the complex dynamics of nonlinear systems. In this work, we propose using quadratic systems as the common structure, inspired by the lifting principle. According to this principle, smooth nonlinear systems can be expressed as quadratic systems in suitable coordinates without approximation errors. However, finding these coordinates solely from data is challenging. Here, we leverage deep learning to identify such lifted coordinates using only data, enabling a quadratic dynamical system to describe the system's dynamics. Additionally, we discuss the asymptotic stability of these quadratic dynamical systems. We illustrate the approach using data collected from various numerical examples, demonstrating its superior performance with the existing well-known techniques.

Model-order reduction techniques allow the construction of low-dimensional surrogate models that can accelerate engineering design processes. Often, these techniques are intrusive, meaning that they require direct access to underlying high-fidelity models. Accessing these models is laborious or may not even be possible in some cases. Therefore, there is an interest in developing non-intrusive model reduction techniques to construct low-dimensional models directly from simulated or experimental data. In this work, we focus on a recent data-driven methodology, namely operator inference, that aims at inferring the reduced operators using only trajectories of high-fidelity models. We present an extension of operator inference for mechanical systems, preserving the second-order structure. We also study a particular case in which complete information about the external forces is available. In this formulation, the reduced operators having certain properties inspired by the original system matrices are enforced by adding constraints to the optimization problem. We illustrate the presented methodology using three numerical examples.

Linear projection schemes like Proper Orthogonal Decomposition can efficiently reduce the dimensions of dynamical systems but are naturally limited, e.g., for convection-dominated problems. Nonlinear approaches have shown to outperform linear methods in terms of dimension reduction versus accuracy but, typically, come with a large computational overhead. In this work, we consider a quadratic reduction scheme which induces nonlinear structures that are well accessible to tensorized linear algebra routines. We discuss that nonintrusive approaches can be used to simultaneously reduce the complexity in the equations and propose an operator inference formulation that respects dynamics on nonlinear manifolds.

We discuss the feedback control problem for a two-dimensional two-phase Stefan problem. In our approach, we use a sharp interface representation in combination with mesh-movement to track the interface position. To attain a feedback control, we apply the linear-quadratic regulator approach to a suitable linearization of the problem. We address details regarding the discretization and the interface representation therein. Further, we document the matrix assembly to generate a non-autonomous generalized differential Riccati equation. To numerically solve the Riccati equation, we use low-rank factored and matrix-valued versions of the non-autonomous backward differentiation formulas, which incorporate implicit index reduction techniques. For the numerical simulation of the feedback controlled Stefan problem, we use a time-adaptive fractional-step-theta scheme. We provide the implementations for the developed methods and test these in several numerical experiments. With these experiments we show that our feedback control approach is applicable to the Stefan control problem and makes this large-scale problem computable. Also, we discuss the influence of several controller design parameters, such as the choice of inputs and outputs.

In this article, we investigate exponential lag synchronization results for the Cohen-Grossberg neural networks (C-GNNs) with discrete and distributed delays on an arbitrary time domain by applying feedback control. We formulate the problem by using the time scales theory so that the results can be applied to any uniform or non-uniform time domains. Also, we provide a comparison of results that shows that obtained results are unified and generalize the existing results. Mainly, we use the unified matrix-measure theory and Halanay inequality to establish these results. In the last section, we provide two simulated examples for different time domains to show the effectiveness and generality of the obtained analytical results., Comment: 20 pages, 18 figures

The purpose of this work is the development of an artificial neural network (ANN) for surrogate modeling of the mechanical response of viscoplastic grain microstructures. To this end, a U-Net-based convolutional neural network (CNN) is trained to account for the history dependence of the material behavior. The training data take the form of numerical simulation results for the von Mises stress field under quasi-static tensile loading. The trained CNN (tCNN) can accurately reproduce both the average response as well as the local von Mises stress field. The tCNN calculates the von Mises stress field of grain microstructures not included in the training dataset about 500 times faster than its calculation based on the numerical solution with a spectral solver of the corresponding initial-boundary-value problem. The tCNN is also successfully applied to other types of microstructure morphologies (e.g., matrix-inclusion type topologies) and loading levels not contained in the training dataset.

In this paper, we consider the problem of finding surrogate models for large-scale second-order linear time-invariant systems with inhomogeneous initial conditions. For this class of systems, the superposition principle allows us to decompose the system behavior into three independent components. The first behavior corresponds to the transfer between the input and output having zero initial conditions. In contrast, the other two correspond to the transfer between the initial position and the initial velocity conditions having zero input, respectively. Based on this superposition of systems, our goal is to propose model reduction schemes allowing to preserve the second-order structure in the surrogate models. To this aim, we introduce tailored second-order Gramians for each system component and compute them numerically, solving Lyapunov equations. As a consequence, two methodologies are proposed. The first one consists in reducing each of the components independently using a suitable balanced truncation procedure. The sum of these reduced systems provides an approximation of the original system. This methodology allows flexibility on the order of the reduced-order model. The second proposed methodology consists in extracting the dominant subspaces from the sum of Gramians to build the projection matrices leading to a surrogate model. Additionally, we discuss error bounds for the overall output approximation. Finally, the proposed methods are illustrated by means of benchmark problems.

In this paper, we consider the structure-preserving model order reduction problem for multi-input/multi-output bilinear control systems by tangential interpolation. We propose a new type of tangential interpolation problem for structured bilinear systems, for which we develop a new structure-preserving interpolation framework. This new framework extends and generalizes different formulations of tangential interpolation for bilinear systems from the literature and also provides a unifying framework. We then derive explicit conditions on the projection spaces to enforce tangential interpolation in different settings, including conditions for tangential Hermite interpolation. The analysis is illustrated by means of three numerical examples., Comment: 33 pages, 9 figures

Measurement noise is an integral part while collecting data of a physical process. Thus, noise removal is necessary to draw conclusions from these data, and it often becomes essential to construct dynamical models using these data. We discuss a methodology to learn differential equation(s) using noisy and irregular sampled measurements. In our methodology, the main innovation can be seen in the integration of deep neural networks with the neural ordinary differential equations (ODEs) approach. Precisely, we aim at learning a neural network that provides (approximately) an implicit representation of the data and an additional neural network that models the vector fields of the dependent variables. We combine these two networks by constraining using neural ODEs. The proposed framework to learn a model describing the vector field is highly effective under noisy measurements. The approach can handle scenarios where dependent variables are not available at the same temporal grid. Moreover, a particular structure, e.g., second-order with respect to time, can easily be incorporated. We demonstrate the effectiveness of the proposed method for learning models using data obtained from various differential equations and present a comparison with the neural ODE method that does not make any special treatment to noise., Comment: arXiv admin note: substantial text overlap with arXiv:2109.11446

One of the most computationally expensive steps of the low-rank ADI method for large-scale Lyapunov equations is the solution of a shifted linear system at each iteration. We propose the use of the extended Krylov subspace method for this task. In particular, we illustrate how a single approximation space can be constructed to solve all the shifted linear systems needed to achieve a prescribed accuracy in terms of Lyapunov residual norm. Moreover, we show how to fully merge the two iterative procedures in order to obtain a novel, efficient implementation of the low-rank ADI method, for an important class of equations. Many state-of-the-art algorithms for the shift computation can be easily incorporated into our new scheme, as well. Several numerical results illustrate the potential of our novel procedure when compared to an implementation of the low-rank ADI method based on sparse direct solvers for the shifted linear systems.

We devise a spectral divide-and-conquer scheme for matrices that are self-adjoint with respect to a given indefinite scalar product (i.e. pseudosymmetic matrices). The pseudosymmetric structure of the matrix is preserved in the spectral division, such that the method can be applied recursively to achieve full diagonalization. The method is well-suited for structured matrices that come up in computational quantum physics and chemistry. In this application context, additional definiteness properties guarantee a convergence of the matrix sign function iteration within two steps when Zolotarev functions are used. The steps are easily parallelizable. Furthermore, it is shown that the matrix decouples into symmetric definite eigenvalue problems after just one step of spectral division.

An algorithm for constructing a $J$-orthogonal basis of the extended Krylov subspace $\mathcal{K}_{r,s}=\operatorname{range}\{u,Hu, H^2u,$ $ \ldots, $ $H^{2r-1}u, H^{-1}u, H^{-2}u, \ldots, H^{-2s}u\},$ where $H \in \mathbb{R}^{2n \times 2n}$ is a large (and sparse) Hamiltonian matrix is derived (for $r = s+1$ or $r=s$). Surprisingly, this allows for short recurrences involving at most five previously generated basis vectors. Projecting $H$ onto the subspace $\mathcal{K}_{r,s}$ yields a small Hamiltonian matrix. The resulting HEKS algorithm may be used in order to approximate $f(H)u$ where $f$ is a function which maps the Hamiltonian matrix $H$ to, e.g., a (skew-)Hamiltonian or symplectic matrix. Numerical experiments illustrate that approximating $f(H)u$ with the HEKS algorithm is competitive for some functions compared to the use of other (structure-preserving) Krylov subspace methods.

Learning dynamical models from data plays a vital role in engineering design, optimization, and predictions. Building models describing dynamics of complex processes (e.g., weather dynamics, or reactive flows) using empirical knowledge or first principles are onerous or infeasible. Moreover, these models are high-dimensional but spatially correlated. It is, however, observed that the dynamics of high-fidelity models often evolve in low-dimensional manifolds. Furthermore, it is also known that for sufficiently smooth vector fields defining the nonlinear dynamics, a quadratic model can describe it accurately in an appropriate coordinate system, conferring to the McCormick relaxation idea in nonconvex optimization. Here, we aim at finding a low-dimensional embedding of high-fidelity dynamical data, ensuring a simple quadratic model to explain its dynamics. To that aim, this work leverages deep learning to identify low-dimensional quadratic embeddings for high-fidelity dynamical systems. Precisely, we identify the embedding of data using an autoencoder to have the desired property of the embedding. We also embed a Runge-Kutta method to avoid the time-derivative computations, which is often a challenge. We illustrate the ability of the approach by a couple of examples, arising in describing flow dynamics and the oscillatory tubular reactor model.

Algebraic Riccati equations with indefinite quadratic terms play an important role in applications related to robust controller design. While there are many established approaches to solve these in case of small-scale dense coefficients, there is no approach available to compute solutions in the large-scale sparse setting. In this paper, we develop an iterative method to compute low-rank approximations of stabilizing solutions of large-scale sparse continuous-time algebraic Riccati equations with indefinite quadratic terms. We test the developed approach for dense examples in comparison to other established matrix equation solvers, and investigate the applicability and performance in large-scale sparse examples., Comment: 19 pages, 2 figures, 5 tables

We develop a compact, reliable model order reduction approach for fast frequency sweeps in microwave circuits by means of the reduced-basis method. Contrary to what has been previously done, special emphasis is placed on certifying the accuracy of the reduced-order model with respect to the original full-order model in an effective and efficient way. Previous works on model order reduction accuracy certification rely on costly $\textit{a posteriori}$ error estimators, which typically require expensive $\textit{inf-sup}$ constant evaluations of the underlying full-order model. This scenario is often too time-consuming and unaffordable in electromagnetic applications. As a result, less expensive and heuristic error estimators are commonly used instead. Very often, one is interested in knowing about the full state vector, instead of just some output quantities derived from the full state. Therefore, error estimators for the full state vector become relevant. In this work, we detail the frequency behavior of both the electric field and the state error when an approximation to the electric field solution is carried out. Both field quantities share the same frequency behavior. Based on this observation, we focus on the efficient estimation of the electric field state error and propose a fast evaluation of the reduced-order model state error in the frequency band of analysis, minimizing the number of full-order model evaluations. This methodology is of paramount importance to carry out a reliable fast frequency sweep in microwave circuits. Finally, real-life applications will illustrate the capabilities and efficiency of the proposed approach., Comment: 24 pages, 13 Figures, 6 Tables

We identify a relationship between the solutions of a nonsymmetric algebraic T-Riccati equation (T-NARE) and the deflating subspaces of a palindromic matrix pencil, obtained by arranging the coefficients of the T-NARE. The interplay between T-NARE and palindromic pencils allows one to derive both theoretical properties of the solutions of the equation, and new methods for its numerical solution. In particular, we propose methods based on the (palindromic) QZ algorithm and the doubling algorithm, whose effectiveness is demonstrated by several numerical tests

The Poisson-Boltzmann equation (PBE) is an implicit solvent continuum model for calculating the electrostatic potential and energies of ionic solvated biomolecules. However, its numerical solution remains a significant challenge due strong singularities and nonlinearity caused by the singular source terms and the exponential nonlinear terms, respectively. An efficient method for the treatment of singularities in the linear PBE was introduced in \cite{BeKKKS:18}, that is based on the RS tensor decomposition for both electrostatic potential and the discretized Dirac delta distribution. In this paper, we extend this regularization method to the nonlinear PBE. We apply the PBE only to the regular part of the solution corresponding to the modified right-hand side via extraction of the long-range part in the discretized Dirac delta distribution. The total electrostatic potential is obtained by adding the long-range solution to the directly precomputed short-range potential. The main computational benefit of the approach is the automatic maintaining of the continuity in the Cauchy data on the solute-solvent interface. The boundary conditions are also obtained from the long-range component of the precomputed canonical tensor representation of the Newton kernel. In the numerical experiments, we illustrate the accuracy of the nonlinear regularized PBE (NRPBE) over the classical variant., Comment: 26 pages, 15 figures

Measurement noise is an integral part while collecting data of a physical process. Thus, noise removal is a necessary step to draw conclusions from these data, and it often becomes quite essential to construct dynamical models using these data. We discuss a methodology to learn differential equation(s) using noisy and sparsely sampled measurements. In our methodology, the main innovation can be seen in of integration of deep neural networks with a classical numerical integration method. Precisely, we aim at learning a neural network that implicitly represents the data and an additional neural network that models the vector fields of the dependent variables. We combine these two networks by enforcing the constraint that the data at the next time-steps can be given by following a numerical integration scheme such as the fourth-order Runge-Kutta scheme. The proposed framework to learn a model predicting the vector field is highly effective under noisy measurements. The approach can handle scenarios where dependent variables are not available at the same temporal grid. We demonstrate the effectiveness of the proposed method to learning models using data obtained from various differential equations. The proposed approach provides a promising methodology to learn dynamic models, where the first-principle understanding remains opaque.

Given a set of solution snapshots of a hyperbolic PDE, we are interested in learning a reduced order model (ROM). To this end, we propose a novel decompose then learn approach. We decompose the solution by expressing it as a composition of a transformed solution and a de-transformer. Our idea is to learn a ROM for both these objects, which, unlike the solution, are well approximable in a linear reduced space. A ROM for the (untransformed) solution is then recovered via a recomposition. The transformed solution results from composing the solution with a spatial transform that aligns the spatial discontinuities. Furthermore, the de-transformer is the inverse of the spatial transform and lets us recover a ROM for the solution. We consider an image registration technique to compute the spatial transform, and to learn a ROM, we resort to the dynamic mode decomposition (DMD) methodology. Several benchmark problems demonstrate the effectiveness our method in representing the data and as a predictive tool.

To overcome many-query optimization, control, or uncertainty quantification work loads in reliable gas and energy network operations, model order reduction is the mathematical technology of choice. To this end, we enhance the model, solver and reductor components of the "morgen" platform, introduced in Himpe et al [J.~Math.~Ind. 11:13, 2021], and conclude with a mathematically, numerically and computationally favorable model-solver-reductor ensemble.

Mathematical models are essential to analyze and understand the dynamics of complex systems. Recently, data-driven methodologies have got a lot of attention which is leveraged by advancements in sensor technology. However, the quality of obtained data plays a vital role in learning a good and reliable model. Therefore, in this paper, we propose an efficient heuristic methodology to collect data both in the frequency domain and time-domain, aiming at the best possible information gain from limited experimental data. The efficiency of the proposed methodology is illustrated by means of several examples, and also, its robustness in the presence of noisy data is shown.

We propose an adaptive moment-matching framework for model order reduction of quadratic-bilinear descriptor systems. In this framework, an important issue is the selection of those shift frequencies where moment-matching is to be achieved. Often, the choice is random or linked to the linear part of the nonlinear system. In this paper, we extend the use of an existing a posteriori error bound for general linear time invariant systems to quadratic-bilinear systems and develop a greedy-type framework to select a good choice of interpolation points for the construction of the projection matrices. The results are compared with standard quadratic-bilinear projection methods and we observe that the approximations obtained by the proposed method yield high accuracy.

Discovering dynamical models to describe underlying dynamical behavior is essential to draw decisive conclusions and engineering studies, e.g., optimizing a process. Experimental data availability notwithstanding has increased significantly, but interpretable and explainable models in science and engineering yet remain incomprehensible. In this work, we blend machine learning and dictionary-based learning with numerical analysis tools to discover governing differential equations from noisy and sparsely-sampled measurement data. We utilize the fact that given a dictionary containing huge candidate nonlinear functions, dynamical models can often be described by a few appropriately chosen candidates. As a result, we obtain interpretable and parsimonious models which are prone to generalize better beyond the sampling regime. Additionally, we integrate a numerical integration framework with dictionary learning that yields differential equations without requiring or approximating derivative information at any stage. Hence, it is utterly effective in corrupted and sparsely-sampled data. We discuss its extension to governing equations, containing rational nonlinearities that typically appear in biological networks. Moreover, we generalized the method to governing equations that are subject to parameter variations and externally controlled inputs. We demonstrate the efficiency of the method to discover a number of diverse differential equations using noisy measurements, including a model describing neural dynamics, chaotic Lorenz model, Michaelis-Menten Kinetics, and a parameterized Hopf normal form.

We propose a data-driven model order reduction (MOR) technique for parametrized partial differential equations that exhibit parameter-dependent jump-discontinuities. Such problems have poor-approximability in a linear space and therefore, are challenging for standard MOR techniques. We build upon the methodology of approximating the map between the parameter domain and the expansion coefficients of the reduced basis via regression. The online stage queries the regression model for the expansion coefficients and recovers a reduced approximation for the solution. We propose to apply this technique to a transformed solution that results from composing the solution with a spatial transform. Unlike the (untransformed) solution, it is sufficiently regular along the parameter domain and thus, is well-approximable in a low-dimensional linear space. To recover an approximation for the (untransformed) solution, we propose an online efficient regression-based technique that approximates the inverse of the spatial transform. Our method features a decoupled online and offline stage, and benchmark problems involving hyperbolic and parabolic equations demonstrate its effectiveness.

A reliable model order reduction process for parametric analysis in electromagnetics is detailed. Special emphasis is placed on certifying the accuracy of the reduced-order model. For this purpose, a sharp state error estimator is proposed. Standard a posteriori state error estimation for model order reduction relies on the inf-sup constant. For parametric systems, the inf-sup constant is parameter-dependent. The a posteriori error estimation for systems with very small or vanishing inf-sup constant poses a challenge, since it is inversely proportional to the inf-sup constant, resulting in rather useless, overly pessimistic error estimators. Such systems appear in electromagnetics since the inf-sup constant values are close to zero at points close to resonant frequencies, where they eventually vanish. We propose a novel a posteriori state error estimator which avoids the calculation of the inf-sup constant. The proposed state error estimator is compared with the standard error estimator and a recently proposed one in the literature. It is shown that our proposed error estimator outperforms both existing estimators. Numerical experiments are performed on real-life microwave devices such as narrowband and wideband antennas, two types of dielectric resonator filters as well as a dual-mode waveguide filter. These examples show the capabilities and efficiency of the proposed methodology., Comment: 20 pages, 30 Figures, 3 Tables