Search

Current methods achieved reasonable success in solving short-term parametric partial differential equations (PDEs). However, solving long-term PDEs remains challenging, and existing techniques also suffer from low efficiency due to requiring finely-resolved datasets. In this paper, we propose a physicsinformed Fourier neural operator (PIFNO) for parametric PDEs, which incorporates physical knowledge through regularization. The numerical PDE problem is reformulated into an unconstrained optimization task, which we solve by using an enhanced architecture that facilitates longer-term datasets. We compare PIFNO against standard FNO on three benchmark PDEs. Results demonstrate improved long-term performance with PIFNO. Moreover, PIFNO only needs coarse dataset resolution, which enhances computational efficiency. [ABSTRACT FROM AUTHOR]

Numerical simulations of seismic wave propagation in heterogeneous 3-D media are central to investigating subsurface structures and understanding earthquake processes, yet are computationally expensive for large problems. This is particularly problematic for full-waveform inversion (FWI), which typically involves numerous runs of the forward process. In machine learning there has been considerable recent work in the area of operator learning, with a new class of models called neural operators allowing for data-driven solutions to partial differential equations. Recent work in seismology has shown that when neural operators are adequately trained, they can significantly shorten the compute time for wave propagation. However, the memory required for the 3-D time domain equations may be prohibitive. In this study, we show that these limitations can be overcome by solving the wave equations in the frequency domain, also known as the Helmholtz equations, since the solutions for a set of frequencies can be determined in parallel. The 3-D Helmholtz neural operator is 40 times more memory-efficient than an equivalent time-domain version. We use a Helmholtz neural operator for 2-D and 3-D elastic wave modelling, achieving two orders of magnitude acceleration compared to a baseline spectral element method. The neural operator accurately generalizes to variable velocity structures and can be evaluated on denser input meshes than used in the training simulations. We also show that when solving for wavefields strictly at the free surface, the accuracy can be significantly improved via a graph neural operator layer. In leveraging automatic differentiation, the proposed method can serve as an alternative to the adjoint-state approach for 3-D FWI, reducing the computation time by a factor of 350. [ABSTRACT FROM AUTHOR]

Gradient-based meta-learning methods have primarily been applied to classical machine learning tasks such as image classification. Recently, PDE-solving deep learning methods, such as neural operators, are starting to make an important impact on learning and predicting the response of a complex physical system directly from observational data. Taking the material modeling problems for example, the neural operator approach learns a surrogate mapping from the loading field to the corresponding material response field, which can be seen as learning the solution operator of a hidden PDE. The microstructure and mechanical parameters of each material specimen correspond to the (possibly heterogeneous) parameter field in this hidden PDE. Due to the limitation on experimental measurement techniques, the data acquisition for each material specimen is commonly challenging and costly. This fact calls for the utilization and transfer of existing knowledge to new and unseen material specimens, which corresponds to sampling efficient learning of the solution operator of a hidden PDE with a different parameter field. Herein, we propose a novel meta-learning approach for neural operators, which can be seen as transferring the knowledge of solution operators between governing (unknown) PDEs with varying parameter fields. Our approach is a provably universal solution operator for multiple PDE solving tasks, with a key theoretical observation that underlying parameter fields can be captured in the first layer of neural operator models, in contrast to typical final-layer transfer in existing meta-learning methods. As applications, we demonstrate the efficacy of our proposed approach on PDE-based datasets and a real-world material modeling problem, illustrating that our method can handle complex and nonlinear physical response learning tasks while greatly improving the sampling efficiency in unseen tasks.

Human tissues are highly organized structures with collagen fiber arrangements varying from point to point. Anisotropy of the tissue arises from the natural orientation of the fibers, resulting in location-dependent anisotropy. Heterogeneity also plays an important role in tissue function. It is therefore critical to discover and understand the distribution of fiber orientations from experimental mechanical measurements such as digital image correlation (DIC) data. To this end, we introduce the Heterogeneous Peridynamic Neural Operator (HeteroPNO) approach for data-driven constitutive modeling of heterogeneous anisotropic materials. Our goal is to learn a nonlocal constitutive law together with the material microstructure, in the form of a heterogeneous fiber orientation field, from load-displacement field measurements. We propose a two-phase learning approach. First, we learn a homogeneous constitutive law in the form of a neural network-based kernel function and a nonlocal bond force. This captures the complex homogeneous material responses from data. Second, we reinitialize the learnt bond force and train it together with the kernel and the fiber orientation field for each material point. Due to the inherent properties of the state-based peridynamic framework that we use, our HeteroPNO-learned material models are guaranteed to be objective and to satisfy the balances of linear and angular momenta. The effects of heterogeneity and the nonlinearity of the constitutive relationship are captured by the nonlocal kernel function, enabling physical interpretability. These features enable our HeteroPNO architecture to learn a constitutive model for biological tissue with anisotropic, heterogeneous response undergoing large deformation. To demonstrate the applicability of our approach, we apply the HeteroPNO in learning a material model and fiber orientation field from DIC measurements in biaxial testing. We evaluate the learnt fiber architecture, which is found to be consistent with observations from polarized spatial frequency domain imaging. The framework is capable of providing displacement and stress field predictions for new and unseen loading instances. [ABSTRACT FROM AUTHOR]

A digital twin (DT), with the components of a physics-based model, a data-driven model, and a machine learning (ML) enabled efficient surrogate model, behaves as a virtual twin of the real-world physical process. In terms of Laser Powder Bed Fusion (L-PBF) based additive manufacturing (AM), a DT can predict the current and future states of the melt pool and the resulting defects corresponding to the input laser parameters, evolve itself by assimilating in-situ sensor data, and optimize the laser parameters to mitigate defect formation. In this paper, we present a deep neural operator enabled DT framework for closed-loop feedback control of the L-PBF process. This is accomplished by building a physics-based computational model to accurately represent the melt pool states, an efficient Fourier neural operator (FNO) based surrogate model to approximate the melt pool solution field, followed by a physics-based procedure to extract information from the computed melt pool simulation that can further be correlated to the defect quantities of interest (e.g., surface roughness). An optimization algorithm is then exercised to control laser input and minimize defects. On the other hand, the constructed DT also evolves with the physical twin via offline finetuning and online material calibration. For instance, the probabilistic distribution of laser absorptivity can be updated to match the real-time captured thermal image data. Finally, a probabilistic framework is adopted for uncertainty quantification. The developed DT is envisioned to guide the AM process and facilitate high-quality manufacturing in L-PBF-based metal AM. [ABSTRACT FROM AUTHOR]

We present our ongoing work aimed at accelerating a particle‐resolved direct numerical simulation model designed to study aerosol–cloud–turbulence interactions. The dynamical model consists of two main components—a set of fluid dynamics equations for air velocity, temperature, and humidity, coupled with a set of equations for particle (i.e., cloud droplet) tracing. Rather than attempting to replace the original numerical solution method in its entirety with a machine learning (ML) method, we consider developing a hybrid approach. We exploit the potential of neural operator learning to yield fast and accurate surrogate models and, in this study, develop such surrogates for the velocity and vorticity fields. We discuss results from numerical experiments designed to assess the performance of ML architectures under consideration as well as their suitability for capturing the behavior of relevant dynamical systems. [ABSTRACT FROM AUTHOR]

Most traditional methods for solving partial differential equations (PDEs) require the costly solving of large linear systems. Neural operators (NOs) offer remarkable speed-ups over classical numerical PDE solvers. Here, we conduct the first exploration and comparison of NOs for three-dimensional topology optimization. Specifically, we propose replacing the PDE solver within the popular Solid Isotropic Material with Penalization (SIMP) algorithm, which is its main computational bottleneck. For this, the NO not only needs to solve the PDE with sufficient accuracy but also has the additional challenge of providing accurate gradients which are necessary for SIMP's density updates. To realize this, we do three things: (i) We introduce a novel loss term to promote gradient-consistency. (ii) We guarantee equivariance in our NOs to increase the physical correctness of predictions. (iii) We introduce a novel NO architecture called U-Net Fourier neural operator (U-Net FNO), which combines the multi-resolution properties of U-Nets with the Fourier neural operator (FNO)'s focus on local features in frequency space. In our experiments we demonstrate that the inclusion of the novel gradient loss term is necessary to obtain good results. Furthermore, enforcing group equivariance greatly improves the quality of predictions, especially on small training datasets. Finally, we show that in our experiments the U-Net FNO outperforms both a standard U-Net, as well as other FNO methods. [ABSTRACT FROM AUTHOR]

Seismic wave arrival time measurements form the basis for numerous downstream applications. State‐of‐the‐art approaches for phase picking use deep neural networks to annotate seismograms at each station independently, yet human experts annotate seismic data by examining the whole network jointly. Here, we introduce a general‐purpose network‐wide phase picking algorithm based on a recently developed machine learning paradigm called Neural Operator. Our model, called Phase Neural Operator, leverages the spatio‐temporal contextual information to pick phases simultaneously for any seismic network geometry. This results in superior performance over leading baseline algorithms by detecting many more earthquakes, picking more phase arrivals, while also greatly improving measurement accuracy. Following similar trends being seen across the domains of artificial intelligence, our approach provides but a glimpse of the potential gains from fully‐utilizing the massive seismic data sets being collected worldwide. Plain Language Summary: Earthquake monitoring often involves measuring arrival times of P‐ and S‐waves of earthquakes from continuous seismic data. With the advancement of artificial intelligence, state‐of‐the‐art phase picking methods use deep neural networks to examine seismic data from each station independently; this is in stark contrast to the way that human experts annotate seismic data, in which waveforms from the whole network containing multiple stations are examined simultaneously. With the performance gains of single‐station algorithms approaching saturation, it is clear that meaningful future advances will require algorithms that can naturally examine data for entire networks at once. Here we introduce a multi‐station phase picking algorithm based on a recently developed machine learning paradigm called Neural Operator. Our algorithm, called Phase Neural Operator, leverages the spatial‐temporal information of earthquake signals from an input seismic network with arbitrary geometry. This results in superior performance over leading baseline algorithms by detecting many more earthquakes, picking many more seismic wave arrivals, yet also greatly improving measurement accuracy. Key Points: We introduce a multi‐station phase picking algorithm, Phase Neural Operator (PhaseNO), that is based on a new machine learning paradigm called Neural OperatorPhaseNO can use data from any number of stations arranged in any arbitrary geometry to pick phases across the input stations simultaneouslyBy leveraging the spatial and temporal contextual information, PhaseNO achieves superior performance over leading baseline algorithms [ABSTRACT FROM AUTHOR]

In recent works, the authors introduced a neural operator: a special type of neural networks that can approximate maps between infinite-dimensional spaces. Using numerical and analytical techniques, we will highlight the peculiarities of the training and evaluation of these operators. In particular, we will show that, for a broad class of neural operators based on integral transforms, a systematic bias is inevitable, owning to aliasing errors. To avoid this bias, we introduce spectral neural operators based on explicit discretization of the domain and the codomain. Although discretization deteriorates the approximation properties, numerical experiments show that the accuracy of spectral neural operators is often superior to the one of neural operators defined on infinite-dimensional Banach spaces. [ABSTRACT FROM AUTHOR]

Operator learning focuses on approximating mappings G † : U → V between infinite-dimensional spaces of functions, such as u : Ω u → R and v : Ω v → R. This makes it particularly suitable for solving parametric nonlinear partial differential equations (PDEs). Recent advancements in machine learning (ML) have brought operator learning to the forefront of research. While most progress in this area has been driven by variants of deep neural networks (NNs), recent studies have demonstrated that Gaussian process (GP)/kernel-based methods can also be competitive. These methods offer advantages in terms of interpretability and provide theoretical and computational guarantees. In this article, we introduce a hybrid GP/NN-based framework for operator learning, leveraging the strengths of both deep neural networks and kernel methods. Instead of directly approximating the function-valued operator G † , we use a GP to approximate its associated real-valued bilinear form G ˜ † : U × V ∗ → R. This bilinear form is defined by the dual pairing G ˜ † (u , φ) ≔ [ φ , G † (u) ] , which allows us to recover the operator G † through G † (u) (y) = G ˜ † (u , δ y). The mean function of the GP can be set to zero or parameterized by a neural operator and for each setting we develop a robust and scalable training mechanism based on maximum likelihood estimation (MLE) that can optionally leverage the physics involved. Numerical benchmarks demonstrate our method's scope, scalability, efficiency, and robustness; showing that (1) it enhances the performance of a base neural operator by using it as the mean function of a GP, and (2) it enables the construction of zero-shot data-driven models that can make accurate predictions without any prior training. Additionally, our framework (a) naturally extends to cases where G † : U → ∏ s = 1 S V s maps into a vector of functions, and (b) benefits from computational speed-ups achieved through product kernel structures and Kronecker product matrix representations of the underlying kernel matrices. 1 1 GitHub repository: https://github.com/Bostanabad-Research-Group/GP-for-Operator-Learning. [ABSTRACT FROM AUTHOR]

Neural operator learning models have emerged as very effective surrogates in data-driven methods for partial differential equations (PDEs) across different applications from computational science and engineering. Such operator learning models not only predict particular instances of a physical or biological system in real-time but also forecast classes of solutions corresponding to a distribution of initial and boundary conditions or forcing terms. DeepONet is the first neural operator model and has been tested extensively for a broad class of solutions, including Riemann problems. Transformers have not been used in that capacity, and specifically, they have not been tested for solutions of PDEs with low regularity. In this work, we first establish the theoretical groundwork that transformers possess the universal approximation property as operator learning models. We then apply transformers to forecast solutions of diverse dynamical systems with solutions of finite regularity for a plurality of initial conditions and forcing terms. In particular, we consider three examples: the Izhikevich neuron model, the tempered fractional-order Leaky Integrate-and-Fire (LIF) model, and the one-dimensional Euler equation Riemann problem. For the latter problem, we also compare with variants of DeepONet, and we find that transformers outperform DeepONet in accuracy but they are computationally more expensive. • Transformers possess the universal approximation property as operator learning models. • Transformers exhibit excellent performance for dynamical systems with long memory and rough solutions. • Transformers learn discontinuous solution of Riemann problems accurately. [ABSTRACT FROM AUTHOR]

This paper presents a novel neural operator learning framework for designing boundary control to mitigate stop-and-go congestion on freeways. The freeway traffic dynamics are described by second-order coupled hyperbolic partial differential equations (PDEs), i.e. the Aw–Rascle–Zhang (ARZ) macroscopic traffic flow model. The proposed framework learns feedback boundary control strategies from the closed-loop PDE solution using backstepping controllers, which are widely employed for boundary stabilization of PDE systems. The PDE backstepping control design is time-consuming and requires intensive depth of expertise, since it involves constructing and solving backstepping control kernels. Existing machine learning methods for solving PDE control problems, such as physics-informed neural networks (PINNs) and reinforcement learning (RL), face the challenge of retraining when PDE system parameters and initial conditions change. To address these challenges, we present neural operator (NO) learning schemes for the ARZ traffic system that not only ensure closed-loop stability robust to parameter and initial condition variations but also accelerate boundary controller computation. The first scheme embeds NO-approximated control gain kernels within a analytical state feedback backstepping controller, while the second one directly learns a boundary control law from functional mapping between model parameters to closed-loop PDE solution. The stability guarantee of the NO-approximated control laws is obtained using Lyapunov analysis. We further propose the physics-informed neural operator (PINO) to reduce the reliance on extensive training data. The performance of the NO schemes is evaluated by simulated and real traffic data, compared with the benchmark backstepping controller, a Proportional Integral (PI) controller, and a PINN-based controller. The NO-approximated methods achieve a computational speedup of approximately 300 times with only a 1% error trade-off compared to the backstepping controller, while outperforming the other two controllers in both accuracy and computational efficiency. The robustness of the NO schemes is validated using real traffic data, and tested across various initial traffic conditions and demand scenarios. The results show that neural operators can significantly expedite and simplify the process of obtaining controllers for traffic PDE systems with great potential application for traffic management. • "Discretize then Design" control methods for freeway traffic control are inherently limited by the discretization schemes, resulting in numerical errors and loss of continuity between cells. Our proposed Neural Operator (NO) control scheme is invariant to numerical discretization. • Backstepping, a representative method of "design then discretize" for boundary control of partial differential equations (PDEs), can incur a significant computational burden due to the complexity of solving the control gain kernels for PDEs. The proposed NO scheme is designed to accelerate the computational process and bypass analytical design through data-driven training. • The proposed NO methods are designed to learn functional mappings from traffic PDE system parameters to the control gains. These mappings remain invariant regardless of changes in traffic's initial and boundary conditions or system parameters, thereby providing a more flexible and robust solution for freeway traffic control. [ABSTRACT FROM AUTHOR]

Uncertainty quantification (UQ) in scientific machine learning (SciML) becomes increasingly critical as neural networks (NNs) are being widely adopted in addressing complex problems across various scientific disciplines. Representative SciML models are physics-informed neural networks (PINNs) and neural operators (NOs). While UQ in SciML has been increasingly investigated in recent years, very few works have focused on addressing the uncertainty caused by the noisy inputs, such as spatial–temporal coordinates in PINNs and input functions in NOs. The presence of noise in the inputs of the models can pose significantly more challenges compared to noise in the outputs of the models, primarily due to the inherent nonlinearity of most SciML algorithms. As a result, UQ for noisy inputs becomes a crucial factor for reliable and trustworthy deployment of these models in applications involving physical knowledge. To this end, we introduce a Bayesian approach to quantify uncertainty arising from noisy inputs–outputs in PINNs and NOs. We show that this approach can be seamlessly integrated into PINNs and NOs, when they are employed to encode the physical information. PINNs incorporate physics by including physics-informed terms via automatic differentiation, either in the loss function or the likelihood, and often take as input the spatial–temporal coordinate. Therefore, the present method equips PINNs with the capability to address problems where the observed coordinate is subject to noise. On the other hand, pretrained NOs are also commonly employed as equation-free surrogates in solving differential equations and Bayesian inverse problems, in which they take functions as inputs. The proposed approach enables them to handle noisy measurements for both input and output functions with UQ. We present a series of numerical examples to demonstrate the consequences of ignoring the noise in the inputs and the effectiveness of our approach in addressing noisy inputs–outputs with UQ when PINNs and pretrained NOs are employed for physics-informed learning. [ABSTRACT FROM AUTHOR]

Neural operator approximations of the gain kernels in PDE backstepping has emerged as a viable method for implementing controllers in real time. With such an approach, one approximates the gain kernel, which maps the plant coefficient into the solution of a PDE, with a neural operator. It is in adaptive control that the benefit of the neural operator is realized, as the kernel PDE solution needs to be computed online, for every updated estimate of the plant coefficient. We extend the neural operator methodology from adaptive control of a hyperbolic PDE to adaptive control of a benchmark parabolic PDE (a reaction–diffusion equation with a spatially-varying and unknown reaction coefficient). We prove global stability and asymptotic regulation of the plant state for a Lyapunov design of parameter adaptation. The key technical challenge of the result is handling the 2 D nature of the gain kernels and proving that the target system with two distinct sources of perturbation terms, due to the parameter estimation error and due to the neural approximation error, is Lyapunov stable. To verify our theoretical result, we present simulations achieving calculation speedups up to 45 × relative to the traditional finite difference solvers for every timestep in the simulation trajectory. [ABSTRACT FROM AUTHOR]

The Deep Neural Operator, as proposed by Lu et al. (2021), marks a considerable advancement in solving parametric partial differential equations. This paper examines the DeepOnet model's neural network design, focusing on the effectiveness of its trunk-branch structure in operator learning tasks. Three key advantages of the trunk-branch structure are identified: the global learning strategy, the independent operation of the trunk and branch networks, and the consistent representation of solutions. These features are especially beneficial for operator learning. Building upon these findings, we have evolved the traditional DeepOnet into a more general form from a network perspective, allowing a nonlinear interfere of the branch net on the trunk net than the linear combination limited by the conventional DeepOnet. The operator model also incorporates physical information for enhanced integration. In a series of experiments tackling partial differential equations, the extended DeepOnet consistently outperforms than the traditional DeepOnet, particularly in complex problems. Notably, the extended DeepOnet model shows substantial advancements in operator learning with nonlinear parametric partial differential equations and exhibits a remarkable capacity for reducing physics loss. • An architectural analysis of the trunk-branch network. • A general extension of the physics-informed DeepOnet model. • Outperforming in solving complex problems, e.g. nonlinear PDEs. [ABSTRACT FROM AUTHOR]

Physics-informed machine learning (PIML) as a means of solving partial differential equations (PDEs) has garnered much attention in the Computational Science and Engineering (CS&E) world. This topic encompasses a broad array of methods and models aimed at solving a single or a collection of PDE problems, called multitask learning. PIML is characterized by the incorporation of physical laws into the training process of machine learning models in lieu of large data when solving PDE problems. Despite the overall success of this collection of methods, it remains incredibly difficult to analyze, benchmark, and generally compare one approach to another. Using Kolmogorov n-widths as a measure of effectiveness of approximating functions, we judiciously apply this metric in the comparison of various multitask PIML architectures. We compute lower accuracy bounds and analyze the model's learned basis functions on various PDE problems. This is the first objective metric for comparing multitask PIML architectures and helps remove uncertainty in model validation from selective sampling and overfitting. We also identify avenues of improvement for model architectures, such as the choice of activation function, which can drastically affect model generalization to "worst-case" scenarios, which is not observed when reporting task-specific errors. We also incorporate this metric into the optimization process through regularization, which improves the models' generalizability over the multitask PDE problem. • Novel scheme to compute n-widths for multitask PIML models to prevent selective sampling. • Novel regularization scheme using n-widths to improve model generalization. • Benchmarking PIML architectures on various multitask PDE problems. • Robust analysis of learnable neural network basis functions. [ABSTRACT FROM AUTHOR]

We consider the problem of sampling lattice field configurations on a lattice from the Boltzmann distribution corresponding to some action. Since such densities arise as approximationw of an underlying functional density, we frame the task as an instance of operator learning. We propose to approximate a time-dependent neural operator whose time integral provides a mapping between the functional distributions of the free and target theories. Once a particular lattice is chosen, the neural operator can be discretized to a finite-dimensional, time-dependent vector field which in turn induces a continuous normalizing flow between finite dimensional distributions over the chosen lattice. This flow can then be trained to be a diffeormorphism between the discretized free and target theories on the chosen lattice, and, by construction, can be evaluated on different discretizations of spacetime. We experimentally validate the proposal on the 2-dimensional φ ^4 -theory to explore to what extent such operator-based flow architectures generalize to lattice sizes they were not trained on, and show that pretraining on smaller lattices can lead to a speedup over training directly on the target lattice size.

Solving partial differential equations (PDEs) is the core of many fields of science and engineering. While classical approaches are often prohibitively slow, machine learning models often fail to incorporate complete system information. Over the past few years, transformers have had a significant impact on the field of Artificial Intelligence and have seen increased usage in PDE applications. However, despite their success, transformers currently lack integration with physics and reasoning. This study aims to address this issue by introducing Physics Informed Token Transformer (PITT). The purpose of PITT is to incorporate the knowledge of physics by embedding PDEs into the learning process. PITT uses an equation tokenization method to learn an analytically-driven numerical update operator. By tokenizing PDEs and embedding partial derivatives, the transformer models become aware of the underlying knowledge behind physical processes. To demonstrate this, PITT is tested on challenging 1D and 2D PDE operator learning tasks. The results show that PITT outperforms popular neural operator models and has the ability to extract physically relevant information from governing equations.

Electronic structure theory calculations offer an understanding of matter at the quantum level, complementing experimental studies in materials science and chemistry. One of the most widely used methods, density functional theory, maps a set of real interacting electrons to a set of fictitious non-interacting electrons that share the same probability density. Ensuring that the density remains the same depends on the exchange-correlation (XC) energy and, by a derivative, the XC potential. Inversions provide a method to obtain exact XC potentials from target electronic densities, in hopes of gaining insights into accuracy-boosting approximations. Neural networks provide a new avenue to perform inversions by learning the mapping from density to potential. In this work, we learn this mapping using physics-informed machine learning methods, namely physics informed neural networks and Fourier neural operators. We demonstrate the capabilities of these two methods on a dataset of one-dimensional atomic and molecular models. The capabilities of each approach are discussed in conjunction with this proof-of-concept presentation. The primary finding of our investigation is that the combination of both approaches has the greatest potential for inverting the Kohn–Sham equations at scale.

Predicting plasma evolution within a Tokamak reactor is crucial to realizing the goal of sustainable fusion. Capabilities in forecasting the spatio-temporal evolution of plasma rapidly and accurately allow us to quickly iterate over design and control strategies on current Tokamak devices and future reactors. Modelling plasma evolution using numerical solvers is often expensive, consuming many hours on supercomputers, and hence, we need alternative inexpensive surrogate models. We demonstrate accurate predictions of plasma evolution both in simulation and experimental domains using deep learning-based surrogate modelling tools, viz., Fourier neural operators (FNO). We show that FNO has a speedup of six orders of magnitude over traditional solvers in predicting the plasma dynamics simulated from magnetohydrodynamic models, while maintaining a high accuracy (Mean Squared Error in the normalised domain ${\approx}10^{-5}$ ). Our modified version of the FNO is capable of solving multi-variable Partial Differential Equations, and can capture the dependence among the different variables in a single model. FNOs can also predict plasma evolution on real-world experimental data observed by the cameras positioned within the MAST Tokamak, i.e. cameras looking across the central solenoid and the divertor in the Tokamak. We show that FNOs are able to accurately forecast the evolution of plasma and have the potential to be deployed for real-time monitoring. We also illustrate their capability in forecasting the plasma shape, the locations of interactions of the plasma with the central solenoid and the divertor for the full (available) duration of the plasma shot within MAST. The FNO offers a viable alternative for surrogate modelling as it is quick to train and infer, and requires fewer data points, while being able to do zero-shot super-resolution and getting high-fidelity solutions.

Solving parametric partial differential equations using artificial intelligence is taking the pace. It is primarily because conventional numerical solvers are computationally expensive and require significant time to converge a solution. However, physics informed deep learning as an alternate learns functional spaces directly and provides approximation reasonably fast compared to conventional numerical solvers. The Fourier transform approach directly learns the generalized functional space using deep learning among various approaches. This work proposes a novel deep Fourier neural network that employs a Fourier neural operator as a fundamental building block and employs spectral feature aggregation to extrude the extended information. The proposed model offers superior accuracy and lower relative error. We employ one and two-dimensional time-independent as well as two-dimensional time-dependent equations. We employ three benchmark datasets to evaluate our contributions, i.e., Burgers’ equation as one dimensional, Darcy Flow equation as two dimensional, and Navier-Stokes as two spatial dimensional with one temporal dimension as benchmark datasets. We further employ a case study of fluid-structure interaction used for the machine component designing process. We employ a computation fluid dynamics simulation dataset generated using the ANSYS-CFX software system to evaluate the regression of the temporal behavior of the fluid. Our proposed method achieves superior performance on all four datasets employed and shows improvements to baseline. We achieve a reduced relative error on the Burgers’ equation by approximately 30%, Darcy Flow equation by approximately 35%, and Navier-Stokes equation by approximately 20%.

Recent research to solve the parametric partial differential equations shifted the focus of conventional neural networks from finite-dimensional Euclidean space to generalized functional spaces. Neural operators learn the generalized function mapping directly, which was achieved primarily using numerical solvers for decades. However, numerical operators are computationally expensive and require enormous time to solve partial differential equations. In this work, we propose a spatio-spectral neural operator combining spectral feature learning and spatial feature learning. We formulate a novel neural network architecture to produce a state-of-the-art reproduction accuracy and a much reduced relative error over partial differential equations solutions. Fluid-structure interaction is a primary concern while designing the process of a machine component. Numerical simulations of fluid flow are a time-intensive task that attracted machine learning researchers to provide solutions to achieve it efficiently. Computational fluid dynamics has made noticeable progress and produced state-of-the-art numerical simulations during the last few decades. We propose a deep learning approach by employing a novel neural operator to deal with Computational fluid dynamics using deep learning. We perform the experiments over one and two-dimensional simulations using the Burgers’ equation, Darcy Flow, and Navier-Stokes equations as benchmarks. In addition, we considered a case study to demonstrate the transient fluid flow prediction past through an immersed body. Our proposed solution achieves superior accuracy to the current level of research on learning-based solvers and Fourier neural operators. Our proposed approach achieves the lowest relative error on the Burgers’ equation, Darcy Flow, and Navier-stokes equation. Furthermore, we achieve a superior relative mean squared error for the case study dataset under experiments.

Neural network based models have emerged as a powerful tool in multiscale modeling of materials. One promising approach is to use a neural network based model, trained using data generated from repeated solution of an expensive small scale model, as a surrogate for the small scale model in application scale simulations. Such approaches have been shown to have the potential accuracy of concurrent multiscale methods like FE 2 , but at the cost comparable to empirical methods like classical constitutive models or parameter passing. A key question is to understand how much and what kind of data is necessary to obtain an accurate surrogate. This paper examines this question for history dependent elastic–plastic behavior of an architected metamaterial modeled as a truss. We introduce an iterative approach where we use the rich arbitrary class of trajectories to train an initial model, but then iteratively update the class of trajectories with those that arise in large scale simulation and use transfer learning to update the model. We show that such an approach converges to a highly accurate surrogate, and one that is transferable. • Proposes Recurrent Neural Operator as a surrogate for multiscale modeling • Addresses history dependent elastic–plastic behavior of an architected metamaterial • Introduces an iterative approach to training neural approximations [ABSTRACT FROM AUTHOR]

Data-driven turbulence modeling is experiencing a surge in interest following algorithmic and hardware developments in the data sciences. We discuss an approach using the differentiable physics paradigm that combines known physics with machine learning to develop closure models for Burgers’ turbulence. We consider the one-dimensional Burgers system as a prototypical test problem for modeling the unresolved terms in advection-dominated turbulence problems. We train a series of models that incorporate varying degrees of physical assumptions on an a posteriori loss function to test the efficacy of models across a range of system parameters, including viscosity, time, and grid resolution. We find that constraining models with inductive biases in the form of partial differential equations that contain known physics or existing closure approaches produces highly data-efficient, accurate, and generalizable models, outperforming state-of-the-art baselines. Addition of structure in the form of physics information also brings a level of interpretability to the models, potentially offering a stepping stone to the future of closure modeling.

We present a critical analysis of physics-informed neural operators (PINOs) to solve partial differential equations (PDEs) that are ubiquitous in the study and modeling of physics phenomena using carefully curated datasets. Further, we provide a benchmarking suite which can be used to evaluate PINOs in solving such problems. We first demonstrate that our methods reproduce the accuracy and performance of other neural operators published elsewhere in the literature to learn the 1D wave equation and the 1D Burgers equation. Thereafter, we apply our PINOs to learn new types of equations, including the 2D Burgers equation in the scalar, inviscid and vector types. Finally, we show that our approach is also applicable to learn the physics of the 2D linear and nonlinear shallow water equations, which involve three coupled PDEs. We release our artificial intelligence surrogates and scientific software to produce initial data and boundary conditions to study a broad range of physically motivated scenarios. We provide the https://github.com/shawnrosofsky/PINO_Applications/tree/main , an interactive https://shawnrosofsky.github.io/PINO_Applications/ to visualize the predictions of our PINOs, and a tutorial for their use at the https://www.dlhub.org .

Solving fluid dynamics equations often requires the use of closure relations that account for missing microphysics. For example, when solving equations related to fluid dynamics for systems with a large Reynolds number, sub-grid effects become important and a turbulence closure is required, and in systems with a large Knudsen number, kinetic effects become important and a kinetic closure is required. By adding an equation governing the growth and transport of the quantity requiring the closure relation, it becomes possible to capture microphysics through the introduction of ‘hidden variables’ that are non-local in space and time. The behavior of the ‘hidden variables’ in response to the fluid conditions can be learned from a higher fidelity or ab-initio model that contains all the microphysics. In our study, a partial differential equation simulator that is end-to-end differentiable is used to train judiciously placed neural networks against ground-truth simulations. We show that this method enables an Euler equation based approach to reproduce non-linear, large Knudsen number plasma physics that can otherwise only be modeled using Boltzmann-like equation simulators such as Vlasov or particle-in-cell modeling.

Neural operators, which can act as implicit solution operators of hidden governing equations, have recently become popular tools for learning the responses of complex real-world physical systems. Nevertheless, most neural operator applications have thus far been data-driven and neglect the intrinsic preservation of fundamental physical laws in data. In this work, we introduce a novel integral neural operator architecture called the Peridynamic Neural Operator (PNO) that learns a nonlocal constitutive law from data. This neural operator provides a forward model in the form of state-based peridynamics, with objectivity and momentum balance laws automatically guaranteed. As applications, we demonstrate the expressivity and efficacy of our model in learning complex material behaviors from both synthetic and experimental data sets. We also compare the performances with baseline models that use predefined constitutive laws. We show that, owing to its ability to capture complex responses, our learned neural operator achieves improved accuracy and efficiency. Moreover, by preserving the essential physical laws within the neural network architecture, the PNO is robust in treating noisy data. The method shows generalizability to different domain configurations, external loadings, and discretizations. • We proposed PNO, which learns a nonlocal constitutive law from spatial measurements. • It captures complex material responses without prior expert-constructed knowledge. • Meanwhile, the model guarantees the physically required balance laws and objectivity. • Learnt model is generalizable to various resolutions, loading, and domain settings. [ABSTRACT FROM AUTHOR]

Deep neural operators, such as DeepONet, have changed the paradigm in high-dimensional nonlinear regression, paving the way for significant generalization and speed-up in computational engineering applications. Here, we investigate the use of DeepONet to infer flow fields around unseen airfoils with the aim of shape constrained optimization, an important design problem in aerodynamics that typically taxes computational resources heavily. We present results that display little to no degradation in prediction accuracy while reducing the online optimization cost by orders of magnitude. We consider NACA airfoils as a test case for our proposed approach, as the four-digit parameterization can easily define their shape. We successfully optimize the constrained NACA four-digit problem with respect to maximizing the lift-to-drag ratio and validate all results by comparing them to a high-order CFD solver. We find that DeepONets have a low generalization error, making them ideal for generating solutions of unseen shapes. Specifically, pressure, density, and velocity fields are accurately inferred at a fraction of a second, hence enabling the use of general objective functions beyond the maximization of the lift-to-drag ratio considered in the current work. Finally, we validate the ability of DeepONet to handle a complex 3D waverider geometry at hypersonic flight by inferring shear stress and heat flux distributions on its surface at unseen angles of attack. The main contribution of this paper is a modular integrated design framework that uses an over-parametrized neural operator as a surrogate model with good generalizability coupled seamlessly with multiple optimization solvers in a plug-and-play mode. • Generating an efficient and inexpensive surrogate model based on DeepONets. • The surrogate model is invariant to the low or high-dimensional parameterizations. • Prediction of flow field is used for various cost functions in the optimization loop. • Computing drag to lift ratio using the inferred high-dimensional flow field. • Integration of the Dakota optimization framework with the DeepONet surrogate. [ABSTRACT FROM AUTHOR]

Numerical solutions to partial differential equations (PDEs) are instrumental for material structural design where extensive data screening is needed. However, traditional numerical methods demand significant computational resources, highlighting the need for innovative optimization algorithms to streamline design exploration. Direct gradient-based optimization algorithms, while effective, rely on design initialization and require complex, problem-specific sensitivity derivations. The advent of machine learning offers a promising alternative to handling large parameter spaces. To further mitigate data dependency, researchers have developed physics-informed neural networks (PINNs) to learn directly from PDEs. However, the intrinsic continuity requirement of PINNs restricts their application in structural mechanics problems, especially for composite materials. Our work addresses this discontinuity issue by substituting the PDE residual with a weak formulation in the physics-informed training process. The proposed approach is exemplified in modeling digital materials, which are mathematical representations of complex composites that possess extreme structural discontinuity. This article also introduces an interactive process that integrates physics-informed loss with design objectives, eliminating the need for pretrained surrogate models or analytical sensitivity derivations. The results demonstrate that our approach can preserve the physical accuracy in data-free material surrogate modeling but also accelerates the direct optimization process without model pretraining., (© The Author(s) 2024. Published by Oxford University Press on behalf of National Academy of Sciences.)

Numerical simulations are computationally demanding in three-dimensional (3D) settings but they are often required to accurately represent physical phenomena. Neural operators have emerged as powerful surrogate models to alleviate the computational costs of simulations. However, neural operators applications in 3D remain sparse, mainly due to the difficulty of obtaining training databases for supervised learning and the size of 3D neural operators that poses memory challenges. This work focuses on the propagation of elastic waves in 3D domains and showcases the Factorized Fourier Neural Operator (F-FNO) as an efficient and accurate surrogate model. The F-FNO is trained on the publicly available HEMEW-3D database of 30 000 wavefields simulations in realistic heterogeneous domains. The F-FNO predicts space- and time-dependent (3D) surface wavefields depending on the characteristics of the propagation domain (characterized by the velocity of shear waves). Four FNO variants are compared and extensive investigations on the influence of hyperparameters and training strategies are conducted. The two most influential hyperparameters are the number of layers and the number of channels, meaning that richer models are more accurate. On the contrary, increasing the number of Fourier modes had little influence and did not reduce the spectral bias that causes an underestimation of high-frequency patterns. The F-FNO is sensitive to heterogeneities in the inputs but robust to the addition of noise. Additionally, it possesses good generalization ability to out-of-distribution data and transfer learning is very beneficial to improve the predictions in tailored applications. [Display omitted] • Fourier Neural Operators accurately predict elastic waves in heterogeneous domains. • Memory-saving 3D predictions are achieved through a depth-to-time conversion. • The F-FNO achieves a relative RMSE of 17% on the validation database. • Generalization to out-of-distribution data is robust and satisfyingly accurate. [ABSTRACT FROM AUTHOR]

The Eikonal equation plays a central role in seismic wave propagation and hypocenter localization, a crucial aspect of efficient earthquake early warning systems. Despite recent progress, real-time earthquake localization remains challenging due to the need to learn a generalizable Eikonal operator. We introduce a novel deep learning architecture, Enriched-DeepONet (En-DeepONet), addressing the limitations of current operator learning models in dealing with moving-solution operators. Leveraging addition and subtraction operations and a novel 'root' network, En-DeepONet is particularly suitable for learning such operators and achieves up to four orders of magnitude improved accuracy without increased training cost. We demonstrate the effectiveness of En-DeepONet in earthquake localization under variable velocity and arrival time conditions. Our results indicate that En-DeepONet paves the way for real-time hypocenter localization for velocity models of practical interest. The proposed method represents a significant advancement in operator learning that is applicable to a gamut of scientific problems, including those in seismology, fracture mechanics, and phase-field problems. [Display omitted] • Introducing enriched DeepONet architecture. • Exploring its application to instant hypocenter localization. • Considering highly heterogeneous fields. • Considering sensitivity to input noise. [ABSTRACT FROM AUTHOR]

We present an approach for solving optimal boundary control problems of nonlinear optics by using deep learning. For computing high resolution approximations of the solution to the nonlinear wave model, we propose higher order space-time finite element methods in combination with collocation techniques. Thereby, C l -regularity in time of the global discrete solution is ensured. The simulation data is used to train solution operators that effectively leverage the higher regularity of the data. The solution operator is represented by Fourier Neural Operators and can be used as the forward solver in the optimal Dirichlet boundary control problem. The proposed algorithm is implemented and tested on high-performance computing platforms, with a focus on efficiency and scalability. The effectiveness of the approach is demonstrated on the problem of generating Terahertz radiation in periodically poled Lithium Niobate. The neural network is used as the solver in the optimal control setting to optimize the parametrization of the optical input pulse and maximize the yield of 0.3 THz-frequency radiation. We exploit the periodic layering of the crystal to design the neural networks. The networks are trained to learn the propagation through one period of the layers. The recursive application of the network onto itself yields an approximation to the full problem. Our results indicate that the proposed method can achieve a speedup by a factor of 360 compared to classical methods. A comparison of our results to experimental data shows the potential to revolutionize the way we approach optimization problems in nonlinear optics. • Solving optimal control problems in nonlinear optics using deep learning and higher-order space-time finite element methods. • Fourier Neural Operators as efficient forward solvers in optimal control problems. • Efficient and scalable implementation on high-performance computing platforms. • Solution of a problem arising in nonlinear optics, showing potential for revolutionizing optimization in nonlinear optics. [ABSTRACT FROM AUTHOR]

We utilize neural operators to learn the solution propagator for challenging systems of differential equations that are representative of stiff chemical kinetics. Specifically, we apply the deep operator network (DeepONet) along with its extensions, such as the autoencoder-based DeepONet and the newly proposed Partition-of-Unity (PoU-) DeepONet to study a range of examples, including the ROBERS problem with three species, the POLLU problem with 25 species, pure kinetics of a skeletal model for CO/H 2 burning of syngas, which contains 11 species and 21 reactions and finally, a temporally developing planar CO/H 2 jet flame (turbulent flame) using the same syngas mechanism. We have demonstrated the advantages of the proposed approach through these numerical examples. Specifically, to train the DeepONet for the syngas model, we solve the skeletal kinetic model for different initial conditions. In the first case, we parameterize the initial conditions based on equivalence ratios and initial temperature values. In the second case, we perform a direct numerical simulation of a two-dimensional temporally developing CO/H 2 jet flame with integrated chemistry modeling as a source of training data for the solution propagator. Then, we initialize the kinetic model by the thermochemical states visited by a subset of grid points at different time snapshots. Stiff chemical kinetics are computationally expensive to solve, thus, this work aims to develop a neural operator-based surrogate model to efficiently solve stiff chemical kinetics. The operator, once trained offline, can accurately integrate the thermochemical state for arbitrarily large time advancements, leading to significant computational gains compared to stiff integration schemes. • We employed Deep Operator Networks and its variant, to solve stiff chemical kinetics problems. • Pure kinetics of a skeletal model is solved for the CO/H 2 burning of syngas. • DeepONet can learn the solution propagator with large time steps (∼ 1000 dt) very accurately. [ABSTRACT FROM AUTHOR]